Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarification Regarding "All Patch Representations" in the Pre-training Diagram #1689

Open
liruixinxinxin opened this issue Feb 10, 2025 · 1 comment

Comments

@liruixinxinxin
Copy link

I hope this email finds you well. I am currently working on understanding your work regarding the pre-training procedure for masked audio prediction, and I have encountered a question related to the "All Patch Representations" in the figure of your paper.
Specifically, I am referring to the notation {M} in the "Label Predictor" block. I would appreciate it if you could clarify the following points:
If {M} refers to directly using the masked feature as it is, I am concerned that the dimensions may not align properly.
Alternatively, if I follow the description provided in the paper, should the {M} be all zeros as part of the masking process? I am unsure if my understanding of this aspect is correct, and I would greatly appreciate your confirmation.

Thank you for your time, and I look forward to your response.

@liruixinxinxin
Copy link
Author

Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant