Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added error when sequence length is bigger than max_position_embeddings #32156

Merged
merged 19 commits into from
Jan 10, 2025

Conversation

Taha1506
Copy link
Contributor

@Taha1506 Taha1506 commented Jul 23, 2024

Fixes # 32154

@Taha1506
Copy link
Contributor Author

Taha1506 commented Jul 23, 2024

Hi, It would be nice If you give some time to reviewing this pull request. @ArthurZucker . It relates to this issue.

Comment on lines +220 to +222
max_position_embedding = self.position_embedding.weight.shape[0]

if seq_length > max_position_embedding:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • No need to create a new variable if it's only used once
  • We should instead store a variable from the config to avoid expensive checks on the weight size for every forward pass
Suggested change
max_position_embedding = self.position_embedding.weight.shape[0]
if seq_length > max_position_embedding:
if seq_length > self.max_position_embeddings:

And in the init

    def __init__(self, config: CLIPTextConfig):
        super().__init__()
        embed_dim = config.hidden_size
        self.max_position_embeddings = config.max_position_embeddings

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realised I asked for this change -- coming back to it, I think that we shouldn't set self.max_position_embeddings as the user can change the embeddings with set_input_embeddings --> the value from the config will become out of date. In this case, checking the shapes as before is less brittle, could you change it back to this? Apologies for flip-flopping here.

@Taha1506 Taha1506 requested a review from amyeroberts July 24, 2024 17:18
@Taha1506
Copy link
Contributor Author

Hi! Can you take a look at this pull request again please? Thank you. @amyeroberts

Copy link
Collaborator

@ArthurZucker ArthurZucker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I checked that all config have this attribute

@ArthurZucker
Copy link
Collaborator

This is gonna break remote code, as they won'strore the self.max_position embedding

@ArthurZucker
Copy link
Collaborator

Fine for me @amyeroberts feel free to merge

@Taha1506
Copy link
Contributor Author

@amyeroberts
Sorry for the delay, I've made the requested changes.

@Rocketknight1
Copy link
Member

LGTM and we have reviews already. Thanks for the helpful error message!

@Rocketknight1 Rocketknight1 merged commit 0aaf124 into huggingface:main Jan 10, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants