[several models] improve readability

[stas00]

Honestly I had no idea what torch.ones([]) * self.config.logit_scale_init_value would return - it's not documented either.

Proposing to change it to a very clear torch.tensor(1.0) which leaves no doubt to what it does.

Proposing to change it to a very clear torch.tensor(self.config.logit_scale_init_value) which leaves no doubt to what it does.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[ydshieh]

size ([int] – a sequence of integers defining the shape of the output tensor.

It's actually mentioned, but I agree it's a bit less explicit than torch.tensor(1).

[ydshieh]

Let's change all such occurrences? Most of them are in clip-family models.

[stas00]

Totally!

[stas00]

done

[sgugger]

Thanks

[stas00]

Gentlemen, sleeping more on it, actually it just came to me that the cleanest most readable solution is just:

self.logit_scale = nn.Parameter(torch.tensor(self.config.logit_scale_init_value))

torch.ones isn't even needed ;)

Do you agree?

and yes, I can then fix other places.

[stas00]

ok, I have used:

self.logit_scale = nn.Parameter(torch.tensor(self.config.logit_scale_init_value))

pattern everywhere I found torch.ones([]), plus in a few places I used from_numpy where the input was numpy, so I touched on other models as well.

Please have a look.

[sgugger]

Thanks a lot!

[stas00]

from_numpy didn't quite work in 2 models where I tried it, so going to use the normal torch.tensor constructor like everywhere else in this PR.

[ydshieh]

Nice finding :-)