[BetterTransformer] Add MobileBERT support for BetterTransformer

[raghavanone]

What does this PR do?

Adds mobilebertlayer

[younesbelkada]

Thanks a lot for your great work @raghavanone and for start adding BetterTransformer support for MobileBert!
I left few comments, several points needs to be addressed before merging. Starting from the naming of the new class. We prefer to call the new class as xxxLayerBetterTransformer to be able to correctly trace whether the conversion has been successfull or not.
Also could you make sure this implementation is tested by adding hf-internal-testing/tiny-random-MobileBertModel here
Let us know if you face into any issue here!

[younesbelkada]

hey @raghavanone !
do you need any help converting the model? It seems that you did forget to add hf-internal-testing/tiny-random-MobileBertModel in the testing suite?

[raghavanone]

hey @raghavanone ! do you need any help converting the model? It seems that you did forget to add hf-internal-testing/tiny-random-MobileBertModel in the testing suite?

Done

[michaelbenayoun]

What is the purpose of *_ in the end of the arguments?

[raghavanone]

I was not sure, I saw *_ in every other forward call definition.

[michaelbenayoun]

Alright, maybe we should also add **__ or something to handle potential keyword arguments?
@younesbelkada

[raghavanone]

I saw both some forward def having both *_ and **_ . We should possibly add more information about it in the guide published.

[michaelbenayoun]

Yes basically it is to "handle" potential arguments that were passed to the original forward function, by allowing this forward function to get them, but ignoring them right after.

[younesbelkada]

great remark, the guide will be updated ;)

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

[younesbelkada]

The test is failing for few reasons, firstly MobileBERT uses a special mechanism called bottleneck_attention and botteneck, if these are enabled it seems that the conversion criteras are not met. Therefore I made a special model here: ybelkada/tiny-mobilebertmodel that you can replace on the testing suite and hopefully the test will pass!

also, it would be cool to add a sanity check on the __init__ of the MobileBertBetterTransformerLayer to check if these values are correctly set on the config file.

[raghavanone]

@younesbelkada let me know what needs to be done to merge this .

[michaelbenayoun]

There are conflicts, you need to fetch on the main branch and merge / rebase on this branch.