Fix example int8_inference_huggingface.py

[alexrs]

What

When running the example int8_inference_huggingface.py I was getting the error:

ValueError: Tokenizer class LLaMATokenizer does not exist or is not currently imported

when using the latest transformers=4.29.2. According to this issue:

the tokenizer in the config on the hub points to LLaMATokenizer. However, the tokenizer in the library is LlamaTokenizer.
This is likely due to the configuration files being created before the final PR was merged in.

(See comment)

Fix

Use LlamaForCausalLM and LlamaTokenizer instead of Auto... classes works as expected.

Test plan

• Run python examples/int8_inference_huggingface.py
• Obtained the following output:

Hamburg is in which country?
The capital of Germany is Berlin.
The capital of Germany is Berlin.
The capital of Germany is Berlin.
The capital of Germany is Berlin.
The capital of Germany is Berlin.
The capital of Germany is Berlin.
The capital of Germany is Berlin.
The capital of Germany is Berlin.
The capital of Germany is Berlin.
The capital of Germany is Berlin.
The capital of Germany is Berlin.
The capital of Germany is Berlin.
The capital of Germany is Berlin.
The capital of Germany is Berlin.
The capital of Germany is Berlin.

[USBhost]

decapoda-research is extremely out of date. Use a newer convert.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

[TimDettmers]

I think we should change the example model used instead of fixed the issue with the tokenizers. @younesbelkada any recommendation which model we should use?

[younesbelkada]

Thanks for fixing ! I left one comment

[younesbelkada]

Thanks a lot for your contribution !

[Titus-von-Koeller]

Thanks a lot @alexrs for your work and @younesbelkada for your work.

Sorry that this took a while to merge :)