-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Getting an error when trying to perform SFT on Tiny Llama #8
Comments
It is due to the version of We conjecture that it may be because some weights of LlamaForCausalLM were not initialized from the model checkpoint. Here is our solution:
|
Thanks for reminding us of this issue and we will add this to README.md. |
original_tinyllama_path - is this the path in our system where we have downloaded tiny llama? |
Yes. |
We are getting this error when trying to execute SFT on TinyLlama:
[rank0]: if f.read(7) == "version": [rank0]: File "/usr/lib/python3.10/codecs.py", line 322, in decode [rank0]: (result, consumed) = self._buffer_decode(data, self.errors, final) [rank0]: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc1 in position 70: invalid start byte
However, we found no issue executing llama and mistral. If you have any idea regarding this issue, please let us know
The text was updated successfully, but these errors were encountered: