You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I used the specific commit on HF to load weights to cpu only on rank0 and tried to build bnb from source from the fork version and from the official gh repo (which I saw the commit was already merged to)
also tried to load with the latest docker image of axolotl but I a problem with dtypes (ValueError: Cannot flatten integer dtype tensors)
I suspected its something related to bnb or peft but can't make it work :\
would really appreciate the help !
thanks !
Gal
File "/root/miniconda3/envs/py3.11/lib/python3.11/site-packages/torch/distributed/fsdp/_flat_param.py", line 768, in _validate_tensors_to_flatten
[rank0]: raise ValueError("Cannot flatten integer dtype tensors")
[rank0]: ValueError: Cannot flatten integer dtype tensors
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
👋
Tried to follow @winglian guide https://medium.com/@winglian/qlora-finetuning-llama-3-1-405b-with-axolotl-with-256gb-system-ram-c2474a3d3fa5
I used the specific commit on HF to load weights to cpu only on rank0 and tried to build bnb from source from the fork version and from the official gh repo (which I saw the commit was already merged to)
also tried to load with the latest docker image of axolotl but I a problem with dtypes (ValueError: Cannot flatten integer dtype tensors)
I suspected its something related to bnb or peft but can't make it work :\
would really appreciate the help !
thanks !
Gal
Beta Was this translation helpful? Give feedback.
All reactions