Skip to content
This repository was archived by the owner on Oct 11, 2024. It is now read-only.

.gitignore magic_wand; updated install instructions to include make -… #2

Closed
wants to merge 1 commit into from

Conversation

afeldman-nm
Copy link

Fixes to make the neuralmagic-vllm install process smoother:

  • Added README instruction to build the magic_wand .so (unless we plan to roll make -C build/ into magic_wand setup.py)
  • .gitignore magic_wand
  • reminder to login to huggingface

@afeldman-nm
Copy link
Author

Oh also - added advice to use python>=3.10; vllm docs recommend installing 3.9 via conda but this breaks magic_wand

@mgoin
Copy link
Member

mgoin commented Feb 2, 2024

Oh also - added advice to use python>=3.10; vllm docs recommend installing 3.9 via conda but this breaks magic_wand

why does 3.9 not work with the lib?

@afeldman-nm
Copy link
Author

Oh also - added advice to use python>=3.10; vllm docs recommend installing 3.9 via conda but this breaks magic_wand

why does 3.9 not work with the lib?

3.9 works fine with neuralmagic-vllm. However, neuralmagic-vllm takes a dependency on magic_wand. @LucasWilkinson In magic_wand one or more of the sparsity related classes call "dataclasses(slot=True)"; according to

https://www.trueblade.com/blogs/news/python-3-10-new-dataclass-features

the "slot" constructor argument is new in python 3.10.

I tried installing neuralmagic-vllm twice, with Python versions 3.9 and 3.10 respectively, and confirmed that under 3.9 the "slot" dataclasses argument is not recognized.

@afeldman-nm
Copy link
Author

Oh also - added advice to use python>=3.10; vllm docs recommend installing 3.9 via conda but this breaks magic_wand

why does 3.9 not work with the lib?

3.9 works fine with neuralmagic-vllm. However, neuralmagic-vllm takes a dependency on magic_wand. @LucasWilkinson In magic_wand one or more of the sparsity related classes call "dataclasses(slot=True)"; according to

https://www.trueblade.com/blogs/news/python-3-10-new-dataclass-features

the "slot" constructor argument is new in python 3.10.

I tried installing neuralmagic-vllm twice, with Python versions 3.9 and 3.10 respectively, and confirmed that under 3.9 the "slot" dataclasses argument is not recognized.

@mgoin

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants