[Installation]: GitHub access required during install for vllm >=0.4.1 (for cu12-libnccl.so.2.18.1)

[mattmalcher]

Your current environment

Current environment has no internet access. Using a local pypi mirror to install packages. Using the CUDA version of vllm on an Nvidia A10.

How you are installing vllm

Using a local pypi mirror to install packages in an environment without internet access.

The following works:

pip install -vvv vllm==0.4.0.post1

The following does not work:

pip install -vvv vllm==0.4.1

  Running setup.py (path:/tmp/pip-install-wot5g_wr/vllm-nccl-cu12_7614e24f467d4cef9205802e1b0b89cb/setup.py) egg_info for package vllm-nccl-cu12
  Created temporary directory: /tmp/pip-pip-egg-info-lwr60hk0
  Running command python setup.py egg_info
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/tmp/pip-install-wot5g_wr/vllm-nccl-cu12_7614e24f467d4cef9205802e1b0b89cb/setup.py", line 83, in <module>
      if get_md5_hash(destination) != file_hash:
         ^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-install-wot5g_wr/vllm-nccl-cu12_7614e24f467d4cef9205802e1b0b89cb/setup.py", line 43, in get_md5_hash
      with open(file_path, "rb") as f:  # Open file in binary read mode
           ^^^^^^^^^^^^^^^^^^^^^
  FileNotFoundError: [Errno 2] No such file or directory: '<my home dir>/.config/vllm/nccl/cu12/libnccl.so.2.18.1'
  Downloading nccl package from https://github.com/vllm-project/vllm-nccl/releases/download/v0.1.0/cu12-libnccl.so.2.18.1
  Failed to download nccl package from https://github.com/vllm-project/vllm-nccl/releases/download/v0.1.0/cu12-libnccl.so.2.18.1
  <urlopen error [Errno 110] Connection timed out>

The install process errors out when running the setup.py, which is expected in this scenario as the environment does not have internet access to be able to pull a file from github.

From looking at the diff vs version 0.4.0.post1 I can see that 0.4.1 has a new requirements-cuda.txt which adds vllm-nccl-cu12>=2.18,<2.19

I think the code erroring out is vllm-nccl setup.py which is attempting to grab stuff from github.

From my point of view it would be amazing if there was a way to install vllm using only files from pypi (or a pypi mirror).

I think I might be able to work around this by fetching the file outside of the installation process and setting the VLLM_NCCL_SO_PATH env var to its location, but I don't think the vllm-nccl setup.py knows to check this, so would still fail on install unless I am able to put the .so file in the user's home directory ahead of time, or find a way to skip attempting to install vllm-nccl.

[Jarvis-K]

You can download it from https://github.com/vllm-project/vllm-nccl/releases/tag/v0.1.0. Then place it in '/home/username/.config/vllm/nccl/cu12' and rename it as "libnccl.so.2.18.1" .

[youkaichao]

Unless either NVIDIA/nccl#1234 or pypi/support#3792 is resolved, we have no choice but to bring libnccl.so this way. Sorry for the trouble. This is not what I want, either.

[mattmalcher]

Fair enough, thanks guys 🙏 fingers crossed one of those comes through!

[yorickvP]

Out of curiosity, why not depend on nvidia-nccl-cu12==2.18.1?

[youkaichao]

Out of curiosity, why not depend on nvidia-nccl-cu12==2.18.1?

Because pytorch already requires nvidia-nccl-cu12==2.19 .

[FangxuLiu]

Unless either NVIDIA/nccl#1234 or pypi/support#3792 is resolved, we have no choice but to bring libnccl.so this way. Sorry for the trouble. This is not what I want, either.

@youkaichao Is this problem solved?

[youkaichao]

yes, it is not required now after #5091 , starting in v0.4.3.