Skip to content

Commit

Permalink
[Hardware][AWS] update neuron to 2.20 (vllm-project#8676)
Browse files Browse the repository at this point in the history
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Signed-off-by: Alvant <alvasian@yandex.ru>
  • Loading branch information
omrishiv authored and Alvant committed Oct 26, 2024
1 parent 2cfca29 commit c4f7847
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 4 deletions.
4 changes: 2 additions & 2 deletions Dockerfile.neuron
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# default base image
ARG BASE_IMAGE="public.ecr.aws/neuron/pytorch-inference-neuronx:2.1.2-neuronx-py310-sdk2.19.1-ubuntu20.04"
ARG BASE_IMAGE="public.ecr.aws/neuron/pytorch-inference-neuronx:2.1.2-neuronx-py310-sdk2.20.0-ubuntu20.04"

FROM $BASE_IMAGE

Expand All @@ -20,7 +20,7 @@ RUN python3 -m pip install --upgrade pip
RUN python3 -m pip install --no-cache-dir fastapi ninja tokenizers pandas
RUN python3 -m pip install sentencepiece transformers==4.36.2 -U
RUN python3 -m pip install transformers-neuronx --extra-index-url=https://pip.repos.neuron.amazonaws.com -U
RUN python3 -m pip install --pre neuronx-cc==2.12.* --extra-index-url=https://pip.repos.neuron.amazonaws.com -U
RUN python3 -m pip install --pre neuronx-cc==2.15.* --extra-index-url=https://pip.repos.neuron.amazonaws.com -U

COPY ./vllm /app/vllm/vllm
COPY ./setup.py /app/vllm/setup.py
Expand Down
4 changes: 2 additions & 2 deletions requirements-neuron.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@
-r requirements-common.txt

# Dependencies for Neuron devices
transformers-neuronx >= 0.9.0
torch-neuronx >= 2.1.0
transformers-neuronx >= 0.12.0
torch-neuronx >= 2.1.2
neuronx-cc

0 comments on commit c4f7847

Please sign in to comment.