Skip to content

Commit d55c43c

Browse files
esmeetujimpang
authored and
jimpang
committed
Don't use cupy when enforce_eager=True (vllm-project#3037)
1 parent 88ffe97 commit d55c43c

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

vllm/engine/llm_engine.py

+4-1
Original file line numberDiff line numberDiff line change
@@ -284,7 +284,10 @@ def _init_workers_ray(self, placement_group: "PlacementGroup",
284284
is_driver_worker=True,
285285
)
286286

287-
self._run_workers("init_model", cupy_port=get_open_port())
287+
# don't use cupy for eager mode
288+
self._run_workers("init_model",
289+
cupy_port=get_open_port()
290+
if not model_config.enforce_eager else None)
288291
self._run_workers(
289292
"load_model",
290293
max_concurrent_workers=self.parallel_config.

0 commit comments

Comments
 (0)