Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docker] Install ninja in lmi container #2762

Merged
merged 1 commit into from
Mar 13, 2025
Merged

Conversation

xyang16
Copy link
Contributor

@xyang16 xyang16 commented Mar 13, 2025

Description

Brief description of what this PR is about

Install ninja to fix the error below when running name tool calling.

INFO  PyProcess W-194-8d7f663aaab4e5d-stdout: Traceback (most recent call last):
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:   File "/tmp/.djl.ai/python/0.33.0-SNAPSHOT/djl_python/rolling_batch/rolling_batch.py", line 47, in try_catch_handling
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:     return func(self, *args, **kwargs)
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:   File "/tmp/.djl.ai/python/0.33.0-SNAPSHOT/djl_python/rolling_batch/vllm_rolling_batch.py", line 185, in inference
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:     self.engine.add_request(request_id=request_id,
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:   File "/opt/djl/vllm_venv/lib/python3.11/site-packages/vllm/utils.py", line 1074, in inner
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:     return fn(*args, **kwargs)
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:            ^^^^^^^^^^^^^^^^^^^
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:   File "/opt/djl/vllm_venv/lib/python3.11/site-packages/vllm/engine/llm_engine.py", line 757, in add_request
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:     self._add_processed_request(
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:   File "/opt/djl/vllm_venv/lib/python3.11/site-packages/vllm/engine/llm_engine.py", line 593, in _add_processed_request
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:     seq_group = self._create_sequence_group_with_sampling(
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:   File "/opt/djl/vllm_venv/lib/python3.11/site-packages/vllm/engine/llm_engine.py", line 810, in _create_sequence_group_with_sampling
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:     sampling_params = self._build_logits_processors(
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:   File "/opt/djl/vllm_venv/lib/python3.11/site-packages/vllm/engine/llm_engine.py", line 1988, in _build_logits_processors
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:     processor = get_local_guided_decoding_logits_processor(
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:   File "/opt/djl/vllm_venv/lib/python3.11/site-packages/vllm/model_executor/guided_decoding/__init__.py", line 132, in get_local_guided_decoding_logits_processor
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:     from vllm.model_executor.guided_decoding.xgrammar_decoding import (  # noqa
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:   File "/opt/djl/vllm_venv/lib/python3.11/site-packages/vllm/model_executor/guided_decoding/xgrammar_decoding.py", line 13, in <module>
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:     import xgrammar as xgr
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:   File "/opt/djl/vllm_venv/lib/python3.11/site-packages/xgrammar/__init__.py", line 1, in <module>
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:     from . import testing
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:   File "/opt/djl/vllm_venv/lib/python3.11/site-packages/xgrammar/testing.py", line 11, in <module>
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:     from .matcher import GrammarMatcher, bitmask_dtype
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:   File "/opt/djl/vllm_venv/lib/python3.11/site-packages/xgrammar/matcher.py", line 13, in <module>
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:     from .kernels import apply_token_bitmask_inplace_kernels
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:   File "/opt/djl/vllm_venv/lib/python3.11/site-packages/xgrammar/kernels/__init__.py", line 12, in <module>
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:     from .apply_token_bitmask_inplace_cuda import apply_token_bitmask_inplace_cuda
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:   File "/opt/djl/vllm_venv/lib/python3.11/site-packages/xgrammar/kernels/apply_token_bitmask_inplace_cuda.py", line 54, in <module>
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:     _load_torch_ops()
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:   File "/opt/djl/vllm_venv/lib/python3.11/site-packages/xgrammar/kernels/apply_token_bitmask_inplace_cuda.py", line 42, in _load_torch_ops
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:     torch.utils.cpp_extension.load_inline(
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:   File "/usr/local/lib/python3.11/dist-packages/torch/utils/cpp_extension.py", line 1646, in load_inline
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:     return _jit_compile(
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:            ^^^^^^^^^^^^^
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:   File "/usr/local/lib/python3.11/dist-packages/torch/utils/cpp_extension.py", line 1721, in _jit_compile
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:     _write_ninja_file_and_build_library(
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:   File "/usr/local/lib/python3.11/dist-packages/torch/utils/cpp_extension.py", line 1803, in _write_ninja_file_and_build_library
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:     verify_ninja_availability()
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:   File "/usr/local/lib/python3.11/dist-packages/torch/utils/cpp_extension.py", line 1852, in verify_ninja_availability
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout:     raise RuntimeError("Ninja is required to load C++ extensions")
INFO  PyProcess W-194-8d7f663aaab4e5d-stdout: RuntimeError: Ninja is required to load C++ extensions

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Checklist:

  • Please add the link of Integration Tests Executor run with related tests.
  • Have you manually built the docker image and verify the change?
  • Have you run related tests? Check how to set up the test environment here; One example would be pytest tests.py -k "TestCorrectnessLmiDist" -m "lmi_dist"
  • Have you added tests that prove your fix is effective or that this feature works?
  • Has code been commented, particularly in hard-to-understand areas?
  • Have you made corresponding changes to the documentation?

Feature/Issue validation/testing

Please describe the Unit or Integration tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

  • Test A
    Logs for Test A

  • Test B
    Logs for Test B

@xyang16 xyang16 requested review from zachgk and a team as code owners March 13, 2025 02:15
@xyang16 xyang16 merged commit 53df6e8 into deepjavalibrary:master Mar 13, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants