Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for prompt adapters #21

Merged
merged 2 commits into from
Jul 11, 2024
Merged

add support for prompt adapters #21

merged 2 commits into from
Jul 11, 2024

Conversation

prashantgupta24
Copy link
Contributor

@prashantgupta24 prashantgupta24 commented Jun 27, 2024

Depends on upstream PR - vllm-project/vllm#4645

@prashantgupta24 prashantgupta24 force-pushed the ibm-release-prep branch 2 times, most recently from 30e6edd to 021112d Compare July 3, 2024 12:08
@dtrifiro
Copy link
Contributor

dtrifiro commented Jul 5, 2024

You can now test this by setting VLLM_VERSION_OVERRIDE=git+https://github.com/SwapnilDreams100/vllm@main and running the nox session or by adding the version in the test workflow matrix:

https://github.com/opendatahub-io/vllm-tgis-adapter/blob/main/.github/workflows/tests.yaml#L33-L37

        vllm_version:
          # - "" # skip the pypi version as it will not work on CPU
          - "git+https://github.com/vllm-project/vllm@v0.5.0.post1"
          - "git+https://github.com/vllm-project/vllm@main"
          - "git+https://github.com/opendatahub-io/vllm@main" 
+         - "git+https://github.com/SwapnilDreams100/vllm@main"

@dtrifiro dtrifiro changed the title ibm-release-prep add support for prompt adapters Jul 10, 2024
@dtrifiro
Copy link
Contributor

dtrifiro commented Jul 10, 2024

@prashantgupta24 with the last push:

  • bumped vllm minimum version to 0.5.1
  • gotten rid of some flags for compatibility pre-0.5.1 compatibility
  • updated __main__ for the new prompt_adapters argument
  • minor change in the adapter store initial setup
  • maintain compatibility with 0.5.1

@codecov-commenter
Copy link

codecov-commenter commented Jul 10, 2024

Codecov Report

Attention: Patch coverage is 63.63636% with 12 lines in your changes missing coverage. Please review.

Project coverage is 63.58%. Comparing base (9cfbf95) to head (b232f9e).

Files Patch % Lines
src/vllm_tgis_adapter/grpc/adapters.py 38.46% 7 Missing and 1 partial ⚠️
src/vllm_tgis_adapter/grpc/grpc_server.py 73.33% 2 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #21      +/-   ##
==========================================
+ Coverage   63.16%   63.58%   +0.42%     
==========================================
  Files          18       18              
  Lines        1208     1211       +3     
  Branches      213      213              
==========================================
+ Hits          763      770       +7     
+ Misses        372      370       -2     
+ Partials       73       71       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
@dtrifiro dtrifiro added this pull request to the merge queue Jul 11, 2024
Merged via the queue into main with commit d152993 Jul 11, 2024
3 checks passed
@dtrifiro dtrifiro deleted the ibm-release-prep branch July 11, 2024 09:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants