Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add option to completion API to truncate prompt tokens #3144

Merged
merged 13 commits into from
Apr 5, 2024
Merged

Add option to completion API to truncate prompt tokens #3144

merged 13 commits into from
Apr 5, 2024

Commits on Mar 1, 2024

  1. Add truncate_input_tokens to openai completions api

    @tdoublep
    tdoublep committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    d36ba54 View commit details
    Browse the repository at this point in the history

  2. Better docstring

    @tdoublep
    tdoublep committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    586b4a0 View commit details
    Browse the repository at this point in the history

  3. Apply formatting

    @tdoublep
    tdoublep committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    559e441 View commit details
    Browse the repository at this point in the history

  4. Update vllm/entrypoints/openai/serving_engine.py 

    tokenizer_kwargs: more efficient allocation

Co-authored-by: Nick Hill <nickhill@us.ibm.com>
    @tdoublep @njhill
    tdoublep and njhill authored Mar 1, 2024
    Configuration menu
    Copy the full SHA
    32bc1f8 View commit details
    Browse the repository at this point in the history

  5. Use truncate_prompt_tokens naming

    @tdoublep
    tdoublep committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    4b92849 View commit details
    Browse the repository at this point in the history

  6. serving_engine.py: fix bug

    @tdoublep
    tdoublep committed Mar 1, 2024
    Configuration menu
    Copy the full SHA
    b6554a8 View commit details
    Browse the repository at this point in the history

Commits on Mar 5, 2024

  1. sampling_params: Docstring update

    @tdoublep
    tdoublep committed Mar 5, 2024
    Configuration menu
    Copy the full SHA
    6efc8d1 View commit details
    Browse the repository at this point in the history

Commits on Mar 26, 2024

  1. Resolve conflicts with latest changes on main

    @tdoublep
    tdoublep committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    7cf8a98 View commit details
    Browse the repository at this point in the history

  2. Use pydantic.conint

    @tdoublep
    tdoublep committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    520f8bb View commit details
    Browse the repository at this point in the history

  3. fix formatting

    @tdoublep
    tdoublep committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    08b3e19 View commit details
    Browse the repository at this point in the history

  4. Fix formatting

    @tdoublep
    tdoublep committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    c8f3429 View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2024

  1. Merge branch 'main' into openai-truncate-input

    @tdoublep
    tdoublep committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    e08bb84 View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2024

  1. Fix conflicts

    @tdoublep
    tdoublep committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    c88be4f View commit details
    Browse the repository at this point in the history