Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Frontend] Add readiness and liveness endpoints to OpenAI API server #7078

Closed
wants to merge 18 commits into from
Closed

[Frontend] Add readiness and liveness endpoints to OpenAI API server #7078

wants to merge 18 commits into from

Commits on Aug 2, 2024

  1. add readiness and liveness k8s probes for openai api_server

    @mfournioux
    mfournioux committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    bac2c90 View commit details
    Browse the repository at this point in the history

  2. update naming for pydantic classes from openai protocol

    @mfournioux
    mfournioux committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    d65bf58 View commit details
    Browse the repository at this point in the history

  3. update naming for pydantic classes from openai protocol and remove aw… 

    …ait in variable definition
    @mfournioux
    mfournioux committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    0c7945d View commit details
    Browse the repository at this point in the history

  4. add tests for readiness and liveness endpoints

    @mfournioux
    mfournioux committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    fa1c549 View commit details
    Browse the repository at this point in the history

  5. correct syntax pydantic class in protocol

    @mfournioux
    mfournioux committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    2fbaa2f View commit details
    Browse the repository at this point in the history

  6. correct ruff errors

    @mfournioux
    mfournioux committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    27ef5ac View commit details
    Browse the repository at this point in the history

  7. correct ruff errors

    @mfournioux
    mfournioux committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    7fa6a37 View commit details
    Browse the repository at this point in the history

  8. fixing isort issues

    @mfournioux
    mfournioux committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    18a9f2c View commit details
    Browse the repository at this point in the history

  9. update some typo

    @mfournioux
    mfournioux committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    32e030b View commit details
    Browse the repository at this point in the history

  10. correct some yapf errors

    @mfournioux
    mfournioux committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    5127e91 View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2024

  1. correct readiness probe regarding its http status

    @mfournioux
    mfournioux committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    c698d76 View commit details
    Browse the repository at this point in the history

  2. replace liveness endpoint by health endpoint and renaming readiness e… 

    …ndpoint
    @mfournioux
    mfournioux committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    ea8be80 View commit details
    Browse the repository at this point in the history

  3. clean some imports and configure error response for readiness endpoint

    @mfournioux
    mfournioux committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    c0baaea View commit details
    Browse the repository at this point in the history

  4. correct model response in readiness endpoint

    @mfournioux
    mfournioux committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    3a8b227 View commit details
    Browse the repository at this point in the history

  5. add return response 500 for readiness if model weights not loaded

    @mfournioux
    mfournioux committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    ac095c1 View commit details
    Browse the repository at this point in the history

  6. Update test_basic.py 

    Remove duplicate.
    @mfournioux
    mfournioux authored Aug 5, 2024
    Configuration menu
    Copy the full SHA
    14b2b91 View commit details
    Browse the repository at this point in the history

Commits on Aug 6, 2024

  1. update the readiness endpoint with a try clause

    @mfournioux
    mfournioux committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    b06d686 View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2024

  1. add check if KV cache has been set up in readiness endpoint

    @mfournioux
    mfournioux committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    e950836 View commit details
    Browse the repository at this point in the history