Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Inference test for llama #4508

Merged
merged 34 commits into from
Aug 30, 2023
Merged

Add Inference test for llama #4508

merged 34 commits into from
Aug 30, 2023

Commits on Aug 23, 2023

  1. add kv cache memory manager

    @yuanheng-zhao
    yuanheng-zhao committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    fbff5d3 View commit details
    Browse the repository at this point in the history

  2. add stateinfo during inference

    @yuanheng-zhao
    yuanheng-zhao committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    2d55ace View commit details
    Browse the repository at this point in the history

  3. add

    @CjhHa1
    CjhHa1 committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    e55e565 View commit details
    Browse the repository at this point in the history

  4. add infer example

    @CjhHa1
    CjhHa1 committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    a971535 View commit details
    Browse the repository at this point in the history

  5. finish

    @CjhHa1
    CjhHa1 committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    0ae5bb7 View commit details
    Browse the repository at this point in the history

  6. finish

    @CjhHa1
    CjhHa1 committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    a8f7386 View commit details
    Browse the repository at this point in the history

  7. format

    @yuanheng-zhao
    yuanheng-zhao committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    cb45cf8 View commit details
    Browse the repository at this point in the history

  8. format

    @yuanheng-zhao
    yuanheng-zhao committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    4f21bc5 View commit details
    Browse the repository at this point in the history

  9. rename file

    @yuanheng-zhao
    yuanheng-zhao committed Aug 23, 2023
    Configuration menu
    Copy the full SHA
    389d0d4 View commit details
    Browse the repository at this point in the history

Commits on Aug 24, 2023

  1. add kv cache test

    @yuanheng-zhao
    yuanheng-zhao committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    bdba1b5 View commit details
    Browse the repository at this point in the history

  2. revise on BatchInferState

    @yuanheng-zhao
    yuanheng-zhao committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    813e23a View commit details
    Browse the repository at this point in the history

  3. Merge commit 'refs/pull/4495/head' of https://github.com/hpcaitech/Co… 

    …lossalAI into feature/colossal-inference
    @isky-cd
    isky-cd committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    5b08d60 View commit details
    Browse the repository at this point in the history

  4. add inference test for llama

    @isky-cd
    isky-cd committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    469a3c5 View commit details
    Browse the repository at this point in the history

  5. fix conflict

    @isky-cd
    isky-cd committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    5993a0f View commit details
    Browse the repository at this point in the history

  6. fix conflict

    @isky-cd
    isky-cd committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    a98000f View commit details
    Browse the repository at this point in the history

  7. fix conflict

    @isky-cd
    isky-cd committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    7686c07 View commit details
    Browse the repository at this point in the history

  8. feature: add some new features for llama engine

    @isky-cd
    isky-cd committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    ba089d7 View commit details
    Browse the repository at this point in the history

  9. adapt colossalai triton interface

    @isky-cd
    isky-cd committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    68b5fe8 View commit details
    Browse the repository at this point in the history

  10. Change the parent class of llama policy

    @isky-cd
    isky-cd committed Aug 24, 2023
    Configuration menu
    Copy the full SHA
    6021b13 View commit details
    Browse the repository at this point in the history

Commits on Aug 25, 2023

  1. add nvtx

    @isky-cd
    isky-cd committed Aug 25, 2023
    Configuration menu
    Copy the full SHA
    6a1bafa View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2023

  1. move llama inference code to tensor_parallel

    @isky-cd
    isky-cd committed Aug 27, 2023
    Configuration menu
    Copy the full SHA
    f79308e View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2023

  1. Merge branch 'feature/colossal-inference' of https://github.com/hpcai… 

    …tech/ColossalAI into feature/colossal-inference
    @isky-cd
    isky-cd committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    a6cc3dd View commit details
    Browse the repository at this point in the history

  2. fix __init__.py

    @isky-cd
    isky-cd committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    2a6a380 View commit details
    Browse the repository at this point in the history

  3. rm tensor_parallel

    @isky-cd
    isky-cd committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    d10dcf4 View commit details
    Browse the repository at this point in the history

  4. fix: fix bugs in auto_policy.py

    @isky-cd
    isky-cd committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    fb2603b View commit details
    Browse the repository at this point in the history

  5. fix:rm some unused codes

    @isky-cd
    isky-cd committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    92fd955 View commit details
    Browse the repository at this point in the history

  6. mv colossalai/tpinference to colossalai/inference/tensor_parallel

    @isky-cd
    isky-cd committed Aug 28, 2023
    Configuration menu
    Copy the full SHA
    c747249 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2023

  1. Merge branch 'feature/colossal-inference' into llama_test_branch

    @isky-cd
    isky-cd committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    8507fc5 View commit details
    Browse the repository at this point in the history

  2. change __init__.py

    @isky-cd
    isky-cd committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    c27088f View commit details
    Browse the repository at this point in the history

  3. save change

    @isky-cd
    isky-cd committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    af16040 View commit details
    Browse the repository at this point in the history

  4. fix conflict

    @isky-cd
    isky-cd committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    bfc55cc View commit details
    Browse the repository at this point in the history

  5. fix engine

    @isky-cd
    isky-cd committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    f30f542 View commit details
    Browse the repository at this point in the history

  6. Bug fix: Fix hang

    @isky-cd
    isky-cd committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    4b52ebd View commit details
    Browse the repository at this point in the history

  7. remove llama_infer_engine.py

    @isky-cd
    isky-cd committed Aug 30, 2023
    Configuration menu
    Copy the full SHA
    6d06421 View commit details
    Browse the repository at this point in the history