Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Streamingllm #1489

Merged
merged 35 commits into from
Sep 5, 2024
Merged

Streamingllm #1489

merged 35 commits into from
Sep 5, 2024

Commits on Aug 24, 2024

  1. feat: avoid patch query

    @chenzhuofu
    chenzhuofu committed Aug 24, 2024
    Configuration menu
    Copy the full SHA
    bc4d9f7 View commit details
    Browse the repository at this point in the history

Commits on Aug 25, 2024

  1. chore: separate apply_pos_encoding from compute_qkv

    @chenzhuofu
    chenzhuofu committed Aug 25, 2024
    Configuration menu
    Copy the full SHA
    e41f374 View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2024

  1. chore: remove unused ptr

    @chenzhuofu
    chenzhuofu committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    5783cf1 View commit details
    Browse the repository at this point in the history

  2. fix: memory pointer alignment

    @chenzhuofu
    chenzhuofu committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    ea580f7 View commit details
    Browse the repository at this point in the history

  3. chore: minor smplification

    @chenzhuofu
    chenzhuofu committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    be93e5c View commit details
    Browse the repository at this point in the history

  4. feat: StreamingCacheInfo

    @chenzhuofu
    chenzhuofu committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    b6bcd4e View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2024

  1. feat: add streamingCache-related meta params

    @chenzhuofu
    chenzhuofu committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    f2634a9 View commit details
    Browse the repository at this point in the history

  2. chore: more acurate definition

    @chenzhuofu
    chenzhuofu committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    828b1b8 View commit details
    Browse the repository at this point in the history

  3. chore: minor

    @chenzhuofu
    chenzhuofu committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    7e71229 View commit details
    Browse the repository at this point in the history

  4. feat: add streamingCacheInfo

    @chenzhuofu
    chenzhuofu committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    a2041ea View commit details
    Browse the repository at this point in the history

  5. feat: apply_pos_encoding & update_qkv_cache, add offset control

    @chenzhuofu
    chenzhuofu committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    694cedf View commit details
    Browse the repository at this point in the history

  6. chore: minor rename

    @chenzhuofu
    chenzhuofu committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    810721e View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2024

  1. feat: kernel implementation for streaming cache usage

    @chenzhuofu
    chenzhuofu committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    70a4c2d View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2024

  1. feat: implement position encoding for streaming cache

    @chenzhuofu
    chenzhuofu committed Aug 31, 2024
    Configuration menu
    Copy the full SHA
    8face9c View commit details
    Browse the repository at this point in the history

  2. fix: params should add (de)serialization method

    @chenzhuofu
    chenzhuofu committed Aug 31, 2024
    Configuration menu
    Copy the full SHA
    e94598f View commit details
    Browse the repository at this point in the history

  3. chore: reduce kv cache size

    @chenzhuofu
    chenzhuofu committed Aug 31, 2024
    Configuration menu
    Copy the full SHA
    f0d56ec View commit details
    Browse the repository at this point in the history

Commits on Sep 1, 2024

  1. chore: minor

    @chenzhuofu
    chenzhuofu committed Sep 1, 2024
    Configuration menu
    Copy the full SHA
    419e0f8 View commit details
    Browse the repository at this point in the history

  2. fix: output misalignment

    @chenzhuofu
    chenzhuofu committed Sep 1, 2024
    Configuration menu
    Copy the full SHA
    fb31261 View commit details
    Browse the repository at this point in the history

  3. chore: minor

    @chenzhuofu
    chenzhuofu committed Sep 1, 2024
    Configuration menu
    Copy the full SHA
    fdf7b86 View commit details
    Browse the repository at this point in the history

  4. fix: speculative decoding update_custom_mask only consider mask withi… 

    …n current batch
    @chenzhuofu
    chenzhuofu committed Sep 1, 2024
    Configuration menu
    Copy the full SHA
    77aa1af View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2024

  1. fix: barrier_flag initial value

    @chenzhuofu
    chenzhuofu committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    425d770 View commit details
    Browse the repository at this point in the history

  2. doc: attention meta info

    @chenzhuofu
    chenzhuofu committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    263d9d6 View commit details
    Browse the repository at this point in the history

  3. docs: minor

    @chenzhuofu
    chenzhuofu committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    689dbd6 View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2024

  1. Added indexing support for streaming cache.

    @chenzhuofu
    chenzhuofu committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    fe5a8ad View commit details
    Browse the repository at this point in the history

  2. Merge branch 'streamingllm' of github.com:flexflow/FlexFlow into stre… 

    …amingllm
    @chenzhuofu
    chenzhuofu committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    3a1cf30 View commit details
    Browse the repository at this point in the history

  3. docs: minor

    @chenzhuofu
    chenzhuofu committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    fc4c1cd View commit details
    Browse the repository at this point in the history

  4. chore: minor rename

    @chenzhuofu
    chenzhuofu committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    e55cc6e View commit details
    Browse the repository at this point in the history

  5. feat: add streaming-llm logic to attention

    @chenzhuofu
    chenzhuofu committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    8f056af View commit details
    Browse the repository at this point in the history

  6. fix: typo

    @chenzhuofu
    chenzhuofu committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    813e43f View commit details
    Browse the repository at this point in the history

  7. fix: minor bugs in streaming llm

    @chenzhuofu
    chenzhuofu committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    e1477d4 View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2024

  1. fix: minor runtime bug

    @chenzhuofu
    chenzhuofu committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    2f9ef18 View commit details
    Browse the repository at this point in the history

  2. chore: minor output

    @chenzhuofu
    chenzhuofu committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    b5eeb26 View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2024

  1. fix: minor offset transition bug

    @chenzhuofu
    chenzhuofu committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    13850bb View commit details
    Browse the repository at this point in the history

  2. chore: minor

    @chenzhuofu
    chenzhuofu committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    30d17a2 View commit details
    Browse the repository at this point in the history

  3. style: format code

    @chenzhuofu
    chenzhuofu committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    61177ee View commit details
    Browse the repository at this point in the history