Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Model][Jamba] Mamba cache single buffer #6739

Merged
merged 22 commits into from
Aug 9, 2024
Merged

[Model][Jamba] Mamba cache single buffer #6739

merged 22 commits into from
Aug 9, 2024

Commits on Aug 9, 2024

  1. Mamba cache single buffer (vllm-project#42) 

    * WIP - working on swaping indices

* WIP

* Save changes

* Orginize indices during assigment, working and passing tests!

* Add TODOs

* Remove diff

* Format

* Remove TODOs

* Remove unused code

* Cleanup

* Cleanup

* Cleanup the redundant 10 blocks

* Small changes

* Simplify code and add comments

* Renaming and simplify

* Remove return

* Clean up

* Cleanup

* Renaming

* Another clean up

* Clean up

* Clean up and simplify more

* Add n > 1 test

* Format

* cosmetics

* Add functionality to find first free

* Raise exception if could not find spot

* Typos

* Add 2 slots as precaution

---------

Co-authored-by: Mor Zusman <morz@ai21.com>
    @mzusman
    mzusman and Mor Zusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    9de1f12 View commit details
    Browse the repository at this point in the history

  2. Format

    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    b9ef930 View commit details
    Browse the repository at this point in the history

  3. Change example

    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    f9d311d View commit details
    Browse the repository at this point in the history

  4. Change tested model (trained), now the tests are more reliable

    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    69c0da8 View commit details
    Browse the repository at this point in the history

  5. Bugfix, the dest index didn't run on the seq ids

    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    9a3a1be View commit details
    Browse the repository at this point in the history

  6. Clean up

    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    c705ed2 View commit details
    Browse the repository at this point in the history

  7. Revert "Clean up" 

    This reverts commit 381c2aa.
    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    7fd4e22 View commit details
    Browse the repository at this point in the history

  8. Revert "Bugfix, the dest index didn't run on the seq ids" 

    This reverts commit f1e792d.
    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    7f97c4e View commit details
    Browse the repository at this point in the history

  9. Revert "Change tested model (trained), now the tests are more reliable" 

    This reverts commit bda9876.
    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    52239d0 View commit details
    Browse the repository at this point in the history

  10. Bugfix, the dest index didn't run on the seq ids

    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    27a15e4 View commit details
    Browse the repository at this point in the history

  11. Cleanup

    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    d7d07fb View commit details
    Browse the repository at this point in the history

  12. Prettier version

    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    12d8648 View commit details
    Browse the repository at this point in the history

  13. Half instead of bf16

    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    4fc3dce View commit details
    Browse the repository at this point in the history

  14. Formattin

    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    f2c7723 View commit details
    Browse the repository at this point in the history

  15. Change test to float

    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    44788c4 View commit details
    Browse the repository at this point in the history

  16. bf16 for the test

    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    e598d96 View commit details
    Browse the repository at this point in the history

  17. Remove n > 1 test for now, need to check why it fails on L4

    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    7d553c9 View commit details
    Browse the repository at this point in the history

  18. Format

    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    df269e5 View commit details
    Browse the repository at this point in the history

  19. Factor out moving out the occupied index

    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    60857a3 View commit details
    Browse the repository at this point in the history

  20. Add comment

    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    9e583d6 View commit details
    Browse the repository at this point in the history

  21. Format

    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    c2e9a1d View commit details
    Browse the repository at this point in the history

  22. Jamba model

    @mzusman
    mzusman committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    3eeeeb7 View commit details
    Browse the repository at this point in the history