Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve FP6-LLM 2+4bit weight splitting + user API #279

Merged
merged 24 commits into from
May 26, 2024
Merged

Improve FP6-LLM 2+4bit weight splitting + user API #279

merged 24 commits into from
May 26, 2024

Commits on May 22, 2024

  1. add annotation

    @gau-nernst
    gau-nernst committed May 22, 2024
    Configuration menu
    Copy the full SHA
    36add71 View commit details
    Browse the repository at this point in the history

Commits on May 25, 2024

  1. Merge branch 'main' into fp6_weight_split

    @gau-nernst
    gau-nernst committed May 25, 2024
    Configuration menu
    Copy the full SHA
    18824a7 View commit details
    Browse the repository at this point in the history

  2. add weight splitting logic

    @gau-nernst
    gau-nernst committed May 25, 2024
    Configuration menu
    Copy the full SHA
    58bcf2f View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into fp6_weight_split

    @gau-nernst
    gau-nernst committed May 25, 2024
    Configuration menu
    Copy the full SHA
    dd84cc2 View commit details
    Browse the repository at this point in the history

  4. update from fp6_quant

    @gau-nernst
    gau-nernst committed May 25, 2024
    Configuration menu
    Copy the full SHA
    5e44f1c View commit details
    Browse the repository at this point in the history

Commits on May 26, 2024

  1. merge to_tc_float6_e3m2

    @gau-nernst
    gau-nernst committed May 26, 2024
    Configuration menu
    Copy the full SHA
    bccc4f6 View commit details
    Browse the repository at this point in the history

  2. add more optimized version

    @gau-nernst
    gau-nernst committed May 26, 2024
    Configuration menu
    Copy the full SHA
    cfa304c View commit details
    Browse the repository at this point in the history

  3. add some notes

    @gau-nernst
    gau-nernst committed May 26, 2024
    Configuration menu
    Copy the full SHA
    ed9cb1d View commit details
    Browse the repository at this point in the history

  4. add from_tc_float6_e3m2

    @gau-nernst
    gau-nernst committed May 26, 2024
    Configuration menu
    Copy the full SHA
    f609b6f View commit details
    Browse the repository at this point in the history

  5. add some docs

    @gau-nernst
    gau-nernst committed May 26, 2024
    Configuration menu
    Copy the full SHA
    dec40af View commit details
    Browse the repository at this point in the history

  6. make fp6_llm.py

    @gau-nernst
    gau-nernst committed May 26, 2024
    Configuration menu
    Copy the full SHA
    5e5dfdc View commit details
    Browse the repository at this point in the history

  7. add test for linear

    @gau-nernst
    gau-nernst committed May 26, 2024
    Configuration menu
    Copy the full SHA
    5bdcd50 View commit details
    Browse the repository at this point in the history

  8. fix fp6 llm

    @gau-nernst
    gau-nernst committed May 26, 2024
    Configuration menu
    Copy the full SHA
    708d485 View commit details
    Browse the repository at this point in the history

  9. switch to v2 since it's faster

    @gau-nernst
    gau-nernst committed May 26, 2024
    Configuration menu
    Copy the full SHA
    ce0ffc1 View commit details
    Browse the repository at this point in the history

  10. fix type hint for old python

    @gau-nernst
    gau-nernst committed May 26, 2024
    Configuration menu
    Copy the full SHA
    59e39ce View commit details
    Browse the repository at this point in the history

  11. simplify further

    @gau-nernst
    gau-nernst committed May 26, 2024
    Configuration menu
    Copy the full SHA
    cb43a7b View commit details
    Browse the repository at this point in the history

  12. fix typing for old python

    @gau-nernst
    gau-nernst committed May 26, 2024
    Configuration menu
    Copy the full SHA
    b90938a View commit details
    Browse the repository at this point in the history

  13. add test

    @gau-nernst
    gau-nernst committed May 26, 2024
    Configuration menu
    Copy the full SHA
    66cbe1d View commit details
    Browse the repository at this point in the history

  14. eliminate indexing.faster on CUDA

    @gau-nernst
    gau-nernst committed May 26, 2024
    Configuration menu
    Copy the full SHA
    5ed6767 View commit details
    Browse the repository at this point in the history

  15. skip fp6_llm on cpu

    @gau-nernst
    gau-nernst committed May 26, 2024
    Configuration menu
    Copy the full SHA
    fa08a3d View commit details
    Browse the repository at this point in the history

  16. improve error message

    @gau-nernst
    gau-nernst committed May 26, 2024
    Configuration menu
    Copy the full SHA
    6945498 View commit details
    Browse the repository at this point in the history

  17. add support for extra batch dims

    @gau-nernst
    gau-nernst committed May 26, 2024
    Configuration menu
    Copy the full SHA
    70b5a4c View commit details
    Browse the repository at this point in the history

  18. cast output to original dtype

    @gau-nernst
    gau-nernst committed May 26, 2024
    Configuration menu
    Copy the full SHA
    d6c6b6a View commit details
    Browse the repository at this point in the history

  19. fix precision error due to dtype

    @gau-nernst
    gau-nernst committed May 26, 2024
    Configuration menu
    Copy the full SHA
    d798eaf View commit details
    Browse the repository at this point in the history