Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Initial FSDP Support for QLoRA Finetuning #970

Merged
merged 11 commits into from
Jan 17, 2024
Merged

Initial FSDP Support for QLoRA Finetuning #970

merged 11 commits into from
Jan 17, 2024

Commits on Jan 4, 2024

  1. Minimal changes for fp32 4bit storage from BNB commit 8278fca

    @warner-benjamin
    warner-benjamin committed Jan 4, 2024
    Configuration menu
    Copy the full SHA
    fcc5bb9 View commit details
    Browse the repository at this point in the history

Commits on Jan 5, 2024

  1. Params4bit with selectable storage dtype

    @warner-benjamin
    warner-benjamin committed Jan 5, 2024
    Configuration menu
    Copy the full SHA
    c51403c View commit details
    Browse the repository at this point in the history

Commits on Jan 8, 2024

  1. possible fix for double quantizing linear weight & quant storage dtype

    @warner-benjamin
    warner-benjamin committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    83c538a View commit details
    Browse the repository at this point in the history

Commits on Jan 9, 2024

  1. minor fixes in Params4bit for peft tests

    @KeremTurgutlu
    KeremTurgutlu committed Jan 9, 2024
    Configuration menu
    Copy the full SHA
    359bfc9 View commit details
    Browse the repository at this point in the history

  2. remove redundant

    @KeremTurgutlu
    KeremTurgutlu committed Jan 9, 2024
    Configuration menu
    Copy the full SHA
    112741a View commit details
    Browse the repository at this point in the history

  3. Merge pull request #1 from KeremTurgutlu/peft_tests 

    Peft tests
    @warner-benjamin
    warner-benjamin authored Jan 9, 2024
    Configuration menu
    Copy the full SHA
    754e75b View commit details
    Browse the repository at this point in the history

Commits on Jan 12, 2024

  1. add float16

    @KeremTurgutlu
    KeremTurgutlu committed Jan 12, 2024
    Configuration menu
    Copy the full SHA
    9e5a15b View commit details
    Browse the repository at this point in the history

  2. Merge branch 'TimDettmers:main' into cuda_fix_quant_storage_dtype

    @warner-benjamin
    warner-benjamin authored Jan 12, 2024
    Configuration menu
    Copy the full SHA
    e8e8eed View commit details
    Browse the repository at this point in the history

  3. update test

    @KeremTurgutlu
    KeremTurgutlu committed Jan 12, 2024
    Configuration menu
    Copy the full SHA
    4aecbdf View commit details
    Browse the repository at this point in the history

  4. Merge pull request #2 from KeremTurgutlu/peft_tests 

    Peft tests
    @warner-benjamin
    warner-benjamin authored Jan 12, 2024
    Configuration menu
    Copy the full SHA
    4f15388 View commit details
    Browse the repository at this point in the history

Commits on Jan 16, 2024

  1. Remove float16 quant cast as there are fp32, bf16, & fp16 quant kernels

    @warner-benjamin
    warner-benjamin committed Jan 16, 2024
    Configuration menu
    Copy the full SHA
    89ebdb2 View commit details
    Browse the repository at this point in the history