Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Additional mask support on FA2 #19

Merged
merged 20 commits into from
Sep 22, 2023
Merged

Additional mask support on FA2 #19

merged 20 commits into from
Sep 22, 2023

Commits on Sep 1, 2023

  1. Add addiction mask support. Tested on flash_attn_fwd.

    @umiswing
    umiswing committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    471f95f View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2023

  1. Bug fix: fix scale setting typo for attn mask.

    @umiswing
    umiswing committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    4c162e8 View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2023

  1. Add gMask tile rule for bwd. 

    Tested on mask addition for one block.
Code to be cleaned.
Softmax in bwd is removed for testing.
    @umiswing
    umiswing committed Sep 9, 2023
    Configuration menu
    Copy the full SHA
    c2383fd View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2023

  1. Remove unused code.

    @umiswing
    umiswing committed Sep 10, 2023
    Configuration menu
    Copy the full SHA
    479ce21 View commit details
    Browse the repository at this point in the history

  2. Fix buggy tile rule in fwd and bwd. 

    Looks correct on ouput and dv.
    @umiswing
    umiswing committed Sep 10, 2023
    Configuration menu
    Copy the full SHA
    29f691d View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2023

  1. Just remove some log.

    @umiswing
    umiswing committed Sep 12, 2023
    Configuration menu
    Copy the full SHA
    d7ad819 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of https://github.com/PaddlePaddle/flash-attention … 

    …into fa2_mask
    @umiswing
    umiswing committed Sep 12, 2023
    Configuration menu
    Copy the full SHA
    e1564d5 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2023

  1. Merge tag 'v2.0.8' into fa2_mask

    @umiswing
    umiswing committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    447c093 View commit details
    Browse the repository at this point in the history

  2. Comment out log to avoid compile error.

    @umiswing
    umiswing committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    c718d53 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' of https://github.com/PaddlePaddle/flash-attention … 

    …into fa2_mask
    @umiswing
    umiswing committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    d85752b View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2023

  1. Add softmax_unscale on kernel and padded api.

    @umiswing
    umiswing committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    49d2a23 View commit details
    Browse the repository at this point in the history

  2. Add api for varlen kernel.

    @umiswing
    umiswing committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    854c4df View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2023

  1. Fix bug when tiles do not fit evenly into seqlen.

    @umiswing
    umiswing committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    a3a1c27 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2023

  1. add residue to avoid oob mem access

    @umiswing
    umiswing committed Sep 18, 2023
    Configuration menu
    Copy the full SHA
    ec8bf4f View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2023

  1. Add template param Is_attn_mask. 

    Remove some unused code and comments.
    @umiswing
    umiswing committed Sep 19, 2023
    Configuration menu
    Copy the full SHA
    f939e9c View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2023

  1. Add a check macro.

    @umiswing
    umiswing committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    2867657 View commit details
    Browse the repository at this point in the history

  2. Untab.

    @umiswing
    umiswing committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    8454bbb View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2023

  1. Remove unused modification. 

    Use params.seqlen for mask predication.
    @umiswing
    umiswing committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    1f8dbad View commit details
    Browse the repository at this point in the history

  2. Rename: mask_seq_mod_size -> mask_seq_q_mod_size.

    @umiswing
    umiswing committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    957a73c View commit details
    Browse the repository at this point in the history

  3. Remove TODO. Modify comment.

    @umiswing
    umiswing committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    9ebf2af View commit details
    Browse the repository at this point in the history