Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Support bfloat16 and ensure valid precision and activation functions consistent everywhere #3601

Merged
merged 11 commits into from
Mar 26, 2024
Merged

feat: Support bfloat16 and ensure valid precision and activation functions consistent everywhere #3601

merged 11 commits into from
Mar 26, 2024

Commits on Mar 25, 2024

  1. Ensure valid precision and activation functions consistent everywhere 

    Fix deepmodeling#3553.

Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
    @njzjz
    njzjz committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    f6e0bfb View commit details
    Browse the repository at this point in the history

Commits on Mar 26, 2024

  1. fix pt 

    Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
    @njzjz
    njzjz committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    f33503b View commit details
    Browse the repository at this point in the history

  2. fix mlp for bfloat16 

    Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
    @njzjz
    njzjz committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    dcc8d5b View commit details
    Browse the repository at this point in the history

  3. add ml_dtypes to support bfloat16 

    Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
    @njzjz
    njzjz committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    5299846 View commit details
    Browse the repository at this point in the history

  4. add comment 

    Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
    @njzjz
    njzjz committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    ee9e32a View commit details
    Browse the repository at this point in the history

  5. Revert "fix pt" 

    This reverts commit f33503b.

Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
    @njzjz
    njzjz committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    d1c85c1 View commit details
    Browse the repository at this point in the history

  6. add tests 

    Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
    @njzjz
    njzjz committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    5b4eadd View commit details
    Browse the repository at this point in the history

  7. it seems we can directly use tf.make_ndarray 

    Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
    @njzjz
    njzjz committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    75b5b98 View commit details
    Browse the repository at this point in the history

  8. fix typo 

    Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
    @njzjz
    njzjz committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    a50e447 View commit details
    Browse the repository at this point in the history

  9. fix typo 

    Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
    @njzjz
    njzjz committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    c0f463e View commit details
    Browse the repository at this point in the history

  10. Revert "fix typo" 

    This reverts commit a50e447.
    @njzjz
    njzjz committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    626e35e View commit details
    Browse the repository at this point in the history