Integrate rmsnorm kernel #54998
Merged
Integrate rmsnorm kernel #54998
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
你的PR提交成功，感谢你对开源项目的贡献! |
heavengate
reviewed
Jun 29, 2023
heavengate
reviewed
Jun 29, 2023
heavengate
reviewed
Jun 29, 2023
heavengate
reviewed
Jun 29, 2023
heavengate
previously approved these changes
Jul 5, 2023
|
这个算子无法支持对于加上一次gemm的Bias和residual，所以我在想是否有可能支持load/store的可配置性 |
heavengate
previously approved these changes
Jul 7, 2023
Aurelius84
previously approved these changes
Jul 7, 2023
raindrops2sea
approved these changes
Jul 11, 2023
kolinwei
approved these changes
Jul 11, 2023
zyfncg
reviewed
Jul 11, 2023
| #pragma once | ||
|
|
||
| #include "paddle/phi/core/dense_tensor.h" | ||
| #include "paddle/phi/core/selected_rows.h" |
There was a problem hiding this comment.
selected_rows.h头文件看上去没有用到，可以去掉
There was a problem hiding this comment.
好的，相关Comment统一下一个PR refine
Comment on lines
+31
to
+71
| template <typename T, typename Context> | ||
| void RmsNormWrapper(const Context& ctx, | ||
| const T* x, | ||
| const T* weight, | ||
| const T* bias, | ||
| const float epsilon, | ||
| const int rows, | ||
| const int cols, | ||
| T* output); | ||
|
|
||
| template <typename T, typename Context> | ||
| void ResidualAddRmsNormWrapper(const Context& ctx, | ||
| const T* x, | ||
| const T* residual, | ||
| const T* bias, | ||
| const T* norm_weight, | ||
| const T* norm_bias, | ||
| const float epsilon, | ||
| const int rows, | ||
| const int cols, | ||
| T* residual_output, | ||
| T* output); | ||
|
|
||
| template <typename T, typename Context> | ||
| void RmsNormInt8OutWrapper(const Context& ctx, | ||
| const T* x, | ||
| const T* weight, | ||
| const T* bias, | ||
| const float epsilon, | ||
| const int rows, | ||
| const int cols, | ||
| const float in_scale, | ||
| const int quant_round_type, | ||
| const float quant_max_bound, | ||
| const float quant_min_bound, | ||
| int8_t* output); | ||
|
|
||
| template <typename T, typename Context> | ||
| void ResidualAddRmsNormInt8OutWrapper(const Context& ctx, | ||
| const T* x, | ||
| const T* residual, |
There was a problem hiding this comment.
这几个Wrapper函数声明在头文件中好像没有起到作用，可以去掉吗？
There was a problem hiding this comment.
这几个wrapper后续会在一个NormHelper里去使用（如果后续可以去掉，下一个PR我会去除）
zyfncg
approved these changes
Jul 11, 2023
Aurelius84
approved these changes
Jul 11, 2023
qingqing01
approved these changes
Jul 11, 2023
Closed
cqulilujia
pushed a commit
to cqulilujia/Paddle
that referenced
this pull request
Jul 24, 2023
* add rmsnorm kernel * add static graph test * fix round type * use alignas to avoid msvc compile error * remove redundant headerfile to avoid rocm compile error * fix rocm compile not found cub * Add document
wz1qqx
pushed a commit
to wz1qqx/Paddle
that referenced
this pull request
Jul 31, 2023
* add rmsnorm kernel * add static graph test * fix round type * use alignas to avoid msvc compile error * remove redundant headerfile to avoid rocm compile error * fix rocm compile not found cub * Add document
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
New features
PR changes
OPs
Description
Integrate RMSNorm CUDA Kernel
Support Residual Load, Int8Out, change use Single Pass for Inference Speed.
Pcard-72603