【Hackathon No.59】addmm 算子FP16/BF16单测完善 #53111
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
你的PR提交成功，感谢你对开源项目的贡献! |
co63oc
force-pushed
the
addmm
branch
8 times, most recently
from
60d7c61
to
11e09f1
Compare
ZzSean
reviewed
Apr 26, 2023
| @@ -982,6 +982,25 @@ inline void Blas<phi::GPUContext>::GEMM(bool transA, | |||
| }); | |||
| } | |||
|
|
|||
| template <> | |||
| template <> | |||
| inline void Blas<phi::GPUContext>::GEMM(bool transA UNUSED, | |||
There was a problem hiding this comment.
这个函数虽然没有用到，但是也可以参考729行的bf16 GEMM实现？
ZzSean
reviewed
Apr 26, 2023
| // The VCOPY does not support the float16, bfloat16 | ||
| if (!is_float16_or_bfloat16) { | ||
| mt_blas.VCOPY( | ||
| total_elems, out_grad.data<MPType>(), input_grad->data<MPType>()); |
There was a problem hiding this comment.
这里既然不支持fp16和bf16，是不是没必要用MPType了
There was a problem hiding this comment.
不使用MPType编译器有错误，编译器不按程序if判断，按T类型编译，这里如果使用out_grad.data，编译器会提示不支持float16类型参数
ZzSean
reviewed
Apr 26, 2023
| @@ -78,19 +107,45 @@ void AddmmGradKernel(const Context& dev_ctx, | |||
| Array2(input_grad->dims()[0], input_grad->dims()[1]); | |||
|
|
|||
| if (row_compress && col_compress) { | |||
| eigen_dinput.device(place) = | |||
| eigen_dout.sum().eval().reshape(eigen_dinput_shape); | |||
| eigen_dinput.device(place) = eigen_dout.template cast<MPType>() | |||
There was a problem hiding this comment.
这里如果是非fp16和bf16的话，多了两个cast感觉会影响性能，要不也增加个分支
|
@ZzSean 修改CI已完成 |
ZzSean
reviewed
Apr 28, 2023
| int ldb, | ||
| phi::dtype::bfloat16 beta, | ||
| phi::dtype::bfloat16 *C, | ||
| int ldc UNUSED) const { |
ZzSean
reviewed
Apr 28, 2023
| context_.GetComputeCapability(), | ||
| 80, | ||
| phi::errors::InvalidArgument( | ||
| "rocblas fp16 gemm requires GPU compute capability >= 80," |
ZzSean
pushed a commit
to ZzSean/Paddle
that referenced
this pull request
May 5, 2023
* Add addmm tests * Fix code
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Others
PR changes
Others
Description
addmm算子FP16/BF16单测完善
float16类型使用cublasHgemm
增加 bfloat16 类型调用cublasGemmEx
使用blas VCOPY， SCAL的float16类型编译错误，增加mt_blas使用MPType类型调用VCOPY， SCAL
反向中VCOPY, SCAL不支持float16, bfloat16，增加CopyOrScaleFunctor修改数据
提交CI测试需要设置精度float16 check_out atol=1e-2
如果float16设置 self.check_output(atol=1e-2), PR-CI-Model-benchmark CI有错误，然后修改为verify_output比较