【pir】add fused_linear_param_grad_add_pass #58401
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
你的PR提交成功，感谢你对开源项目的贡献! |
yuanlehome
approved these changes
Oct 26, 2023
Comment on lines
+56
to
+60
| if (y_trans) { | ||
| return false; | ||
| } else { | ||
| return true; | ||
| } |
There was a problem hiding this comment.
Suggested change
| if (y_trans) { | |
| return false; | |
| } else { | |
| return true; | |
| } | |
| return !y_trans; |
Comment on lines
+65
to
+70
| if (match_ctx.Tensor("dweight").Dtype() == | ||
| match_ctx.Tensor("weight_grad").Dtype()) { | ||
| return false; | ||
| } else { | ||
| return true; | ||
| } |
There was a problem hiding this comment.
Suggested change
| if (match_ctx.Tensor("dweight").Dtype() == | |
| match_ctx.Tensor("weight_grad").Dtype()) { | |
| return false; | |
| } else { | |
| return true; | |
| } | |
| return match_ctx.Tensor("dweight").Dtype() != match_ctx.Tensor("weight_grad").Dtype(); |
Comment on lines
+123
to
+127
| if (y_trans) { | ||
| return false; | ||
| } else { | ||
| return true; | ||
| } |
There was a problem hiding this comment.
Suggested change
| if (y_trans) { | |
| return false; | |
| } else { | |
| return true; | |
| } | |
| return !y_trans; |
Comment on lines
+132
to
+137
| if (match_ctx.Tensor("dweight").Dtype() == | ||
| match_ctx.Tensor("weight_grad").Dtype()) { | ||
| return false; | ||
| } else { | ||
| return true; | ||
| } |
There was a problem hiding this comment.
Suggested change
| if (match_ctx.Tensor("dweight").Dtype() == | |
| match_ctx.Tensor("weight_grad").Dtype()) { | |
| return false; | |
| } else { | |
| return true; | |
| } | |
| return match_ctx.Tensor("dweight").Dtype() != match_ctx.Tensor("weight_grad").Dtype(); |
Comment on lines
+187
to
+192
| if (match_ctx.Tensor("dweight").Dtype() == | ||
| match_ctx.Tensor("weight_grad").Dtype()) { | ||
| return false; | ||
| } else { | ||
| return true; | ||
| } |
There was a problem hiding this comment.
Suggested change
| if (match_ctx.Tensor("dweight").Dtype() == | |
| match_ctx.Tensor("weight_grad").Dtype()) { | |
| return false; | |
| } else { | |
| return true; | |
| } | |
| return match_ctx.Tensor("dweight").Dtype() != match_ctx.Tensor("weight_grad").Dtype(); |
Comment on lines
+236
to
+241
| if (match_ctx.Tensor("dweight").Dtype() == | ||
| match_ctx.Tensor("weight_grad").Dtype()) { | ||
| return false; | ||
| } else { | ||
| return true; | ||
| } |
There was a problem hiding this comment.
Suggested change
| if (match_ctx.Tensor("dweight").Dtype() == | |
| match_ctx.Tensor("weight_grad").Dtype()) { | |
| return false; | |
| } else { | |
| return true; | |
| } | |
| return match_ctx.Tensor("dweight").Dtype() != match_ctx.Tensor("weight_grad").Dtype(); |
Comment on lines
+288
to
+294
| res.Attr([](const pir::drr::MatchContext &match_ctx) -> bool { | ||
| if (match_ctx.Tensor("dweight").Dtype() == | ||
| match_ctx.Tensor("weight_grad").Dtype()) { | ||
| return false; | ||
| } else { | ||
| return true; | ||
| } |
There was a problem hiding this comment.
Suggested change
| res.Attr([](const pir::drr::MatchContext &match_ctx) -> bool { | |
| if (match_ctx.Tensor("dweight").Dtype() == | |
| match_ctx.Tensor("weight_grad").Dtype()) { | |
| return false; | |
| } else { | |
| return true; | |
| } | |
| return match_ctx.Tensor("dweight").Dtype() != match_ctx.Tensor("weight_grad").Dtype(); |
Comment on lines
+338
to
+343
| if (match_ctx.Tensor("dweight").Dtype() == | ||
| match_ctx.Tensor("weight_grad").Dtype()) { | ||
| return false; | ||
| } else { | ||
| return true; | ||
| } |
There was a problem hiding this comment.
Suggested change
| if (match_ctx.Tensor("dweight").Dtype() == | |
| match_ctx.Tensor("weight_grad").Dtype()) { | |
| return false; | |
| } else { | |
| return true; | |
| } | |
| return match_ctx.Tensor("dweight").Dtype() != match_ctx.Tensor("weight_grad").Dtype(); |
| #include "paddle/pir/pass/pass.h" | ||
| #include "paddle/pir/pass/pass_registry.h" | ||
| #include "paddle/pir/pattern_rewrite/pattern_rewrite_driver.h" | ||
| namespace { |
There was a problem hiding this comment.
感谢！#58420
xiaoguoguo626807
added a commit
that referenced
this pull request
Oct 27, 2023
cxxly
pushed a commit
to cxxly/Paddle
that referenced
this pull request
Oct 30, 2023
* fused_linear_param_grad_add_pass * modify * skip cpu ci * add muti_presion modify * modify * modify * modify * modify
cxxly
pushed a commit
to cxxly/Paddle
that referenced
this pull request
Oct 30, 2023
* fused_linear_param_grad_add_pass * modify * skip cpu ci * add muti_presion modify * modify * modify * modify * modify
cxxly
pushed a commit
to cxxly/Paddle
that referenced
this pull request
Oct 30, 2023
* fused_linear_param_grad_add_pass * modify * skip cpu ci * add muti_presion modify * modify * modify * modify * modify
cxxly
pushed a commit
to cxxly/Paddle
that referenced
this pull request
Oct 30, 2023
* fused_linear_param_grad_add_pass * modify * skip cpu ci * add muti_presion modify * modify * modify * modify * modify
cxxly
pushed a commit
to cxxly/Paddle
that referenced
this pull request
Oct 30, 2023
* fused_linear_param_grad_add_pass * modify * skip cpu ci * add muti_presion modify * modify * modify * modify * modify
cxxly
pushed a commit
to cxxly/Paddle
that referenced
this pull request
Oct 31, 2023
* fused_linear_param_grad_add_pass * modify * skip cpu ci * add muti_presion modify * modify * modify * modify * modify
cxxly
pushed a commit
to cxxly/Paddle
that referenced
this pull request
Oct 31, 2023
* fused_linear_param_grad_add_pass * modify * skip cpu ci * add muti_presion modify * modify * modify * modify * modify
cxxly
pushed a commit
to cxxly/Paddle
that referenced
this pull request
Oct 31, 2023
* fused_linear_param_grad_add_pass * modify * skip cpu ci * add muti_presion modify * modify * modify * modify * modify
danleifeng
pushed a commit
to danleifeng/Paddle
that referenced
this pull request
Nov 14, 2023
* fused_linear_param_grad_add_pass * modify * skip cpu ci * add muti_presion modify
danleifeng
pushed a commit
to danleifeng/Paddle
that referenced
this pull request
Nov 14, 2023
* fused_linear_param_grad_add_pass * modify * skip cpu ci * add muti_presion modify * modify * modify * modify * modify
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
others
PR changes
others
Description
pcard-67164
增加fused_linear_param_grad_add_pass相关pattern
add_grad + matmul_grad + add_ -> matmul + fused_liner_param_gard_add
matmul_grad + add_ -> matmul + fused_liner_param_gard_add
matmul + 0 = add_(0,1) -> fused_liner_param_gard_add
matmul + 1 = add_(1,0) -> fused_liner_param_gard_add
add_grad + matmul + 0 = add_(0,1) -> fused_liner_param_gard_add
add_grad + matmul + 1 = add_(1,0) -> fused_liner_param_gard_add
注意：
1.fused_liner_param_gard_add muti_precision 参数推导为（dweight.dtype （其他结果）== weight_grad(matmul结果) 时为false， 否则为true。
2.matmul_grad 的拆解融合目前只支持trans_x=false 的匹配，trans_x=true未支持
add_ 的inplace tensor 为其他结果，inplace matmul_grad结果未支持。
3. 模式匹配 前向matmul 的 第二个参数为weight , 不支持第一个参数为weight
4. 由于新ir组网得到的梯度累加算子是add_n, python 测无法组出上述pattern，在c++测进行测试，只测试组网替换，执行结果未测。