Delete extra input (Bias, ResidualData) in OpMaker of conv2d #49121

zyfncg · 2022-12-16T12:42:03Z

PR types

Others

PR changes

OPs

Describe

删除conv2d算子OpMaker中的Extra类型输入参数（Bias, ResidualData）。

在前序工作（PR47579, PR48848）的基础上，conv2d算子Extra输入参数的使用依赖基本被解除，因此本PR移除了conv2d算子的Extra输入参数（Bias, ResidualData）。
对于conv2d中仅用于fuse场景的Extra属性（如fuse_activation, fuse_residual_connection等）由于不再被使用在本PR中也进行了删除。

Based on previous work (PR47579, PR48848), The dependence on the use of the extra input parameters(Bias, ResidualData) of the conv2d operator has been removed, so this PR deletes the extra input parameters (Bias, ResidualData) in OpMaker of conv2d.
Extra attributes (such asfuse_activation, fuse_residual_connection, etc.) that are only used in fused kernel for conv2d are no longer used, so we also delete them in this PR.

paddle-bot · 2022-12-16T12:42:06Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

… clear_conv2d_extra

Silv3S

LGTM

Silv3S · 2022-12-28T08:31:49Z

paddle/fluid/framework/ir/mkldnn/cpu_quantize_squash_pass.cc

@@ -354,7 +354,9 @@ void CPUQuantizeSquashPass::OpDequantSquash(Graph* graph) const {
          FindOutputNameByVarName(any_op->Op(), dequant_in->Name());

      if (output_name.empty()) return;
-
+      if (any_op->Op()->Type() == "conv2d") {
+        any_op->Op()->SetType("fused_conv2d");


So all int8-oneDNN kernels should be executed as fused kernels by default?

Yes, we are trying to delete the extra inputs and attributes in base op, so some extra attributes for int8-oneDNN kernel are also removed, currently we have to put them into fused kernel to execute because no better choice.
I think a good way to execute int8-oneDNN kernel is creating a new kernel for int8-oneDNN, but it is difficult to implement at the current stage, maybe we could come up with a good solution in the future.

Ok, thank you for explaining

Silv3S · 2022-12-28T08:40:14Z

paddle/phi/api/yaml/op_compat.yaml

@@ -556,6 +553,11 @@
  extra :
    attrs : [bool use_mkldnn = false]

+- op : fused_conv2d


fused_conv2d has own operator with extra attributes declared. Is it necessary to add op_compat anyway?

Yes, it is necessary.
The extra attributes declared in fused_conv2d.pbtxt are used in Pass. The info of extra attributes is getted from op_compat.yaml when executor run the kernel, so it is needed to add op_compat for extra attributes.

… clear_conv2d_extra

zyfncg · 2023-01-04T11:54:39Z

@jczaja Is this PR need to run the tests?

jczaja · 2023-01-04T14:17:41Z

@zyfncg Hi, I just noticed this PR. Yes, I started our tests and will be able to share results within 2 days.

jczaja · 2023-01-05T10:23:38Z

@zyfncg We tested this PR and noticed performance problem on ERNIE 3.0 bf16 model on most recent platform

Results:

QPS (this PR): 122.81
QPS(develop: 23c1ac2): 154.50

Commandline

FLAGS_use_mkldnn=true python /root/models/PaddleNLP/model_zoo/ernie-3.0/infer.py --task_name tnews --model_path /data/PaddlePaddle/pp_models/ernie3.0/float32 --perf --device cpu --num_threads 1 --enable_bf16

zyfncg · 2023-01-06T12:18:04Z

@zyfncg We tested this PR and noticed performance problem on ERNIE 3.0 bf16 model on most recent platform

Results:

QPS (this PR): 122.81 QPS(develop: 23c1ac2): 154.50

Commandline

FLAGS_use_mkldnn=true python /root/models/PaddleNLP/model_zoo/ernie-3.0/infer.py --task_name tnews --model_path /data/PaddlePaddle/pp_models/ernie3.0/float32 --perf --device cpu --num_threads 1 --enable_bf16

@jczaja This command doesn't work in my local machine, where can I get the model file of ernie3.0/float32?

jczaja · 2023-01-09T15:51:11Z

@zyfncg We tested this PR and noticed performance problem on ERNIE 3.0 bf16 model on most recent platform

Results:

QPS (this PR): 122.81 QPS(develop: 23c1ac2): 154.50

Commandline

FLAGS_use_mkldnn=true python /root/models/PaddleNLP/model_zoo/ernie-3.0/infer.py --task_name tnews --model_path /data/PaddlePaddle/pp_models/ernie3.0/float32 --perf --device cpu --num_threads 1 --enable_bf16

@jczaja This command doesn't work in my local machine, where can I get the model file of ernie3.0/float32?

Ok. @yaomichael will help in providing model for you.

zyfncg · 2023-01-17T02:50:46Z

@zyfncg We tested this PR and noticed performance problem on ERNIE 3.0 bf16 model on most recent platform

Results:

QPS (this PR): 122.81 QPS(develop: 23c1ac2): 154.50

Commandline

FLAGS_use_mkldnn=true python /root/models/PaddleNLP/model_zoo/ernie-3.0/infer.py --task_name tnews --model_path /data/PaddlePaddle/pp_models/ernie3.0/float32 --perf --device cpu --num_threads 1 --enable_bf16

@jczaja This command doesn't work in my local machine, where can I get the model file of ernie3.0/float32?

Ok. @yaomichael will help in providing model for you.

@yaomichael Hi~ could you send the model file to my email: zhangyunfei07@baidu.com ?

zyfncg · 2023-01-31T09:05:47Z

@zyfncg We tested this PR and noticed performance problem on ERNIE 3.0 bf16 model on most recent platform

Results:

QPS (this PR): 122.81 QPS(develop: 23c1ac2): 154.50

Commandline

FLAGS_use_mkldnn=true python /root/models/PaddleNLP/model_zoo/ernie-3.0/infer.py --task_name tnews --model_path /data/PaddlePaddle/pp_models/ernie3.0/float32 --perf --device cpu --num_threads 1 --enable_bf16

@jczaja Does this performance problem only occur on the specific CPU platform? The results of performance test are same between this PR and develop on my local machine. Do I need a CPU platform which support bf16 to debug?

jczaja · 2023-01-31T10:04:20Z

@zyfncg We tested this PR and noticed performance problem on ERNIE 3.0 bf16 model on most recent platform

Results:

QPS (this PR): 122.81 QPS(develop: 23c1ac2): 154.50

Commandline

FLAGS_use_mkldnn=true python /root/models/PaddleNLP/model_zoo/ernie-3.0/infer.py --task_name tnews --model_path /data/PaddlePaddle/pp_models/ernie3.0/float32 --perf --device cpu --num_threads 1 --enable_bf16

@jczaja Does this performance problem only occur on the specific CPU platform? The results of performance test are same between this PR and develop on my local machine. Do I need a CPU platform which support bf16 to debug?

@zyfncg This performance regression was found on recently published Intel processor: Sapphire Rapids (SPR) as this one has hardware support of bf16 instructions. I do not have results from other processors. So I have started this test on other processors that you have to see if problem is also there (bf16 instructions are emulated there). I will update you once I got some results.

zyfncg · 2023-01-31T12:15:51Z

@zyfncg This performance regression was found on recently published Intel processor: Sapphire Rapids (SPR) as this one has hardware support of bf16 instructions. I do not have results from other processors. So I have started this test on other processors that you have to see if problem is also there (bf16 instructions are emulated there). I will update you once I got some results.

Thanks!

jczaja · 2023-02-03T10:52:30Z

@zyfncg This performance regression was found on recently published Intel processor: Sapphire Rapids (SPR) as this one has hardware support of bf16 instructions. I do not have results from other processors. So I have started this test on other processors that you have to see if problem is also there (bf16 instructions are emulated there). I will update you once I got some results.

Thanks!

@zyfncg We run more tests and found that actually this test ernie 3.0 got some issue (problem on our side). Your PR is fine according to our other tests. Please proceed with review and merge

zyfncg · 2023-02-03T11:35:22Z

@zyfncg This performance regression was found on recently published Intel processor: Sapphire Rapids (SPR) as this one has hardware support of bf16 instructions. I do not have results from other processors. So I have started this test on other processors that you have to see if problem is also there (bf16 instructions are emulated there). I will update you once I got some results.

Thanks!

@zyfncg We run more tests and found that actually this test ernie 3.0 got some issue (problem on our side). Your PR is fine according to our other tests. Please proceed with review and merge

@jczaja Thank you very much! This PR will be merged after resolving conflicts.

… clear_conv2d_extra

jiahy0825

LGTM

zyfncg added 3 commits December 21, 2022 07:21

remove extra input of conv2d

ef09b81

fix bug

837e9be

fix unittest bug

dc02b62

zyfncg force-pushed the clear_conv2d_extra branch from cccaafe to dc02b62 Compare December 21, 2022 07:29

zyfncg added 9 commits December 21, 2022 07:49

adjust conv2d.pbtxt

817ef4c

fix cpu_quantize_pass_tester

18ca252

revert use_addto of conv2d

47e6952

fix runtime attribute

8913298

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

8f39135

… clear_conv2d_extra

fix bug

07e1425

recover force_fp32_output in conv2d

5480b1a

refine error info

acd1b1d

fix bug

08b918e

chenwhql previously approved these changes Dec 27, 2022

View reviewed changes

zyfncg requested review from jczaja and Silv3S December 27, 2022 11:35

jiahy0825 previously approved these changes Dec 27, 2022

View reviewed changes

YuanRisheng previously approved these changes Dec 27, 2022

View reviewed changes

Silv3S previously approved these changes Dec 28, 2022

View reviewed changes

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

01a072f

… clear_conv2d_extra

jczaja closed this Feb 2, 2023

jczaja reopened this Feb 2, 2023

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

bdb9a0e

… clear_conv2d_extra

zyfncg dismissed stale reviews from Silv3S, YuanRisheng, jiahy0825, and chenwhql via bdb9a0e February 5, 2023 15:44

Silv3S approved these changes Feb 5, 2023

View reviewed changes

jiahy0825 approved these changes Feb 6, 2023

View reviewed changes

YuanRisheng approved these changes Feb 6, 2023

View reviewed changes

chenwhql approved these changes Feb 6, 2023

View reviewed changes

zyfncg merged commit 2deada9 into PaddlePaddle:develop Feb 6, 2023

zyfncg deleted the clear_conv2d_extra branch February 6, 2023 09:02

wanghuancoder mentioned this pull request Dec 12, 2023

[PIR] delete Bias for onednn conv2d_grad #59917

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Delete extra input (Bias, ResidualData) in OpMaker of conv2d #49121

Delete extra input (Bias, ResidualData) in OpMaker of conv2d #49121

zyfncg commented Dec 16, 2022 •

edited

Loading

paddle-bot bot commented Dec 16, 2022

Silv3S left a comment

Silv3S Dec 28, 2022

zyfncg Dec 28, 2022 •

edited

Loading

Silv3S Dec 28, 2022

Silv3S Dec 28, 2022

zyfncg Dec 28, 2022

zyfncg commented Jan 4, 2023

jczaja commented Jan 4, 2023

jczaja commented Jan 5, 2023 •

edited

Loading

zyfncg commented Jan 6, 2023

Results:

Commandline

jczaja commented Jan 9, 2023

Results:

Commandline

zyfncg commented Jan 17, 2023

Results:

Commandline

zyfncg commented Jan 31, 2023

Results:

Commandline

jczaja commented Jan 31, 2023

Results:

Commandline

zyfncg commented Jan 31, 2023

jczaja commented Feb 3, 2023

zyfncg commented Feb 3, 2023

jiahy0825 left a comment

Delete extra input (Bias, ResidualData) in OpMaker of conv2d #49121

Delete extra input (Bias, ResidualData) in OpMaker of conv2d #49121

Conversation

zyfncg commented Dec 16, 2022 • edited Loading

PR types

PR changes

Describe

paddle-bot bot commented Dec 16, 2022

Silv3S left a comment

Choose a reason for hiding this comment

Silv3S Dec 28, 2022

Choose a reason for hiding this comment

zyfncg Dec 28, 2022 • edited Loading

Choose a reason for hiding this comment

Silv3S Dec 28, 2022

Choose a reason for hiding this comment

Silv3S Dec 28, 2022

Choose a reason for hiding this comment

zyfncg Dec 28, 2022

Choose a reason for hiding this comment

zyfncg commented Jan 4, 2023

jczaja commented Jan 4, 2023

jczaja commented Jan 5, 2023 • edited Loading

Results:

Commandline

zyfncg commented Jan 6, 2023

Results:

Commandline

jczaja commented Jan 9, 2023

Results:

Commandline

zyfncg commented Jan 17, 2023

Results:

Commandline

zyfncg commented Jan 31, 2023

Results:

Commandline

jczaja commented Jan 31, 2023

Results:

Commandline

zyfncg commented Jan 31, 2023

jczaja commented Feb 3, 2023

zyfncg commented Feb 3, 2023

jiahy0825 left a comment

Choose a reason for hiding this comment

zyfncg commented Dec 16, 2022 •

edited

Loading

zyfncg Dec 28, 2022 •

edited

Loading

jczaja commented Jan 5, 2023 •

edited

Loading