optimization of depthwise_conv2d grad

[s5u13b]

PR types

Performance optimization

PR changes

OPs

Describe

• Environment:
  • V100-32G, CUDA 11.2, cuDNN 8
• Feature：
  • For depthwise_conv2d grad in nchw format, replace the unnessary atomic add operation with if and assign operation. It's unnessary to use atomic add operation for each fliter grad element, because the gradient of each fliter element is computed by one thread block according to the gpu launch config.
• Performance of depthwise_conv2d grad in nchw format after optimization (OP Benchmark):

Config ID	Perf Before(ms)	Perf After(ms)	Improvement	Perf Pytorch(ms)	Outperform
0	3.3735	1.1366	196.8%	1.4876	30.9%
2	5.0860	1.9956	154.9%	2.7723	38.9%
4	1.9932	0.9705	105.4%	1.0652	9.8%
6	1.1021	0.9002	22.4%	1.2308	36.7%
8	1.7422	1.3151	32.5%	2.6702	103.0%
10	1.4936	1.1460	30.3%	2.1160	84.6%
12	3.5876	2.1562	66.4%	3.8364	77.9%

• Performance of depthwise_conv2d compared with pytorch (OP Benchmark):

Config ID	Perf Paddle(ms)	Perf Pytorch(ms)	Outperform
0	2.7242	3.6453	33.8%
1	2.7587	-	-
2	4.6421	7.0541	52.0%
3	7.2652	-	-
4	2.0535	2.6483	29.0%
5	2.6485	-	-
6	3.3847	4.4502	31.5%
7	3.0287	-	-
8	3.0288	5.1939	71.5%
9	4.1266	-	-
10	2.6863	4.2438	58.0%
11	2.9651	-	-
12	5.0887	8.2912	62.9%
13	5.8193	-	-

The depthwise_conv2d of pytorch use 1D gpu launch config, which introduces significant overhead of index computation. Meawhile, paddle use 3D gpu launch config, therefore the depthwise_conv2d performance of paddle can outperform pytorch.

[paddle-bot]

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[paddle-bot]

✅ This PR's description meets the template requirements!
Please wait for other CI results.

[JamesLim-sy]

Good work!