Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Paddle Inference] Move down the transfer_layout #52997

Merged
merged 20 commits into from
May 24, 2023

Conversation

zhoutianzi666
Copy link
Contributor

@zhoutianzi666 zhoutianzi666 commented Apr 17, 2023

PR types

Others

PR changes

Others

Description

  • fp16计算中会插入大量的transfer_layout用于促使cuDNN调用TensorCore, 这个PR完成transfer_layout的部分消除

解决思路:

  • 首先对于图中每个op类节点
    • 调用AllInputIsTransferlayout(const Node *op_node),这个函数判断此op_node的输入是否全是transfer_layout节点
    • 如果是的话,分两种情况
      • 如果op_node也是transfer_layout 节点的话,就会调用ElimTwoTranferlayout将这两个节点抵消掉
      • 否则就调用PutTranferlayoutAfterOp将输入的transfer_layout 节点下移动到此op_node的输出上

@paddle-bot
Copy link

paddle-bot bot commented Apr 17, 2023

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@paddle-bot
Copy link

paddle-bot bot commented Apr 17, 2023

❌ The PR is not created using PR's template. You can refer to this Demo.
Please use PR's template, it helps save our maintainers' time so that more developers get helped.

@paddle-ci-bot
Copy link

paddle-ci-bot bot commented Apr 29, 2023

Sorry to inform you that fe0b7ce's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

@zhoutianzi666 zhoutianzi666 changed the title add tranfer_elim [Paddle Inference] Move down the transfer_layout May 16, 2023
act_like_ops.end();
// For these Ops, you can move down the transfer_layout, but MUST change
// the data_format attribute!
std::vector<std::string> pool_like_ops = {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

对于这类OP也是可以下移的,只不过可能会改变这个Op的执行性能,因此这里只是做一个示范

@@ -0,0 +1,323 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

改成2023

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

改成2023

done

YuanRisheng
YuanRisheng previously approved these changes May 23, 2023
Copy link
Contributor

@zhangjun zhangjun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment on lines 249 to 253
std::vector<std::string> act_like_ops = {
"elementwise_add",
"hard_swish",
"silu",
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

后面可以增加更多激活

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

后面可以增加更多激活

可以,后面遇到的case里可以都加入。

@zhangjun zhangjun merged commit 44044d8 into PaddlePaddle:develop May 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants