Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add weight_only_linear_grad kernel to support weight_only_linear backward #57685

Merged

Conversation

yuanlehome
Copy link
Contributor

PR types

New features

PR changes

APIs

Description

  • add weight_only_linear_grad to support weight_only_liear backward
  • 为weight_only_linear添加backward算子,因为weight_only中weight更新困难,backward中只计算输入x的grad
  • 修复amp中输入x与weight_scale被强制转换为同一dtype的问题

Others

Pcard-71500

@paddle-bot
Copy link

paddle-bot bot commented Sep 25, 2023

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@yuanlehome yuanlehome changed the title add weight_only_linear_grad to support weight_only_liear backward add weight_only_linear_grad kernel to support weight_only_linear backward Sep 25, 2023
Comment on lines +1 to +15
/*
* Copyright (c) 2022-2023, NVIDIA CORPORATION. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里需要加这个licenses吗?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个我确认下,下面有个相关的PR我一并修改下

Comment on lines +1 to +10
/* Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. */
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

格式看着有点问题

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个我确认下,下面有个相关的PR我再修改下

@yuanlehome
Copy link
Contributor Author

yuanlehome commented Sep 27, 2023

下个PR TODO:

  • 添加单测
  • fix comment

@yuanlehome yuanlehome merged commit 3ce66d5 into PaddlePaddle:develop Sep 27, 2023
27 checks passed
@yuanlehome
Copy link
Contributor Author

下个PR TODO:

  • 添加单测
  • fix comment

done. #57844

Frida-a pushed a commit to Frida-a/Paddle that referenced this pull request Oct 14, 2023
…ward (PaddlePaddle#57685)

* add weight_only_linear_grad to support weight_only_liear backward

* update

---------

Co-authored-by: xiaoxiaohehe001 <hiteezsf@163.com>
jiahy0825 pushed a commit to jiahy0825/Paddle that referenced this pull request Oct 16, 2023
…ward (PaddlePaddle#57685)

* add weight_only_linear_grad to support weight_only_liear backward

* update

---------

Co-authored-by: xiaoxiaohehe001 <hiteezsf@163.com>
danleifeng pushed a commit to danleifeng/Paddle that referenced this pull request Nov 14, 2023
…ward (PaddlePaddle#57685)

* add weight_only_linear_grad to support weight_only_liear backward

* update

---------

Co-authored-by: xiaoxiaohehe001 <hiteezsf@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants