【Hackathon 5th No.52】为 Paddle 新增 squeeze 和 unsqueeze 的 spmd 切分推导规则 #57877

WintersMontagne10335 · 2023-10-04T15:05:21Z

PR types

Others

PR changes

Others

Description

为 Paddle 新增 squeeze 和 unsqueeze 的 spmd 切分推导规则
#57262

paddle-bot · 2023-10-04T15:05:26Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

WintersMontagne10335 · 2023-10-06T14:53:57Z

@Ligoml 梦老师您好，能不能先让审核老师看一下这个PR的现存问题呀？我遇到瓶颈，进行不下去了 /大哭

pkuzyc · 2023-10-11T02:57:10Z

@WintersMontagne10335 回复你的两个问题：

本地好像没法看CI-coverage。单测的话注册了规则之后直接在本地运行 python 单测代码就可以：python test_xxx.py。写了单测文件之后需要在 Paddle/test/auto_parallel/spmd_rules/CMakeLists.txt 里加一下，这个文档里漏写了，在另外两个 pr 里也一样加一下吧。
这个想法是对的。这类规则确实可以看成是 reshape 的特例。但是 reshape 的函数比较复杂，都调这个函数的话对效率可能会有一定的影响。目前还是单独写吧，除了效率高些单独写对不同的算子规则也更加清楚些。

paddle-ci-bot · 2023-10-14T03:07:39Z

Sorry to inform you that 6c2f23f's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

WintersMontagne10335 · 2023-10-15T01:01:41Z

@pkuzyc
老师您好，还有两个问题想向您请教一下。

单测运行不了。在ai studio框架开发环境中，Paddle/build/test/auto_parallel/spmd_rules下，运行python test_reshape_rule.py，结果如下：
squeeze如何判断没有指定 axis。目前的实现是判断axis.size()是否为0，这种实现可以吗？
相关API：https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/squeeze_cn.html

WintersMontagne10335 · 2023-10-15T06:24:39Z

@pkuzyc
老师您好。目前情况是，flatten、unsqueeze ci过了，squeeze、slice有段错误。但是因为本地单测不能用，只能到ci上去测，所以调试又麻烦又慢。老师有什么比较好的解决方法没？

pkuzyc · 2023-10-16T02:40:16Z

@WintersMontagne10335 单测跑不了是因为上面的报错吗？这个报错看上去是规则没有注册，是不是没有用最新的 develop 编译。reshape 的规则是已经合入了的，重新源码编译、安装 python 包之后应该不会报这个错。

WintersMontagne10335 · 2023-10-16T03:07:07Z

@pkuzyc 一直是用的最新的源码编译安装的，也一直有这个规则没注册的问题。除了这个问题之外，sequeeze.cc里面代码结构有问题，编译的时候也不会报错（ #57870 ）。

WintersMontagne10335 · 2023-10-17T01:06:06Z

@pkuzyc 一直是用的最新的源码编译安装的，也一直有这个规则没注册的问题。除了这个问题之外，sequeeze.cc里面代码结构有问题，编译的时候也不会报错（ #57870 ）。

试了下ai studio框架开发双卡环境，还是报一样的错误。

WintersMontagne10335 · 2023-10-18T03:41:38Z

找到问题，已经解决了。需要在cmake的时候开启-DWITH_DISTRIBUTE=ON：
cmake .. -DPY_VERSION=3.8 -DWITH_GPU=ON -DWITH_TESTING=ON -DWITH_DISTRIBUTE=ON
然后在Paddle/build路径下运行ctest即可，如：
ctest -R test_reshape_rule -V

pkuzyc · 2023-10-19T08:29:00Z

@WintersMontagne10335 看看能不能把 squeeze 和 unsqueeze 分开提？一个 pr 内容有点多

WintersMontagne10335 · 2023-10-19T09:21:37Z

@pkuzyc 收到

WintersMontagne10335 · 2023-10-22T02:09:27Z

@pkuzyc 老师您好，这个PR也可以review一下。
paddle目前存在一个BUG：切分推导规则的输入参数，不支持空list，导致这个PR的Line Coverage不够高。
目前PR的代码结构应该没问题了。那个Bug解决之后，CI-Coverage就可以通过了。

pkuzyc

输入axis为空的情况我们看一下

pkuzyc · 2023-10-24T02:52:45Z

paddle/phi/infermeta/spmd_rules/squeeze.cc

+void MakeSqueezeDimTransWithAxis(const std::vector<int64_t>& x_shape,
+                                 std::vector<int64_t>* out_shape,
+                                 const std::vector<int64_t>& axis,
+                                 std::vector<DimTrans*>* trans) {


应该可以写得更简单些，先算出输出的size，然后遍历输出的维度，如果某一维在 axis 里且 shape 是 1就用Singleton，否则用 InputDim。

pkuzyc · 2023-10-24T02:57:23Z

paddle/phi/infermeta/spmd_rules/squeeze.cc

+bool SqueezeCompare(const int64_t& a, const int64_t& b) { return a > b; }
+
+bool SqueezeReverseCompare(const int64_t& a, const int64_t& b) { return a < b; }
+


不需要自己定义比较函数，sort对vector排序的时候默认从小到大排，标准库有自带的greater函数从大到小排

根据老师的 review ，简化了 MakeSqueezeDimTransWithAxis 的实现逻辑，已不需要排序

pkuzyc · 2023-10-24T02:59:19Z

paddle/phi/infermeta/spmd_rules/squeeze.cc

+        axis_copy[i] += x_ndim;
+      }
+    }
+    std::sort(axis_copy.begin(), axis_copy.end(), SqueezeCompare);


去掉自己定义的比较函数

pkuzyc · 2023-10-24T03:00:20Z

paddle/phi/infermeta/spmd_rules/squeeze.cc

+        axis_copy[i] += x_ndim;
+      }
+    }
+    std::sort(axis_copy.begin(), axis_copy.end(), SqueezeReverseCompare);


用标准库自带的比较函数

pkuzyc · 2023-10-24T03:11:16Z

test/auto_parallel/spmd_rules/test_squeeze_rule.py

+        self.x_dist_tensor_spec = DistTensorSpec(x_shape, x_tensor_dist_attr)
+        self.attrs = OrderedDict()
+
+    def test_squeeze_infer_forward(self):


补一个 '1' 的维度被切的单测，看看推导完有没有变成-1，例如：
[1, 8, 1, 16] --> [8, 1, 16]
[-1, 0, 1, -1] --> [-1, 0, -1, -1], [0, -1, -1]

Done。本地看了，原先的实现是有问题的，已修改，目前推导完变成-1了。

WintersMontagne10335 · 2023-10-29T23:56:29Z

@pkuzyc 老师您好，unsqueeze CI已过，可以review了。

unsqueeze
【Hackathon 5th No.52】为 Paddle 新增 unsqueeze 的 spmd 切分推导规则 -part #58296

paddle-ci-bot · 2023-11-05T03:16:48Z

Sorry to inform you that 9f95328's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

luotao1 · 2023-11-06T06:51:35Z

@WintersMontagne10335 unsqueeze合入了，可以更新squeeze 了

luotao1 · 2023-11-24T02:21:09Z

切分推导规则的输入参数，不支持空list #58297

pkuzyc · 2023-11-24T02:37:30Z

@WintersMontagne10335 最新一个pr把推导规则里用指针的地方改成智能指针了，需要按最新的改下：#59101

输入空list的情况我这两天再看看，要是支持起来比较复杂这个pr可以先合

WintersMontagne10335 · 2023-11-24T02:44:30Z

@WintersMontagne10335 最新一个pr把推导规则里用指针的地方改成智能指针了，需要按最新的改下：#59101

输入空list的情况我这两天再看看，要是支持起来比较复杂这个pr可以先合

收到！我今天改一下哈。
我把空list的测试用例注释掉了，不知道到时候可不可以申请CI Coverage豁免。

WintersMontagne10335 · 2023-11-24T08:35:42Z

@pkuzyc Done

pkuzyc

一些小地方再修改一下，之前 slice (#57866) 里的一些没有修改的看看也一起改了吧

pkuzyc · 2023-11-24T09:09:18Z

paddle/phi/infermeta/spmd_rules/squeeze.cc

+    if (x_shape[i] == 1) {
+      auto it = find(axis.begin(), axis.end(), i);
+      if (it == axis.end()) {
+        trans->emplace_back(new Singleton());


这里改成 make_shared

pkuzyc · 2023-11-24T09:10:03Z

paddle/phi/infermeta/spmd_rules/squeeze.cc

+    if (x_shape[i] != 1) {
+      trans->emplace_back(std::make_shared<InputDim>(j++));
+    } else {
+      trans->emplace_back(new Singleton());


改成 make_shared

pkuzyc · 2023-11-24T09:15:45Z

paddle/phi/infermeta/spmd_rules/squeeze.cc

+  for (int64_t i = 0, j = 0, n = static_cast<int64_t>(x_shape.size()); i < n;
+       i++) {
+    if (x_shape[i] == 1) {
+      trans->emplace_back(new Singleton());


改成 make_shared

pkuzyc · 2023-11-24T09:18:35Z

paddle/phi/infermeta/spmd_rules/squeeze.cc

+  VLOG(4) << "Transformation from input to output:";
+  for (int64_t i = 0, n = static_cast<int64_t>(trans.size()); i < n; i++) {
+    std::shared_ptr<DimTrans> t = trans[i];
+    VLOG(4) << "\tOut axis[" << i << "]: " << t->to_string();


直接输出 trans[i]->to_string() 就行，不用多一个赋值

pkuzyc · 2023-11-24T09:19:11Z

paddle/phi/infermeta/spmd_rules/squeeze.cc

+  VLOG(4) << "Transformation from output to input:";
+  for (int64_t i = 0, n = trans.size(); i < n; i++) {
+    std::shared_ptr<DimTrans> t = trans[i];
+    VLOG(4) << "\tX axis[" << i << "]: " << t->to_string();


同上，直接用 trans[i]->to_string() 就行

WintersMontagne10335 · 2023-11-24T12:42:33Z

一些小地方再修改一下，之前 slice (#57866) 里的一些没有修改的看看也一起改了吧

收到！slice 周末修掉。
之前比较忙，所以 slice 一直拖着没改~~

pkuzyc

LGTM

zhiqiu

LGTM

…addlePaddle#57877) * Add spmd segmentation and derivation rules for squeeze to Paddle * Add spmd segmentation derivation rule for unsqueeze to Paddle * fix bugs * fix bugs * fix bugs * fix bugs * Add unit test code * modify squeeze.cc and CMakeLists.txt * write separate rules * fix bugs * fix bugs * fix bugs * remove unsqueeze spmd rule * modified: test/auto_parallel/spmd_rules/test_squeeze_rule.py * re-run CI * fix bugs * modify pointer to smart pointer * fix bugs * fix bugs

WintersMontagne10335 added 2 commits October 4, 2023 09:39

Add spmd segmentation and derivation rules for squeeze to Paddle

e326aee

Add spmd segmentation derivation rule for unsqueeze to Paddle

148925a

paddle-bot bot added the contributor External developers label Oct 4, 2023

WintersMontagne10335 added 5 commits October 4, 2023 16:21

fix bugs

fd3f1db

fix bugs

4012a49

fix bugs

4aff5eb

fix bugs

9e9140f

Add unit test code

6c2f23f

Ligoml mentioned this pull request Oct 7, 2023

【PaddlePaddle Hackathon 5th】开源贡献个人挑战赛 #57262

Open

luotao1 added the PaddlePaddle Hackathon label Oct 9, 2023

luotao1 assigned luotao1 and pkuzyc Oct 9, 2023

WintersMontagne10335 added 3 commits October 13, 2023 06:01

modify squeeze.cc and CMakeLists.txt

b727efe

Merge remote-tracking branch 'upstream/develop' into winters009

1efc51d

Merge remote-tracking branch 'upstream/develop' into winters009

98af5a4

WintersMontagne10335 added 4 commits October 14, 2023 08:41

write separate rules

34b7024

Merge remote-tracking branch 'upstream/develop' into winters009

32e0ed8

fix bugs

80c07b2

fix bugs

f13fcdd

fix bugs

8acaa6d

Merge remote-tracking branch 'upstream/develop' into winters009

504006f

WintersMontagne10335 mentioned this pull request Oct 21, 2023

【Hackathon 5th No.52】为 Paddle 新增 unsqueeze 的 spmd 切分推导规则 -part #58296

Merged

WintersMontagne10335 added 3 commits October 21, 2023 05:40

remove unsqueeze spmd rule

97452fb

Merge remote-tracking branch 'upstream/develop' into winters009

c129162

modified: test/auto_parallel/spmd_rules/test_squeeze_rule.py

63983b9

WintersMontagne10335 mentioned this pull request Oct 21, 2023

切分推导规则的输入参数，不支持空list #58297

Closed

re-run CI

1fbb2cf

pkuzyc reviewed Oct 24, 2023

View reviewed changes

WintersMontagne10335 added 2 commits October 28, 2023 08:37

fix bugs

ea1e6fc

Merge remote-tracking branch 'upstream/develop' into winters009

9f95328

Merge remote-tracking branch 'upstream/develop' into winters009

f57ee3a

WintersMontagne10335 added 2 commits November 24, 2023 03:30

modify pointer to smart pointer

5301bdc

fix bugs

6114774

pkuzyc reviewed Nov 24, 2023

View reviewed changes

fix bugs

dd3d96c

pkuzyc approved these changes Nov 27, 2023

View reviewed changes

zhiqiu approved these changes Nov 27, 2023

View reviewed changes

zhiqiu merged commit 708acf0 into PaddlePaddle:develop Nov 27, 2023
28 checks passed

		bool SqueezeCompare(const int64_t& a, const int64_t& b) { return a > b; }

		bool SqueezeReverseCompare(const int64_t& a, const int64_t& b) { return a < b; }

【Hackathon 5th No.52】 为 Paddle 新增 squeeze 和 unsqueeze 的 spmd 切分推导规则 #57877

【Hackathon 5th No.52】 为 Paddle 新增 squeeze 和 unsqueeze 的 spmd 切分推导规则 #57877

Conversation

WintersMontagne10335 commented Oct 4, 2023 • edited Loading

PR types

PR changes

Description

paddle-bot bot commented Oct 4, 2023

WintersMontagne10335 commented Oct 6, 2023

pkuzyc commented Oct 11, 2023 • edited Loading

paddle-ci-bot bot commented Oct 14, 2023

WintersMontagne10335 commented Oct 15, 2023

WintersMontagne10335 commented Oct 15, 2023

pkuzyc commented Oct 16, 2023

WintersMontagne10335 commented Oct 16, 2023

WintersMontagne10335 commented Oct 17, 2023

WintersMontagne10335 commented Oct 18, 2023

pkuzyc commented Oct 19, 2023

WintersMontagne10335 commented Oct 19, 2023

WintersMontagne10335 commented Oct 22, 2023

pkuzyc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WintersMontagne10335 commented Oct 29, 2023

paddle-ci-bot bot commented Nov 5, 2023

luotao1 commented Nov 6, 2023

luotao1 commented Nov 24, 2023

pkuzyc commented Nov 24, 2023 • edited Loading

WintersMontagne10335 commented Nov 24, 2023

WintersMontagne10335 commented Nov 24, 2023

pkuzyc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WintersMontagne10335 commented Nov 24, 2023

pkuzyc left a comment

Choose a reason for hiding this comment

zhiqiu left a comment

Choose a reason for hiding this comment

【Hackathon 5th No.52】为 Paddle 新增 squeeze 和 unsqueeze 的 spmd 切分推导规则 #57877

【Hackathon 5th No.52】为 Paddle 新增 squeeze 和 unsqueeze 的 spmd 切分推导规则 #57877

WintersMontagne10335 commented Oct 4, 2023 •

edited

Loading

pkuzyc commented Oct 11, 2023 •

edited

Loading

pkuzyc commented Nov 24, 2023 •

edited

Loading