[Semi Auto] Refactor Replicated Rule #56839
Conversation
|
你的PR提交成功，感谢你对开源项目的贡献! |
|
❌ The PR is not created using PR's template. You can refer to this Demo. |
…o semi-auto/default-rule
| @@ -375,6 +376,44 @@ void BindAutoParallel(py::module *m) { | |||
| } | |||
| return self.InferForward(ctx); | |||
| }) | |||
| .def("infer_forward", /*for op that have vector argument*/ | |||
| [](const phi::distributed::SpmdRule &self, | |||
| const std::vector<std::pair<int, int>> input_ranges, | |||
| paddle::small_vector<phi::distributed::DistMetaTensor, | ||
| phi::kInputSmallVectorSize> | ||
| ins; | ||
| for (auto range : input_ranges) { |
| * all the input and ouput tensors is the batch dimension (broadcast dimension), | ||
| * therefore, if any tensor's first dimension is sharded, the sharding would be | ||
| * propagating to all the other tensors (for tensor first dimension). All the | ||
| *other axes of tensors would be set as unshard (-1). |
|
|
||
| #include "paddle/phi/core/distributed/auto_parallel/dist_meta_tensor.h" | ||
| #include "paddle/phi/core/distributed/type_defs.h" | ||
| #include "paddle/phi/infermeta/spmd_rules/utils.h" |
There was a problem hiding this comment.
utils.h是不是在.cc中include就可以了
There was a problem hiding this comment.
不太行。因为下面phi 接口方法的定义中，需要把 helper template 实例化，所以必须include helper template 的实现，无法通过 forward declaration 引入 helper。
| const std::vector<phi::Attribute> &attrs) { | ||
| /* | ||
| to distingish between single tensor argument and vector argument of | ||
| one tensor: start - end == 0: single tensor start - end == 1: |
There was a problem hiding this comment.
Commonly, the range is [start, end), i.e., start is inclusive and end is exclusive. Maybe wo need another way to distingish between tensor and [tensor],
There was a problem hiding this comment.
Yes，we plan to use PyObject* to deal with variadic arguments Python end and parser them in C++ end in Next PR
| inputs_.emplace_back(std::move(input)); | ||
| int index = static_cast<int>(inputs_.size()); |
There was a problem hiding this comment.
exchange line 21 and 22 to get right index?
There was a problem hiding this comment.
fixed, it is a bug, thx~
* adapt general spmd rule * polish details * add new rules * bugfix for set_partial * bugfix * unitest * adapt for argument for tensor and vector of tensor --------- Co-authored-by: Chen Weihang <chenweihang@baidu.com>
PR types
Function optimization
PR changes
Others
Description
Pcard-70448