Support control flow for static build [Step 2: support conditional_block] #56696

AndSonder · 2023-08-28T02:21:04Z

PR types

Performance optimization

PR changes

Others

Description

当下 conditional_block 是直接采用跑算子的方法，本 PR 的核心目的是将 conditional_block 加入到 OperatorBasesHandledInStaticBuild 中并使其支持 static_build

相关Issue：

静态图执行器预分析性能优化 #55299

paddle-bot · 2023-08-28T02:21:29Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

… control_flow_support-step-2

…he last op in the block

… control_flow_support-step-2

From00 · 2023-09-11T03:02:48Z

test/legacy_test/CMakeLists.txt

@@ -1272,6 +1272,8 @@ set_tests_properties(
 set_tests_properties(
  test_cuda_graph_static_mode_error
  PROPERTIES ENVIRONMENT "FLAGS_CUDA_GRAPH_USE_STANDALONE_EXECUTOR=1")
+set_tests_properties(test_conditional_block


加个注释说明下这个单测的情况吧

From00 · 2023-09-11T03:05:04Z

paddle/fluid/framework/new_executor/interpreter/static_build.h

@@ -14,6 +14,7 @@

 #pragma once

+#include "paddle/fluid/framework/new_executor/new_executor_defs.h"


这个头文件中的符号在static_build.h中不需要使用，加到static_build.cc中防止头文件污染。

From00 · 2023-09-11T03:08:28Z

paddle/fluid/framework/new_executor/program_interpreter.cc

+                  !FLAGS_new_executor_use_cuda_graph &&
+                  interpreter::BlockCanBeStaticBuilt(block);
+
+  for (auto& op : block.AllOps()) {


这个逻辑合并到BlockCanBeStaticBuilt里吧

From00 · 2023-09-11T03:15:19Z

paddle/fluid/framework/new_executor/interpreter/static_build.cc

@@ -361,6 +365,37 @@ void FakeInitializeOutputsForOperatorBase(const OperatorBase& op,
            *dev_ctx, target_place, dtype, out_tensor->layout(), out_tensor);
      }
    }
+  } else if (op_type == "conditional_block") {


多个并行分支处理不同OP，建议按OP名称字典序排序

From00 · 2023-09-11T03:25:33Z

paddle/fluid/framework/new_executor/program_interpreter.cc

@@ -185,6 +196,34 @@ FetchList ProgramInterpreter::Run(const std::vector<std::string>& feed_names,
  }
 }

+void ProgramInterpreter::PreStaticBuild() {
+  SetDeviceId(place_);
+#ifdef PADDLE_WITH_DNNL


这段代码在Run函数里有类似的实现，建议合并成一处，统一实现成Build接口

From00 · 2023-09-11T03:45:33Z

paddle/fluid/framework/operator.h

@@ -104,6 +104,27 @@ constexpr char kEnableCacheRuntimeContext[] = "@ENABLE_CACHE_RUNTIME_CONTEXT@";
 constexpr char kAllKernelsMustComputeRuntimeShape[] =
    "@ALL_KERNELS_MUST_COMPUTE_RUNTIME_SHAPE@";

+struct VarInfo {
+  std::string name_;
+  std::string dtype_;


dtype和place类型建议用框架定义的数据类型，不用std::string。

这里用string应该可以更好的兼容没有init的情况，像 GetDtype 这类接口的返回值也是 string，用string也更方便后面打印debug log

From00 · 2023-09-11T03:49:47Z

paddle/fluid/framework/operator.cc

@@ -953,6 +953,66 @@ std::string OperatorBase::DebugStringEx(const Scope* scope) const {
  return ss.str();
 }

+std::vector<VarInfo> OperatorBase::InputVarsInfo(const Scope* scope) const {


InputVarsInfo和OutputVarsInfo代码逻辑非常类似，考虑重复代码是否可以合并成一处，一些差异化的信息通过函数参数传入，减少重复代码。no_need_buffer_vars的处理，也是适用于output的。

From00 · 2023-09-11T03:56:26Z

paddle/fluid/framework/new_executor/interpreter/interpreter_util.cc

+        bool is_skip_fake_init = false;
+        std::unordered_set<std::string> following_input_vars;
+
+        if (static_build && op->Type() == "conditional_block") {


这个逻辑代码建议放到HandleOperatorBase处理控制流算子的分支中，而不需要在这里处理一些控制流算子特有的信息后再通过函数参数传入。

From00 · 2023-09-11T04:05:55Z

paddle/fluid/framework/new_executor/interpreter/static_build.cc

+
+    // Note(sonder): static_build is not supported if the output of
+    // conditional_block is changed after static build.
+    if (out_var_info_before_build.size() != out_var_info_after_build.size()) {


在什么情况下会出现static_build前后var数量不一致？

没遇到过这种情况，现在已经删除了这个判断

From00 · 2023-09-11T04:06:22Z

paddle/fluid/framework/new_executor/interpreter/static_build.cc

+        if (out_var_info_before_build[i] != out_var_info_after_build[i]) {
+          auto var_name = out_var_info_before_build[i].name_;
+          if (following_input_vars.count(var_name)) {
+            PADDLE_THROW(phi::errors::PreconditionNotMet(


建议在报错信息中强调下是输出的dtype或place发生了改变，直接写output改变容易被误解成数据改变。另外，建议报错信息中输出具体改变的变量名称，以及out_var_info_before_build和out_var_info_after_build信息，在用户出错时容易通过log判断原因。

… control_flow_support-step-2

From00 · 2023-09-18T02:41:38Z

paddle/fluid/framework/new_executor/interpreter/static_build.cc

    if (!OpsCanSkipedFakeAllocInStaticBuild.count(op_type)) {
      if (in_black_list ||
          (is_operator_base &&
           !OperatorBasesHandledInStaticBuild.count(op_type)) ||
-          is_custom_op || use_mkldnn) {
+          is_custom_op || use_mkldnn || !is_sub_block_static_build) {


建议在下面The following OPs are unable to static build:的log里也加上sub_block_static_build的信息输出，方便通过log判断static_build是因为哪种情况退化的。另外，为了和其它情况统一，布尔值为True表示满足不支持的情况，建议将is_sub_block_static_build改成sub_block_can_not_static_build。

From00 · 2023-09-18T03:02:20Z

paddle/fluid/framework/new_executor/interpreter/interpreter_util.cc

+    bool is_skip_fake_init = false;
+    std::unordered_set<std::string> following_input_vars;
+
+    if (op->Type() == "conditional_block") {


这里的逻辑好像没改？这个逻辑代码建议放到HandleOperatorBase处理控制流算子的分支中，而不需要在这里处理一些控制流算子特有的信息后再通过函数参数传入。

这个已经是在HandleOperatorBase中了吧，之前是在 HandleOperatorBase 外处理好传入进 HandleOperatorBase 中的

From00 · 2023-09-18T03:05:55Z

paddle/fluid/framework/new_executor/interpreter/static_build.cc

+          PADDLE_THROW(phi::errors::PreconditionNotMet(
+              "The outputs' dtype/place of conditional_block is "
+              "changed after static build. Befer static build, the "
+              "output %s's dtype is %s, place is %s. After static "
+              "build, the output %s's dtype is %s, place is %s.",
+              var_name,
+              out_var_info_before_build[i].dtype_,
+              out_var_info_before_build[i].place_,
+              var_name,
+              out_var_info_after_build[i].dtype_,
+              out_var_info_after_build[i].place_));


Suggested change

PADDLE_THROW(phi::errors::PreconditionNotMet(

"The outputs' dtype/place of conditional_block is "

"changed after static build. Befer static build, the "

"output %s's dtype is %s, place is %s. After static "

"build, the output %s's dtype is %s, place is %s.",

var_name,

out_var_info_before_build[i].dtype_,

out_var_info_before_build[i].place_,

var_name,

out_var_info_after_build[i].dtype_,

out_var_info_after_build[i].place_));

PADDLE_THROW(phi::errors::PreconditionNotMet(

"The output %s s' dtype/place of conditional_block is "

"changed after static build. Befer static build, the "

"dtype is %s, place is %s. After static "

"build, the dtype is %s, place is %s.",

var_name,

out_var_info_before_build[i].dtype_,

out_var_info_before_build[i].place_,

out_var_info_after_build[i].dtype_,

out_var_info_after_build[i].place_));

From00

LGTM

…ock] (PaddlePaddle#56696) * add conditional_block to OperatorBasesHandledInStaticBuild * run op in FakeInitializeOutputsForOperatorBase * add init_success judge * fix build error * fix * add SetSubBlockCore func * add PreStaticRun func * add PreStaticRun to interpreter_base and new_ir_inter * recover codes * add PreStaticBuild and BlockCanBeStaticBuilt * fix logic about RunPreStaticBuild * change CreateOpFromOpDesc type * fix build error * fix build error * remove IsOperatorBasesHandledInStaticBuild * recover BlockCanBeStaticBuilt * add logic about conditional_block run static build * recover codes * recover BlockCanBeStaticBuilt * support static build condational block op when condational block is the last op in the block * fix error * fix logic about last op * fit for sub block can't open static build * add IsStaticBuild * fix build error * fit logic when sub block can't open static build * close static build when sub_block don't support static_build * recover third party * add is_skil_fake_init logic * set the backend of the lamb * change start index * add if conditional for cal is_skip_fake_init * change name * close static_build for test_conditional_block * add static buiild support for conditional block in case of the output's dtype/place is changed but the following op is not use this output * fix logic error * fix timeout error * fix * remove useless codes * fix * fix * fix build error * move GetVarsInfo and RunPreStaticBuild from opeartor to static_build * fix lamb backend registe * fix build error * fix build error * remove lamp op test from new_ir_op_test_white_list * fix * move generating following_input_vars logic to static_build.cc * remove HasInfo * fix build error * recover codes and turn off the flag

add conditional_block to OperatorBasesHandledInStaticBuild

9b469eb

paddle-bot bot added the contributor External developers label Aug 28, 2023

AndSonder added 27 commits August 28, 2023 11:31

run op in FakeInitializeOutputsForOperatorBase

db8a69c

add init_success judge

8770ed9

fix build error

483ea72

fix

224ccfa

add SetSubBlockCore func

7fc6735

add PreStaticRun func

7ced658

add PreStaticRun to interpreter_base and new_ir_inter

32ec0cf

recover codes

274cc0e

add PreStaticBuild and BlockCanBeStaticBuilt

889e7b8

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle

8a8225b

fix logic about RunPreStaticBuild

8902d01

change CreateOpFromOpDesc type

11a9a5d

fix build error

4b3a0f1

fix build error

5e40452

remove IsOperatorBasesHandledInStaticBuild

8171ef7

recover BlockCanBeStaticBuilt

bab22a8

add logic about conditional_block run static build

522a005

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

6fb3fc7

… control_flow_support-step-2

recover codes

9b872c8

recover BlockCanBeStaticBuilt

9c3a0f4

support static build condational block op when condational block is t…

3b5c905

…he last op in the block

fix error

32b5d4c

fix logic about last op

87538c7

fit for sub block can't open static build

b56f7fa

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

c4b3d82

… control_flow_support-step-2

add IsStaticBuild

1f68f61

fix build error

9456c05

remove useless codes

0e777f4

From00 reviewed Sep 11, 2023

View reviewed changes

AndSonder added 3 commits September 11, 2023 06:55

fix

c1a2123

fix

d1dddce

fix build error

71fc516

AndSonder requested a review from From00 September 12, 2023 05:58

AndSonder added 5 commits September 12, 2023 13:48

move GetVarsInfo and RunPreStaticBuild from opeartor to static_build

d13bc09

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

4795474

… control_flow_support-step-2

fix lamb backend registe

611c035

fix build error

b180105

fix build error

a640ee4

AndSonder mentioned this pull request Sep 13, 2023

静态图执行器预分析性能优化 #55299

Closed

AndSonder added 2 commits September 15, 2023 07:25

remove lamp op test from new_ir_op_test_white_list

90292fb

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

e7baf2c

… control_flow_support-step-2

From00 reviewed Sep 18, 2023

View reviewed changes

AndSonder added 2 commits September 18, 2023 04:38

fix

7cd51be

move generating following_input_vars logic to static_build.cc

222008c

AndSonder requested a review from From00 September 18, 2023 13:08

AndSonder added 2 commits September 19, 2023 04:41

remove HasInfo

914ec7b

fix build error

e3ed639

From00 previously approved these changes Sep 20, 2023

View reviewed changes

recover codes and turn off the flag

4d8dfea

AndSonder dismissed From00’s stale review via 4d8dfea September 20, 2023 06:29

From00 approved these changes Sep 21, 2023

View reviewed changes

From00 merged commit 164abf2 into PaddlePaddle:develop Sep 21, 2023

AndSonder mentioned this pull request Dec 7, 2023

WAVE SUMMIT+2023下半年飞桨开源之星评选-信息征集 PaddlePaddle/community#765

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support control flow for static build [Step 2: support conditional_block] #56696

Support control flow for static build [Step 2: support conditional_block] #56696

AndSonder commented Aug 28, 2023 •

edited

Loading

paddle-bot bot commented Aug 28, 2023

From00 Sep 11, 2023

AndSonder Sep 11, 2023

From00 Sep 11, 2023

AndSonder Sep 11, 2023

From00 Sep 11, 2023

AndSonder Sep 11, 2023

From00 Sep 11, 2023

AndSonder Sep 11, 2023

From00 Sep 11, 2023

AndSonder Sep 11, 2023

From00 Sep 11, 2023

AndSonder Sep 11, 2023

From00 Sep 11, 2023

AndSonder Sep 11, 2023

From00 Sep 11, 2023

AndSonder Sep 11, 2023

From00 Sep 11, 2023

AndSonder Sep 11, 2023

From00 Sep 11, 2023

AndSonder Sep 11, 2023

From00 Sep 18, 2023

AndSonder Sep 18, 2023

From00 Sep 18, 2023

AndSonder Sep 18, 2023

From00 Sep 18, 2023

AndSonder Sep 18, 2023

From00 left a comment

		@@ -14,6 +14,7 @@

		#pragma once

		#include "paddle/fluid/framework/new_executor/new_executor_defs.h"

Support control flow for static build [Step 2: support conditional_block] #56696

Support control flow for static build [Step 2: support conditional_block] #56696

Conversation

AndSonder commented Aug 28, 2023 • edited Loading

PR types

PR changes

Description

paddle-bot bot commented Aug 28, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

From00 left a comment

Choose a reason for hiding this comment

AndSonder commented Aug 28, 2023 •

edited

Loading