[Paddle Inference] Predictor support paddle::Tensor #50445

yuanlehome · 2023-02-13T07:31:10Z

PR types

New features

PR changes

APIs

Describe

Predictor新增Run接口，支持paddle::Tensor输入输出；
jit layer predictor engine适配新的Run接口，同时移除旧Tensor的转换逻辑；
tensor.h依赖的complex.h中部分特性实现需要-std=c++14支持，因此升级inference lib编译对c++标准的最低需求为c++14;
refine部分代码，补充单测代码

docs ref to 链接.

TODO：

补充c/go接口实现
更新c/go api文档

NOTES:

bug fix [Inference] Fix predictor.run([Tensor..]): holder_ should not be null #58048

paddle-bot · 2023-02-13T07:31:15Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

…hi_tensor_for_inference

zhwesky2010

LGTM for ut rename

…hi_tensor_for_inference

jiweibo · 2023-04-10T05:05:17Z

paddle/fluid/inference/api/analysis_predictor.cc

+    LOG(ERROR) << "fail to get fetches";
+    return false;
+  }
+  VLOG(3) << "predict cost: " << timer.toc() << "ms";


异步情况下不加sync耗时统计有问题吧

异步情况下不加sync耗时统计有问题吧

对，需要加同步，准备放在getfetch内部

异步情况下不加sync耗时统计有问题吧

讨论后不加同步了。paddle::Tensor有copy_to（c++）或to_tensor接口做这样的工作～

paddle/fluid/inference/api/analysis_predictor.cc

…hi_tensor_for_inference

qingqing01 · 2023-04-10T07:54:44Z

paddle/fluid/inference/api/analysis_predictor.cc

@@ -1037,6 +1048,73 @@ bool AnalysisPredictor::Run(const std::vector<PaddleTensor> &inputs,
  return true;
 }

+bool AnalysisPredictor::Run(const std::vector<paddle::Tensor> &inputs,
+                            std::vector<paddle::Tensor> *outputs) {


和上面Run接口，重复的实现内容能否被复用？

和上面Run接口，重复的实现内容能否被复用？

想过抽出公共代码出来，但感觉抽出来的代码没有清晰的作用

qingqing01 · 2023-04-10T07:58:07Z

paddle/fluid/inference/api/demo_ci/CMakeLists.txt

@@ -83,7 +83,7 @@ else()
  if(WITH_MKL)
    set(FLAG_OPENMP "-fopenmp")
  endif()
-  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 ${FLAG_OPENMP}")
+  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++14 ${FLAG_OPENMP}")


这里升级对其他是否有影响？

这里升级对其他是否有影响？

会影响到paddle-inference-demo中的cmakelists文件设置，已经提pr改过了。

paddle/fluid/pybind/inference_api.cc

jiweibo

LGTM

leiqing1

主要review了对应了API文档。目前却Go和C的文档

yuanlehome marked this pull request as draft February 13, 2023 07:31

yuanlehome marked this pull request as ready for review February 17, 2023 11:19

yuanlehome force-pushed the phi_tensor_for_inference branch 2 times, most recently from b511447 to b3796f3 Compare March 1, 2023 12:30

yuanlehome marked this pull request as draft March 2, 2023 01:38

yuanlehome changed the title ~~[Paddle Inference] support phi tensor~~ [Paddle Inference] support paddle::Tensor Mar 2, 2023

yuanlehome force-pushed the phi_tensor_for_inference branch 2 times, most recently from fd31565 to c28e71b Compare March 9, 2023 06:09

yuanlehome marked this pull request as ready for review March 9, 2023 06:10

yuanlehome changed the title ~~[Paddle Inference] support paddle::Tensor~~ [Paddle Inference] Predictor support paddle::Tensor Mar 9, 2023

yuanlehome force-pushed the phi_tensor_for_inference branch 8 times, most recently from 8e0b229 to a293262 Compare March 15, 2023 14:16

yuanlehome force-pushed the phi_tensor_for_inference branch 4 times, most recently from abcfadd to 335898b Compare March 21, 2023 05:41

qingqing01 requested review from xiaoxiaohehe001, JZZ-NOTE, jiweibo and qingqing01 March 23, 2023 14:11

yuanlehome added 2 commits March 30, 2023 05:35

inference predictor support paddle::Tensor

533f9d0

add unitest

d6997ff

yuanlehome force-pushed the phi_tensor_for_inference branch from 335898b to d6997ff Compare March 30, 2023 07:43

yuanlehome force-pushed the phi_tensor_for_inference branch from 16a7db5 to 0736089 Compare March 30, 2023 11:22

yuanlehome added 2 commits March 30, 2023 12:27

fix jit layer

4f8b943

Merge branch 'develop' of https://github.com/yuanlehome/Paddle into p…

6d06b4e

…hi_tensor_for_inference

yuanlehome force-pushed the phi_tensor_for_inference branch from 0736089 to 6d06b4e Compare March 30, 2023 12:28

zhwesky2010 previously approved these changes Mar 31, 2023

View reviewed changes

yuanlehome dismissed zhwesky2010’s stale review via 2151a34 March 31, 2023 07:05

Merge branch 'develop' of https://github.com/yuanlehome/Paddle into p…

2296742

…hi_tensor_for_inference

yuanlehome force-pushed the phi_tensor_for_inference branch from 2151a34 to 2296742 Compare March 31, 2023 07:35

fix

d2460c6

jiweibo reviewed Apr 10, 2023

View reviewed changes

paddle/fluid/inference/api/analysis_predictor.cc Show resolved Hide resolved

yuanlehome closed this Apr 10, 2023

Merge branch 'develop' of https://github.com/yuanlehome/Paddle into p…

1a554b7

…hi_tensor_for_inference

yuanlehome reopened this Apr 10, 2023

qingqing01 reviewed Apr 10, 2023

View reviewed changes

yuanlehome added 2 commits April 10, 2023 11:29

fix win bug

8175bfb

add (Recommended)

2d546c6

wanghuancoder approved these changes Apr 11, 2023

View reviewed changes

Shixiaowei02 approved these changes Apr 11, 2023

View reviewed changes

jiweibo reviewed Apr 11, 2023

View reviewed changes

paddle/fluid/pybind/inference_api.cc Show resolved Hide resolved

jiweibo approved these changes Apr 11, 2023

View reviewed changes

leiqing1 approved these changes Apr 11, 2023

View reviewed changes

yuanlehome closed this Apr 11, 2023

yuanlehome reopened this Apr 11, 2023

jiweibo merged commit 10fd4a9 into PaddlePaddle:develop Apr 11, 2023

EmmonsCurse mentioned this pull request Apr 12, 2023

update -std=c++11 to -std=c++14 PaddlePaddle/continuous_integration#276

Merged

yuanlehome mentioned this pull request Oct 12, 2023

[Inference] Fix predictor.run([Tensor..]): holder_ should not be null #58048

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Paddle Inference] Predictor support paddle::Tensor #50445

[Paddle Inference] Predictor support paddle::Tensor #50445

yuanlehome commented Feb 13, 2023 •

edited

Loading

paddle-bot bot commented Feb 13, 2023

zhwesky2010 left a comment

jiweibo Apr 10, 2023

yuanlehome Apr 10, 2023

yuanlehome Apr 10, 2023 •

edited

Loading

qingqing01 Apr 10, 2023

yuanlehome Apr 10, 2023

qingqing01 Apr 10, 2023

yuanlehome Apr 10, 2023

jiweibo left a comment

leiqing1 left a comment

[Paddle Inference] Predictor support paddle::Tensor #50445

[Paddle Inference] Predictor support paddle::Tensor #50445

Conversation

yuanlehome commented Feb 13, 2023 • edited Loading

PR types

PR changes

Describe

paddle-bot bot commented Feb 13, 2023

zhwesky2010 left a comment

Choose a reason for hiding this comment

jiweibo Apr 10, 2023

Choose a reason for hiding this comment

yuanlehome Apr 10, 2023

Choose a reason for hiding this comment

yuanlehome Apr 10, 2023 • edited Loading

Choose a reason for hiding this comment

qingqing01 Apr 10, 2023

Choose a reason for hiding this comment

yuanlehome Apr 10, 2023

Choose a reason for hiding this comment

qingqing01 Apr 10, 2023

Choose a reason for hiding this comment

yuanlehome Apr 10, 2023

Choose a reason for hiding this comment

jiweibo left a comment

Choose a reason for hiding this comment

leiqing1 left a comment

Choose a reason for hiding this comment

yuanlehome commented Feb 13, 2023 •

edited

Loading

yuanlehome Apr 10, 2023 •

edited

Loading