add MKL for fluid static and shared library #8887

luotao1 · 2018-03-08T12:46:29Z

add MKL for fluid static and shared library， the structure is:

third_party/install/mklml/
├── include
└── lib

remove PADDLE_USE_ATLAS which is forgot by remove ATLAS library #6530
Examples: https://github.com/luotao1/fluid_inference_example.

Xreki

https://github.com/luotao1/fluid_inference_example/blob/f97cd04e6e51753ea7fbe91aab286585c51466b4/CMakeLists.txt#L14-L16

这个示例里面，我认为链接libpaddle_fluid.so的时候也是需要链接libmklml_intel.so的，我后面会测试一下。

======Update======
我本地测试过了，fluid_inference_example项目make时会出如下错误：

/home/liuyiqun01/.jumbo/opt/gcc48/bin/g++      CMakeFiles/example.dir/example.cc.o  -o example -rdynamic /home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/dist/paddle/fluid/inference/libpaddle_fluid.so /home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/dist/third_party/install/gflags/lib/libgflags.a -lrt -ldl -lpthread -Wl,-rpath,/home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/dist/paddle/fluid/inference 
/home/liuyiqun01/.jumbo/opt/binutils/bin/ld: warning: libiomp5.so, needed by /home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/third_party/install/mklml/lib/libmklml_intel.so, not found (try using -rpath or -rpath-link)
/home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/third_party/install/mklml/lib/libmklml_intel.so: undefined reference to `__kmpc_atomic_float4_max@VERSION'
/home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/third_party/install/mklml/lib/libmklml_intel.so: undefined reference to `__kmpc_master@VERSION'
/home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/third_party/install/mklml/lib/libmklml_intel.so: undefined reference to `__kmpc_single@VERSION'
/home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/third_party/install/mklml/lib/libmklml_intel.so: undefined reference to `__kmpc_end_serialized_parallel@VERSION'
/home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/third_party/install/mklml/lib/libmklml_intel.so: undefined reference to `omp_in_parallel@VERSION'
...

需要链接mkl_intel和iomp5两个动态库。

Xreki · 2018-03-21T12:11:58Z

cmake/inference_lib.cmake

+    copy(mklml_lib
+      SRCS ${MKLML_LIB_DIR} ${MKLML_INC_DIR}
+      DSTS ${dst_dir} ${dst_dir}
+    )


这里应该加if(WITH_MKLML)判断？

~~如果链接了mklml库，貌似还需要链接iomp库？~~看错了，已经全部拷贝过去了。另外问一句，libmklml_gnu.so是不是用不上？

Paddle/cmake/util.cmake

Lines 118 to 120 in 4f9c996

if(WITH_MKLML AND MKLML_LIB_DIR AND MKLML_IOMP_LIB)

target_link_libraries(${TARGET_NAME} "-L${MKLML_LIB_DIR} -liomp5 -Wl,--as-needed")

endif()

这里应该加if(WITH_MKLML)判断

不需要加，if(CBLAS_FOUND)时能正确拷贝MKLML相关的库：
https://paddleci.ngrok.io/viewLog.html?buildId=30302&buildTypeId=Paddle_PrCi&tab=buildLog&_focus=13970#_state=71

libmklml_gnu.so是不是用不上

对的，这个库用不上。所以需要把这个库给去掉么？

需要加if(WITH_MKLML)判断的。如果用户cmake设置WITH_MKL=OFF并且设置了OPEBLAS_ROOT，这里不加判断应该会出错。

所以需要把这个库给去掉么？

我觉得可以去掉，至少make inference_lib_dist的时候不要拷贝过去吧。

Xreki · 2018-03-21T12:17:14Z

cmake/external/mklml.cmake

@@ -46,7 +46,7 @@ INCLUDE_DIRECTORIES(${MKLML_INC_DIR})
 FILE(WRITE ${MKLML_DOWNLOAD_DIR}/CMakeLists.txt
  "PROJECT(MKLML)\n"
  "cmake_minimum_required(VERSION 3.0)\n"
-  "install(DIRECTORY ${MKLML_VER}\n"
+  "install(DIRECTORY ${MKLML_VER}/include ${MKLML_VER}/lib \n"
  "        DESTINATION ${MKLML_DST_DIR})\n")


mklml的依赖是不是不太对？我把third_party和third_party/install下面的mklml删掉了之后，make -j12出现了如下编译错误：

[ 13%] Building NVCC (Device) object paddle/fluid/operators/math/CMakeFiles/selected_rows_functor.dir/selected_rows_functor_generated_selected_rows_functor.cu.o In file included from /home/liuyiqun01/PaddlePaddle/Paddle/paddle/fluid/operators/math/selected_rows_functor.cu:17:0: /home/liuyiqun01/PaddlePaddle/Paddle/paddle/fluid/operators/math/math_function.h:17:23: fatal error: mkl_cblas.h: No such file or directory #include <mkl_cblas.h> ^ compilation terminated. CMake Error at selected_rows_functor_generated_selected_rows_functor.cu.o.cmake:202 (message): Error generating /home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/build/paddle/fluid/operators/math/CMakeFiles/selected_rows_functor.dir//./selected_rows_functor_generated_selected_rows_functor.cu.o make[2]: *** [paddle/fluid/operators/math/CMakeFiles/selected_rows_functor.dir/selected_rows_functor_generated_selected_rows_functor.cu.o] Error 1 make[1]: *** [paddle/fluid/operators/math/CMakeFiles/selected_rows_functor.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs....

======Update======
develop分支下载hang住了，但似乎没出现这个错误。

mklml的依赖是不是不太对

依赖是对的，teamcity上的编译能正常通过。

develop分支下载hang住了

是在厂内测试机上下载mklml的时候hang住了么？厂内测试机的下载会受到网络或SSL因素影响，导致下载失败，可以参考这里进行手动安装。

手动安装就没法正确测试这个依赖了。。。依赖从直观上来看觉得没什么问题，但是想不通为啥会遇到这个错误。。。

手动安装的时候，只是在third_party/mklml中手动下载和解压了mklml.tgz这个包，后面install到third_party/install/mklml的时候还需要使用make命令，所以是能正确测试依赖的。

这个其实不一定，因为没有了下载这个过程，解压的过程很快，很可能因为这个任务很快就完成了，而其他地方即使没有正确地依赖，也不会出错。不过因为cmake我看不出来问题，develop分支也没有遇到这个错误，可能是解决了吧。

luotao1 · 2018-03-26T04:49:41Z

libmklml_gnu.so已删除安装，if(WITH_MKLML)判断已加，fluid_inference_example例子已更新。

Xreki

LGTM

Xreki · 2018-03-26T11:59:11Z

cmake/external/mklml.cmake

@@ -46,7 +46,7 @@ INCLUDE_DIRECTORIES(${MKLML_INC_DIR})
 FILE(WRITE ${MKLML_DOWNLOAD_DIR}/CMakeLists.txt
  "PROJECT(MKLML)\n"
  "cmake_minimum_required(VERSION 3.0)\n"
-  "install(DIRECTORY ${MKLML_VER}\n"
+  "install(DIRECTORY ${MKLML_VER}/include ${MKLML_VER}/lib \n"
  "        DESTINATION ${MKLML_DST_DIR})\n")


这个其实不一定，因为没有了下载这个过程，解压的过程很快，很可能因为这个任务很快就完成了，而其他地方即使没有正确地依赖，也不会出错。不过因为cmake我看不出来问题，develop分支也没有遇到这个错误，可能是解决了吧。

luotao1 added 4 commits March 8, 2018 20:41

add MKL for fluid static and shared library

5030681

remove PADDLE_USE_ATLAS

bc0cfb2

Merge branch 'develop' into infer_mkl

de13f0e

fix mklml install path

e42b8f8

luotao1 requested a review from Xreki March 13, 2018 08:41

luotao1 added the 预测原名Inference，包含Capi预测问题等 label Mar 15, 2018

Xreki reviewed Mar 21, 2018

View reviewed changes

luotao1 added 3 commits March 23, 2018 11:54

Merge branch 'develop' into infer_mkl

ae820a3

Merge branch 'develop' into infer_mkl

6332bd1

rm libmklml_gnu.so

d573195

Xreki approved these changes Mar 26, 2018

View reviewed changes

luotao1 merged commit c858f48 into PaddlePaddle:develop Mar 26, 2018

luotao1 deleted the infer_mkl branch March 26, 2018 12:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add MKL for fluid static and shared library #8887

add MKL for fluid static and shared library #8887

luotao1 commented Mar 8, 2018

Xreki left a comment •

edited

Loading

Xreki Mar 21, 2018 •

edited

Loading

luotao1 Mar 23, 2018

Xreki Mar 23, 2018 •

edited

Loading

Xreki Mar 21, 2018 •

edited

Loading

luotao1 Mar 23, 2018

Xreki Mar 23, 2018

luotao1 Mar 26, 2018

Xreki Mar 26, 2018

luotao1 commented Mar 26, 2018

Xreki left a comment

Xreki Mar 26, 2018

	if(WITH_MKLML AND MKLML_LIB_DIR AND MKLML_IOMP_LIB)
	target_link_libraries(${TARGET_NAME} "-L${MKLML_LIB_DIR} -liomp5 -Wl,--as-needed")
	endif()

add MKL for fluid static and shared library #8887

add MKL for fluid static and shared library #8887

Conversation

luotao1 commented Mar 8, 2018

Xreki left a comment • edited Loading

Choose a reason for hiding this comment

Xreki Mar 21, 2018 • edited Loading

Choose a reason for hiding this comment

luotao1 Mar 23, 2018

Choose a reason for hiding this comment

Xreki Mar 23, 2018 • edited Loading

Choose a reason for hiding this comment

Xreki Mar 21, 2018 • edited Loading

Choose a reason for hiding this comment

luotao1 Mar 23, 2018

Choose a reason for hiding this comment

Xreki Mar 23, 2018

Choose a reason for hiding this comment

luotao1 Mar 26, 2018

Choose a reason for hiding this comment

Xreki Mar 26, 2018

Choose a reason for hiding this comment

luotao1 commented Mar 26, 2018

Xreki left a comment

Choose a reason for hiding this comment

Xreki Mar 26, 2018

Choose a reason for hiding this comment

Xreki left a comment •

edited

Loading

Xreki Mar 21, 2018 •

edited

Loading

Xreki Mar 23, 2018 •

edited

Loading

Xreki Mar 21, 2018 •

edited

Loading