Unable to Build GTSAM Python Wrapper

[alliWong]

Description

I followed the instructions on https://github.com/borglab/gtsam/tree/develop/python but I am unable to successfully
cmake .. -DGTSAM_BUILD_PYTHON=1 -DGTSAM_PYTHON_VERSION=3.x.x for python 3.9.2, 3.6.10, and 3.8.3.
I've also tried building the older version that worked with my acquaintance (4.0.0 and cython)

Steps to reproduce

1. cmake .. -DGTSAM_BUILD_PYTHON=1 -DGTSAM_PYTHON_VERSION=3.9.3

-- The CXX compiler identification is GNU 7.5.0
-- The C compiler identification is GNU 7.5.0
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Performing Test COMPILER_HAS_WSUGGEST_OVERRIDE
-- Performing Test COMPILER_HAS_WSUGGEST_OVERRIDE - Success
-- Performing Test COMPILER_HAS_WMISSING_OVERRIDE
-- Performing Test COMPILER_HAS_WMISSING_OVERRIDE - Failed
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found Boost: /usr/include (found suitable version "1.65.1", minimum required is "1.58") found components:  serialization system filesystem thread program_options date_time timer chrono regex atomic 
-- Found Eigen version: 3.3.7
-- GTSAM_POSE3_EXPMAP=ON, enabling GTSAM_ROT3_EXPMAP as well
-- Could NOT find MKL (missing: MKL_INCLUDE_DIR MKL_LIBRARIES) 
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- Found TBB: /usr/include (found suitable version "2017.0", minimum required is "4.4") found components:  tbb tbbmalloc 
CMake Warning at wrap/cmake/GtwrapUtils.cmake:68 (find_package):
  By not providing "FindPython.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "Python", but
  CMake did not find one.

  Could not find a package configuration file provided by "Python" (requested
  version 3.9) with any of the following names:

    PythonConfig.cmake
    python-config.cmake

  Add the installation prefix of "Python" to CMAKE_PREFIX_PATH or set
  "Python_DIR" to a directory containing one of the above files.  If "Python"
  provides a separate development package or SDK, be sure it has been
  installed.
Call Stack (most recent call first):
  wrap/CMakeLists.txt:14 (gtwrap_get_python_version)


CMake Error at wrap/CMakeLists.txt:36 (install):
  install PROGRAMS given no DESTINATION!


-- pybind11 v2.6.0 dev1
-- Found PythonInterp: /home/allison/anaconda3/envs/gtsam/bin/python (found suitable version "3.9.2", minimum required is "3.9") 
-- Found PythonLibs: /home/allison/anaconda3/envs/gtsam/lib/libpython3.9.so
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- Building 3rdparty
-- Looking for execinfo.h
-- Looking for execinfo.h - found
-- Looking for getline
-- Looking for getline - found
-- checking for thread-local storage - found
-- Could NOT find GeographicLib (missing: GeographicLib_LIBRARY_DIRS GeographicLib_LIBRARIES GeographicLib_INCLUDE_DIRS) 
-- Building base
-- Building geometry
-- Building inference
-- Building symbolic
-- Building discrete
-- Building linear
-- Building nonlinear
-- Building sam
-- Building sfm
-- Building slam
-- Building navigation
-- GTSAM Version: 4.1.0
-- Install prefix: /usr/local
-- Building GTSAM - shared: ON
-- Building base_unstable
-- Building geometry_unstable
-- Building linear_unstable
-- Building discrete_unstable
-- Building dynamics_unstable
-- Building nonlinear_unstable
-- Building slam_unstable
-- Building partition_unstable
-- GTSAM_UNSTABLE Version: 4.1.0
-- Install prefix: /usr/local
-- Wrote /home/allison/Workspace/gtsam_ws/src/gtsam/build/GTSAMConfig.cmake
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) 
-- ===============================================================
-- ================  Configuration Options  ======================
--  CMAKE_CXX_COMPILER_ID type                       : GNU
--  CMAKE_CXX_COMPILER_VERSION                       : 7.5.0
--  CMake version                                    : 3.10.2
--  CMake generator                                  : Unix Makefiles
--  CMake build tool                                 : /usr/bin/make
-- Build flags                                               
--  Build Tests                                      : Enabled
--  Build examples with 'make all'                   : Enabled
--  Build timing scripts with 'make all'             : Disabled
--  Build shared GTSAM libraries                     : Enabled
--  Put build type in library name                   : Enabled
--  Build libgtsam_unstable                          : Enabled
--  Build GTSAM unstable Python                      : Enabled
--  Build MATLAB Toolbox for unstable                : Disabled
--  Build for native architecture                    : Enabled
--  Build type                                       : Release
--  C compilation flags                              :  -O3 -DNDEBUG
--  C++ compilation flags                            :  -O3 -DNDEBUG
--  GTSAM_COMPILE_FEATURES_PUBLIC                    : cxx_std_11
--  GTSAM_COMPILE_OPTIONS_PUBLIC                     : -march=native
--  GTSAM_COMPILE_DEFINITIONS_PUBLIC                 : 
--  GTSAM_COMPILE_OPTIONS_PUBLIC_DEBUG               : 
--  GTSAM_COMPILE_DEFINITIONS_PUBLIC_DEBUG           : 
--  GTSAM_COMPILE_OPTIONS_PUBLIC_RELEASE             : 
--  GTSAM_COMPILE_DEFINITIONS_PUBLIC_RELEASE         : 
--  GTSAM_COMPILE_OPTIONS_PUBLIC_TIMING              : 
--  GTSAM_COMPILE_DEFINITIONS_PUBLIC_TIMING          : 
--  GTSAM_COMPILE_OPTIONS_PUBLIC_PROFILING           : 
--  GTSAM_COMPILE_DEFINITIONS_PUBLIC_PROFILING       : 
--  GTSAM_COMPILE_OPTIONS_PUBLIC_RELWITHDEBINFO      : 
--  GTSAM_COMPILE_DEFINITIONS_PUBLIC_RELWITHDEBINFO  : 
--  GTSAM_COMPILE_OPTIONS_PUBLIC_MINSIZEREL          : 
--  GTSAM_COMPILE_DEFINITIONS_PUBLIC_MINSIZEREL      : 
--  Use System Eigen                                 : OFF (Using version: 3.3.7)
--  Use Intel TBB                                    : Yes (Version: 2017.0)
--  Eigen will use MKL                               : MKL not found
--  Eigen will use MKL and OpenMP                    : OpenMP found but GTSAM_WITH_EIGEN_MKL is disabled
--  Default allocator                                : TBB
--  Cheirality exceptions enabled                    : YES
--  Build with ccache                                : No
-- Packaging flags
--  CPack Source Generator                           : TGZ
--  CPack Generator                                  : TGZ
-- GTSAM flags                                               
--  Quaternions as default Rot3                      : Disabled
--  Runtime consistency checking                     : Disabled
--  Rot3 retract is full ExpMap                      : Enabled
--  Pose3 retract is full ExpMap                     : Enabled
--  Allow features deprecated in GTSAM 4.1           : Enabled
--  Metis-based Nested Dissection                    : Enabled
--  Use tangent-space preintegration                 : Enabled
-- MATLAB toolbox flags
--  Install MATLAB toolbox                           : Disabled
-- Python toolbox flags                                      
--  Build Python module with pybind                  : Enabled
--  Python version                                   : 3.9
-- ===============================================================
-- Configuring incomplete, errors occurred!
See also "/home/allison/Workspace/gtsam_ws/src/gtsam/build/CMakeFiles/CMakeOutput.log".
See also "/home/allison/Workspace/gtsam_ws/src/gtsam/build/CMakeFiles/CMakeError.log".

2. make python-install

make: *** No rule to make target 'python-install'.  Stop.

Expected behavior

Environment

I am using Anaconda, my cmake is up-to date, and I have met the requirements.txt. Weird thing is I am able to import gtsam in python3 terminal.....but I am trying to work on the python-wrapper-project and haven't been able to get it working without the parent gtsam successfully building. I have also tried to cmake with
1.
cmake ..
-DGTSAM_BUILD_PYTHON=1
-DPYTHON_INCLUDE_DIR=$(python -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())")
-DPYTHON_LIBRARY=$(python -c "import distutils.sysconfig as sysconfig; print(sysconfig.get_config_var('LIBDIR'))")
-DPYTHON_EXECUTABLE:FILEPATH=which python

cmake .. \
> -DPYTHON_INCLUDE_DIR=$(python -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())")  \
> -DPYTHON_LIBRARY=$(python -c "import distutils.sysconfig as sysconfig; print(sysconfig.get_config_var('LIBDIR'))") \
> -DPYTHON_EXECUTABLE:FILEPATH=`which python`
-- The CXX compiler identification is GNU 7.5.0
-- The C compiler identification is GNU 7.5.0
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Performing Test COMPILER_HAS_WSUGGEST_OVERRIDE
-- Performing Test COMPILER_HAS_WSUGGEST_OVERRIDE - Success
-- Performing Test COMPILER_HAS_WMISSING_OVERRIDE
-- Performing Test COMPILER_HAS_WMISSING_OVERRIDE - Failed
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found Boost: /usr/include (found suitable version "1.65.1", minimum required is "1.58") found components:  serialization system filesystem thread program_options date_time timer chrono regex atomic 
-- Found Eigen version: 3.3.7
-- GTSAM_POSE3_EXPMAP=ON, enabling GTSAM_ROT3_EXPMAP as well
-- Could NOT find MKL (missing: MKL_INCLUDE_DIR MKL_LIBRARIES) 
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- Found TBB: /usr/include (found suitable version "2017.0", minimum required is "4.4") found components:  tbb tbbmalloc 
-- Building 3rdparty
-- Looking for execinfo.h
-- Looking for execinfo.h - found
-- Looking for getline
-- Looking for getline - found
-- checking for thread-local storage - found
-- Could NOT find GeographicLib (missing: GeographicLib_LIBRARY_DIRS GeographicLib_LIBRARIES GeographicLib_INCLUDE_DIRS) 
-- Building base
-- Building geometry
-- Building inference
-- Building symbolic
-- Building discrete
-- Building linear
-- Building nonlinear
-- Building sam
-- Building sfm
-- Building slam
-- Building navigation
-- GTSAM Version: 4.1.0
-- Install prefix: /usr/local
-- Building GTSAM - shared: ON
-- Building base_unstable
-- Building geometry_unstable
-- Building linear_unstable
-- Building discrete_unstable
-- Building dynamics_unstable
-- Building nonlinear_unstable
-- Building slam_unstable
-- Building partition_unstable
-- GTSAM_UNSTABLE Version: 4.1.0
-- Install prefix: /usr/local
-- Wrote /home/allison/Workspace/gtsam_ws/src/gtsam/build/GTSAMConfig.cmake
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) 
-- ===============================================================
-- ================  Configuration Options  ======================
--  CMAKE_CXX_COMPILER_ID type                       : GNU
--  CMAKE_CXX_COMPILER_VERSION                       : 7.5.0
--  CMake version                                    : 3.10.2
--  CMake generator                                  : Unix Makefiles
--  CMake build tool                                 : /usr/bin/make
-- Build flags                                               
--  Build Tests                                      : Enabled
--  Build examples with 'make all'                   : Enabled
--  Build timing scripts with 'make all'             : Disabled
--  Build shared GTSAM libraries                     : Enabled
--  Put build type in library name                   : Enabled
--  Build libgtsam_unstable                          : Enabled
--  Build GTSAM unstable Python                      : Enabled
--  Build MATLAB Toolbox for unstable                : Disabled
--  Build for native architecture                    : Enabled
--  Build type                                       : Release
--  C compilation flags                              :  -O3 -DNDEBUG
--  C++ compilation flags                            :  -O3 -DNDEBUG
--  GTSAM_COMPILE_FEATURES_PUBLIC                    : cxx_std_11
--  GTSAM_COMPILE_OPTIONS_PUBLIC                     : -march=native
--  GTSAM_COMPILE_DEFINITIONS_PUBLIC                 : 
--  GTSAM_COMPILE_OPTIONS_PUBLIC_DEBUG               : 
--  GTSAM_COMPILE_DEFINITIONS_PUBLIC_DEBUG           : 
--  GTSAM_COMPILE_OPTIONS_PUBLIC_RELEASE             : 
--  GTSAM_COMPILE_DEFINITIONS_PUBLIC_RELEASE         : 
--  GTSAM_COMPILE_OPTIONS_PUBLIC_TIMING              : 
--  GTSAM_COMPILE_DEFINITIONS_PUBLIC_TIMING          : 
--  GTSAM_COMPILE_OPTIONS_PUBLIC_PROFILING           : 
--  GTSAM_COMPILE_DEFINITIONS_PUBLIC_PROFILING       : 
--  GTSAM_COMPILE_OPTIONS_PUBLIC_RELWITHDEBINFO      : 
--  GTSAM_COMPILE_DEFINITIONS_PUBLIC_RELWITHDEBINFO  : 
--  GTSAM_COMPILE_OPTIONS_PUBLIC_MINSIZEREL          : 
--  GTSAM_COMPILE_DEFINITIONS_PUBLIC_MINSIZEREL      : 
--  Use System Eigen                                 : OFF (Using version: 3.3.7)
--  Use Intel TBB                                    : Yes (Version: 2017.0)
--  Eigen will use MKL                               : MKL not found
--  Eigen will use MKL and OpenMP                    : OpenMP found but GTSAM_WITH_EIGEN_MKL is disabled
--  Default allocator                                : TBB
--  Cheirality exceptions enabled                    : YES
--  Build with ccache                                : No
-- Packaging flags
--  CPack Source Generator                           : TGZ
--  CPack Generator                                  : TGZ
-- GTSAM flags                                               
--  Quaternions as default Rot3                      : Disabled
--  Runtime consistency checking                     : Disabled
--  Rot3 retract is full ExpMap                      : Enabled
--  Pose3 retract is full ExpMap                     : Enabled
--  Allow features deprecated in GTSAM 4.1           : Enabled
--  Metis-based Nested Dissection                    : Enabled
--  Use tangent-space preintegration                 : Enabled
-- MATLAB toolbox flags
--  Install MATLAB toolbox                           : Disabled
-- Python toolbox flags                                      
--  Build Python module with pybind                  : Disabled
-- ===============================================================
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    PYTHON_EXECUTABLE
    PYTHON_INCLUDE_DIR
    PYTHON_LIBRARY


-- Build files have been written to: /home/allison/Workspace/gtsam_ws/src/gtsam/build

Would appreciate it if I can get any help with this~!

[varunagrawal]

The CMake error message is helpful here:

CMake Warning at wrap/cmake/GtwrapUtils.cmake:68 (find_package):
  By not providing "FindPython.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "Python", but
  CMake did not find one.

  Could not find a package configuration file provided by "Python" (requested
  version 3.9) with any of the following names:

    PythonConfig.cmake
    python-config.cmake

  Add the installation prefix of "Python" to CMAKE_PREFIX_PATH or set
  "Python_DIR" to a directory containing one of the above files.  If "Python"
  provides a separate development package or SDK, be sure it has been
  installed.

Looks like your Anaconda installation is not providing the PythonConfig.cmake file, or the file is not in the environment path variable which CMake uses. This is a very custom issue since it depends on how your Python interpreter is set up, but my recommendation would be to see how (via Google or StackOverflow) to add that path containing the appropriate CMake files so that CMake can find it.

[varunagrawal]

@alliWong you're in luck. Apparently I encountered a similar issue in the past and I upvoted an answer on SO. Sharing it here for your convenience. https://stackoverflow.com/a/38121972/1236990

If you'd like, you can create a PR that adds this info to the README file and we'll gladly review and accept it (plus you get to be a GTSAM contributor!!). 🙂

[alliWong]

@varunagrawal Thanks for the answer.

I did try to follow the suggestion given the link, but I am still unable to build pybind11. I've also manually set the path to PYTHON_EXECUTABLE, PYTHON_INCLUDE_DIR, and PYTHON_LIBRARY, so I wasn't too sure if it's due to my Anaconda system or the CMakeList.txt problem...

[varunagrawal]

You mean the python interpreter, and not Pybind11?

You can try looking for those specific CMake files (e.g. use locate on Ubuntu/Linux). If you can't find them, you may want to check what build files you need anaconda to install.

[varunagrawal]

Also, what version of CMake are you using?

[alliWong]

Actually, I just reinstalled gtsam and was able to get it compiled working perfectly!

[alliWong]

Thanks for the help and I'll also create a PR :)

[varunagrawal]

Huh that's interesting. I wonder what the issue was but I'm glad it works out of the box for you now.

[catproof]

I'm coming across the same issue. I've tried everything mentioned in this thread... no luck.

I confirmed I have cmake version 3.10.2, as well as Python 3.6.9

[catproof]

@varunagrawal @alliWong

alli, what did you mean by 'reinstall GTSAM'? do you just mean you deleted the build folder and started form scratch?

1. delete build folder
2. recreate build folder
3. cd into build folder
4. run: cmake .. -DGTSAM_BUILD_PYTHON=1 -DGTSAM_PYTHON_VERSION=3.x.x

[varunagrawal]

That should be sufficient. Please post the full error message.