Support for building VTK wheels

[jcfr]

In this post I will be describing the current process of building VTK wheels and its limitation.

VTK is the Visualization ToolKit, to learn more see https://docs.vtk.org/en/latest/about.html

I am looking forward to get input on how to streamline the process and potentially leverage scikit-build-core, inform its design and/or learn about existing tools/approach to help simplify.

cc: @henryiii

Current process

The content below provides details related to https://docs.vtk.org/en/latest/advanced/build_python_wheels.html

As of Kitware/VTK@ba70d091, the process for building VTK wheels involves the following steps¹:

cmake \
  -GNinja \
  -DVTK_WHEEL_BUILD=ON \
  -DVTK_WRAP_PYTHON=ON \
  path/to/vtk/source
ninja
python3 setup.py bdist_wheel

During the configure process if VTK_WHEEL_BUILD is ON, it includes:

• vtkWheelPreparation.cmake²
  • invoke the custom script wheel_extract_platlib.py³ to compute the name of the plat lib directory "usually" expected by setuptools/distutils
  • generate files setup.py and MANIFEST.in in the VTK build directory
• vtkWheelFinalization.cmake⁴
  • generates the file vtk_features.py

Once the build is completed, the CI workflow calls python setup.py bdist_wheel and platform specific tools:

• auditwheel on Linux. See .gitlab/os-linux.yml⁵
• delocate used to be called on macOS but since it was not able to deal with the structure/layout of VTK, handling of @rpath is now done explicitly, see .gitlab/os-macos.yml⁶
• delvewheel. See .gitlab/os-windows.yml⁷

Limitations

• "Manually" craft the structure of the wheel by generating files like setup.py and MANIFEST.in
• Rely on setup.py to build the wheels
• Rely on distutils in wheel_extract_platlib.py³
  • This is breaking now that we are adding support for Python 3.12. See https://open.cdash.org/build/8913426/configure and MR#10406

Footnotes

1. https://docs.vtk.org/en/latest/advanced/build_python_wheels.html ↩

2. https://github.com/Kitware/VTK/blob/ba70d0912615cc10a69789541f949ec170572902/CMake/vtkWheelPreparation.cmake ↩

3. https://github.com/Kitware/VTK/blob/ba70d0912615cc10a69789541f949ec170572902/CMake/wheel_extract_platlib.py ↩ ↩²

4. https://github.com/Kitware/VTK/blob/ba70d0912615cc10a69789541f949ec170572902/CMake/vtkWheelFinalization.cmake ↩

5. https://github.com/Kitware/VTK/blob/8f07f83261608c15dc700b29dcb45369e5b87c86/.gitlab/os-linux.yml#L461-L466 ↩

6. https://github.com/Kitware/VTK/blob/8f07f83261608c15dc700b29dcb45369e5b87c86/.gitlab/os-macos.yml#L245-L253 ↩

7. https://github.com/Kitware/VTK/blob/8f07f83261608c15dc700b29dcb45369e5b87c86/.gitlab/os-windows.yml#L312-L317 ↩

[mathstuf]

distutils replacement requested from Python forum: https://discuss.python.org/t/python-3-12-help-porting-from-distutils-to-get-the-expected-build-directory/31720