PYBIND11_FINDPYTHON=ON

[ax3l]

Reuse our find_package(Python ...) call and use new CMake logic in pybind11.
https://pybind11.readthedocs.io/en/stable/compiling.html#modules-with-cmake
https://cmake.org/cmake/help/latest/command/find_package.html#config-mode-version-selection

Fix #1677 (comment)

[franzpoeschel]

Should we add this logic to our superbuilds in CMake? Set PYTHON_EXECUTABLE before adding pybind11 if it is not yet set?

[ax3l]

Possibly, yes!

We should set the executable that was found here:
https://github.com/openPMD/openPMD-api/blob/0.16.0/cmake/dependencies/pybind11.cmake#L80-L86

So a simple

set(PYTHON_EXECUTABLE ${Python_EXECUTABLE})

might make it already more robust.

[ax3l]

Ok, so if we set PYBIND11_FINDPYTHON=ON
https://pybind11.readthedocs.io/en/stable/compiling.html#modules-with-cmake

Then pybind11 will call find_package(Python ...) again, which will see that we already called it (default find_package behavior), use our cached results and rely on the new logic
https://github.com/pybind/pybind11/blob/v2.13.6/tools/pybind11Common.cmake#L215-L223

Well, not obvious from the docs, but ok.

[ax3l]

MSVC: calling setup.py with a python 3.9 now finds newest on the system first (outside of setup-python, venv and conda envs):

# python3.exe is c:\hostedtoolcache\windows\python\3.9.13\x64\python3.exe
$ python3.exe -m pip wheel -v .
...
  -- Found Python: C:/hostedtoolcache/windows/Python/3.12.7/x64/python3.exe (found suitable version "3.12.7", minimum required is "3.7.0") found components: Interpreter Development.Module
...

Which hints do we want to set in setup.py to make sure the calling Python process is exactly the Python that is used? (Note: Python_EXECUTABLE is an overwritable output, not an input. Setting it can still mismatch libs, etc.)

MSVC: NumPy requires newer Python:

  Link requires a different Python (3.9.13 not in: '>=3.10'): https://files.pythonhosted.org/packages/d1/0f/8d2b5ebb01dc49d20ae0a282d6baff7202b7bf0df8acdd4a6abeffe98070/numpy-2.1.0rc1.tar.gz (from https://pypi.org/simple/numpy/) (requires-python:>=3.10)

[ax3l]

Found a way to hint location and, if multiple versions are installed, version:

• https://cmake.org/cmake/help/latest/command/find_package.html#config-mode-version-selection
• https://cmake.org/cmake/help/latest/module/FindPython.html#hints

[ax3l]

Ok, that should do the trick :)

[ax3l]

Equivalent PRs to:

• pyAMReX: PYBIND11_FINDPYTHON=ON AMReX-Codes/pyamrex#381
• WarpX: PYBIND11_FINDPYTHON=ON ECP-WarpX/WarpX#5390
• ImpactX: PYBIND11_FINDPYTHON=ON ECP-WarpX/impactx#738

all pass :)