Rebuild for python 3.13

[regro-cf-autotick-bot]

This PR has been triggered in an effort to update python313.

Notes and instructions for merging this PR:

1. Please merge the PR only after the tests have passed.
2. Feel free to push to the bot's branch to update this PR if needed.

Please note that if you close this PR we presume that the feedstock has been rebuilt, so if you are going to perform the rebuild yourself don't close this PR until the your rebuild has been merged.

---

This package has the following downstream children:

• fenics-basix
• gnuradio
• heavydb-ext
• pyopencl
  and potentially more.

---

If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase @conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

_{This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. Feel free to drop us a line if there are any issues! This PR was generated by https://github.com/regro/cf-scripts/actions/runs/10607148608 - please use this URL for debugging.}

[conda-forge-webservices]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

[github-actions]

Hi! This is the friendly conda-forge automerge bot!

I considered the following status checks when analyzing this PR:

• linter: passed
• azure: passed

Thus the PR was passing and merged! Have a great day!

[h-vetinari]

@conda-forge/core, we're a bit stuck here, mainly because ustream numpy was only testing with Cython 3.1.0 nightlies, but there's no tag we can use for that (short of building from head).

OTOH, the failures here are limited to osx (for reasons unknown), and it's only about ~20 test failures out of almost 50k tests.

Do we want to:

• run the 3.13 migration with a cython 3.1.0.dev0 built from some commit?
• put cython 3.1.0.dev0 into a separate label and use it where necessary? (mostly, 3.0.11 should be good enough...)
• skip the failing tests here?
• wait?

[beckermr]

IMHO we wait. This is an issue in numpy and numpy needs to either fix it or make everyone wait for cython 3.1. That's for them to decide.

[h-vetinari]

make everyone wait for cython 3.1

Given how things went with cython 3.0, that's potentially many months if not longer.

IMHO we wait.

Normally I'd agree, though in this case I think a migration without numpy will not get us very far. However, where impact justified it (and the expected fallout was ~0), we have skipped a handful of tests here and there for numpy/scipy/etc. many times already. Skipping 20 tests out of 50000 does not break the package in any meaningful way IMO. And doubly so for RC builds.

As maintainer here, I wouldn't hesitate in skipping those tests under current circumstances, my question was more around the fact that it's quite possible that we'll run into the "actually requires cython 3.1.0" with several packages (i.e. everyone who's been working on nogil support must use those nightlies), and how we want to deal with that.

That's why I asked for a 3.1.0 alpha, and that's why I kind of favour the second point above, i.e. put a cython 3.1.0 build under conda-forge/label/cython_dev and be able to pull that in where we need during the migration. But it still has some question marks to me w.r.t. possible ABI impact.

[hmaarrfk]

These dtype failures are scary and really hard to debug.

We should submit patches to numpy or cython or wait

[h-vetinari]

These dtype failures are scary and really hard to debug.

Which dtype failures are you referring to? All but one failure are due to some test module (presumably compiled by cython) running into: SystemError: initialization of <module> did not return an extension module

We should submit patches to numpy or cython or wait

Thanks for the input!

[hmaarrfk]

FAILED _core/tests/test_cython.py::test_is_timedelta64_object - SystemError: initialization of checks did not return an extension module
FAILED _core/tests/test_cython.py::test_is_datetime64_object - SystemError: initialization of checks did not return an extension module
FAILED _core/tests/test_cython.py::test_get_datetime64_value - SystemError: initialization of checks did not return an extension module
FAILED _core/tests/test_cython.py::test_get_timedelta64_value - SystemError: initialization of checks did not return an extension module
FAILED _core/tests/test_cython.py::test_get_datetime64_unit - SystemError: initialization of checks did not return an extension module

from the test names, they just seem to have to do with "date time dtypes". I didn't look into it much further but:

1. We use datetime dtimes
2. We know they are somewhat fragile
3. They have been terribly hard to troubleshoot in the past for us.

[h-vetinari]

from the test names

key part here is (IMO)

FAILED _core/tests/test_cython.py::test_get_datetime64_unit - SystemError: initialization of checks did not return an extension module
                   ^^^^^^^^^^^^^^

IOW, this is not about numpy datetypes not working per se, but about testing interaction with cython. There are many other tests for datetimes (example) that are passing.

Looking closer, the tests in that module are all failing because this fails to get transpiled/compiled correctly. Which is admittedly more impactful than I first thought, because IIUC it means you cannot use (parts of) the numpy API through cython.

[rgommers]

I added a failure from the log to the issue description of numpy/numpy#27314 (logs disappear ....). What stands out there is that pkg-config is missing in the test stage:

Did not find pkg-config by name 'pkg-config'
Found pkg-config: NO

That is likely causing a difference between how cython is used during the build and test phases. It's odd, because pkg-config is both a build and a host dependency. Any idea how it can go missing?

[rgommers]

Ah of course, missed the separate test: requires: section. I bet that will resolve it.

That leaves explaining why it's only happening for Python 3.13 on macOS x86-64 and not for Python 3.12. Is libpython statically linked for 3.13 but dynamically for 3.12 or something like that? @jakirkham's comment here hinted at that being a possible root cause.

[isuruf]

No, all our libraries are linked against a static library. For some reason, the test adds -lpython3.13 which it's not supposed to do.

[rgommers]

For some reason, the test adds -lpython3.13 which it's not supposed to do.

That may be a bug in the SystemDependency for Python from Meson then. It passed now, so PkgConfigDependency does the right thing. I'll look into it.

[isuruf]

LIBPYTHON is not a sysconfig variable anymore. https://github.com/mesonbuild/meson/blob/18427adbf21909f66a307a54ba4c47fd91e18fba/mesonbuild/scripts/python_info.py#L74C36-L74C45

[rgommers]

LIBPYTHON is not a sysconfig variable anymore.

It should be, since we got that change reverted: python/cpython#122842. That commit made it into 1.13.0rc1 it looks like.

[h-vetinari]

Thanks for the debugging & fix! I've rebased out the debug commits. From my POV this is OK now, but happy to pick up other fixes around libpython if we want to wait for that.

[rgommers]

This should be fine to merge if CI is green; the libpython stuff isn't inside numpy.

[isuruf]

It should be, since we got that change reverted: python/cpython#122842. That commit made it into 1.13.0rc1 it looks like.

]3.13.0rc1 was released on Jul 31, but the PR is from Aug 8.

[rgommers]

3.13.0rc1 was released on Jul 31, but the PR is from Aug 8.

Yes, you're right. I was looking at the wrong commit. Okay, that explains it then. The problem has been fixed and will go away for the final 3.13.0 release.