Add a numpy dependency with pkg-config and configtool methods

[rgommers]

These are being added for NumPy 2.0 (see numpy/numpy#25570 (comment)). The implementation closely follows the one for the pybind11 dependency.

[rgommers]

The two failing CI jobs are unrelated, those 3 failures are visible on other recent PRs as well.

dependency('pybind11') had no tests, so I didn't add any here for dependency('numpy') either. Would not be very useful now anyway, since the latest release on PyPI doesn't yet ship numpy-config. I did write a standalone tests and also tried it on another package I maintain, so I'm pretty sure this is in good shape.

[dcbaker]

It's really unfortunate that the pkg-config files end up somewhere hard to find. I wonder if we should do some internal checking to add python's site-packages to the pkg-config path for python based dependencies? Even if we do that's out of scope for this PR

[rgommers]

I wonder if we should do some internal checking to add python's site-packages to the pkg-config path for python based dependencies?

I would like to push that forward at some point. It needs a change/agreement on the Python packaging side though for what goes where. It'd make sense to have site-packages/lib/pkgconfig, or otherwise at least a per-package standard location. Right now it just goes wherever (for numpy it's site-packages/numpy/_core/lib/pkgconfig/numpy.pc, for pybind11 it's site-packages/pybind11/share/pkgconfig/), so there is no way for Meson to do anything reasonable.

[eli-schwartz]

It'd make sense to have site-packages/lib/pkgconfig, or otherwise at least a per-package standard location.

A per package "standard" location is essentially what pybind11 does, in the sense that {package}/ is treated effectively identical to --prefix and has a share/pkgconfig subdirectory + an include/ one.

A global one that covers all packages would not make much sense in site-packages since then users could do import lib and get a highly confusing namespace package, so instead it should just be:

• replace site-packages/ with sysconfig.get_path('data')

which means, use /usr/include 😜

[bruchar1]

I would also update the dependencies.md page...

[rgommers]

A global one that covers all packages would not make much sense in site-packages since then users could do import lib and get a highly confusing namespace package

That is not hard to avoid, e.g. by using site-packages/.pkgconfig or some other name that can't be confused with a regular package. If .pth files can be a thing inside site-packages, so can other mechanisms.

which means, use /usr/include 😜

I'm going to see if I can add a NumPy build option for that, but I do not have much hope of getting any kind of agreement on that from Python packaging folks.

I would also update the dependencies.md page...

Good catch, thanks. Added now.

[dnicolodi]

@rgommers Would it be possible to have the numpy-config tool backported to numpy 1.26 maybe in the form or a numpy-config extra package? The way the numpy headers have to be discovered with the current numpy release is very ugly. The only drawback I see is that I don't think the Python packaging dependencies specification is able to encode something like numpy >= 2.0 or (numpy >= 1.26.0 and numpy-config).

[rgommers]

Agreed that it is very ugly for older versions.

The only drawback I see is that I don't think the Python packaging dependencies specification is able to encode something like numpy >= 2.0 or (numpy >= 1.26.0 and numpy-config).

Yes, that's a problem. Also, it could only be in numpy 1.26.5, which I'm not sure is all that helpful here (too new) since many projects want to support numpy versions that are 1 or 2 years old.

The nicer alternative, which makes dependency('numpy') work for any version without the user having to do anything, is to also add a SystemDependency for it in Meson. I think that's worth doing as a follow-up. I thought about including that in this PR already, but was a bit short on time. It's not too difficult though. WDYT?

[dnicolodi]

Yes, that's a problem. Also, it could only be in numpy 1.26.5

That's the reason why I was thinking about distributing a numpy-config in a separate package installable alongside old releases of numpy. AFAIK the way the numpy headers are looked up hasn't changed in a long time (before numpy 2.0). Packages building against old numpy releases could build-depend on the numpy-config package.

Adding support in meson is also an option, especially if meson already has special handling for numpy anyway.

[rgommers]

I'll give it a go then, to prevent users having to add a separate dependency just for this.

[rgommers]

I put up gh-12891 with a numpy system dependency for discussion.