Release 1.1: Regression breaks setuptools builds

[jaraco]

In pypa/setuptools#4333, setuptools reports some emergent failures implicating the recent pyproject-hooks 1.1 release.

[jaraco]

I suspect this change is relevant.

• More careful handling of the backend-path key from pyproject.toml.
  Previous versions would load the backend and then check that it was loaded
  from the specified path; the new version only loads it from the specified path.
  The BackendInvalid exception is now a synonym for :exc:BackendUnavailable,
  and code should move to using the latter name.

[pradyunsg]

#165 is the relevant PR -- /cc @abravalheri for awareness.

It's unclear to me what exactly is happening here from the setuptools failure, so not quite sure what the fix should/would be?

[jaraco]

My initial thought is that setuptools' builds were previously relying on metadata from an existing installation... and this change keeps that metadata hidden so that all that's around is the bootstrap metadata, which isn't sufficient to build setuptools (maybe?).

[jaraco]

Or, maybe the changes to avoid altering sys.path cause the metadata to be missing altogether.

[pradyunsg]

The sys.path with 1.0.0:

['/Users/pradyunsg/Developer/github/setuptools', '/Users/pradyunsg/Developer/github/setuptools/.venv/lib/python3.12/site-packages/pyproject_hooks/_in_process', '/Users/pradyunsg/.asdf/installs/python/3.12.0/lib/python312.zip', '/Users/pradyunsg/.asdf/installs/python/3.12.0/lib/python3.12', '/Users/pradyunsg/.asdf/installs/python/3.12.0/lib/python3.12/lib-dynload', '/Users/pradyunsg/Developer/github/setuptools/.venv/lib/python3.12/site-packages']

The sys.path with 1.1.0:

['/Users/pradyunsg/Developer/github/setuptools/.venv/lib/python3.12/site-packages/pyproject_hooks/_in_process', '/Users/pradyunsg/.asdf/installs/python/3.12.0/lib/python312.zip', '/Users/pradyunsg/.asdf/installs/python/3.12.0/lib/python3.12', '/Users/pradyunsg/.asdf/installs/python/3.12.0/lib/python3.12/lib-dynload', '/Users/pradyunsg/Developer/github/setuptools/.venv/lib/python3.12/site-packages']

It's... suboptimal... that _in_process is on sys.path with this setup.

[pradyunsg]

Yup, it's the changeover to using sys.meta_path rather than sys.path for loading the backend that's affected things here.

[jaraco]

I suspect adding a find_distributions (i.e. making it an importlib.metadata.DistributionFinder) will allow it to find the metadata for a backend and will likely address the concern.

[pradyunsg]

@mdagost That sounds unrelated to this project, and specific to how Poetry is handling its environment and lockfile.

Please, let's keep any further Poetry-specific discussion in the Poetry issue tracker or the other issue in this repo.

[mdagost]

Correct. The root root cause was managing poetry with the lockfile, which is no bueno.

[jacobfriedman]

CPython should stop shipping with prebuilt pip and invoking an install of prebuilt setuptools. Even Fedora has a problem with this.

[nanonyme]

@jacobfriedman CPython shipping pip has absolutely nothing to do with this problem, this same issue happens with pyproject stack with no pip and only build+installer.