Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python310Packages.aiohttp: add missing build inputs: setuptools, wheel #212602

Closed
wants to merge 1 commit into from
Closed

python310Packages.aiohttp: add missing build inputs: setuptools, wheel #212602

wants to merge 1 commit into from

Conversation

ghost
Copy link

@ghost ghost commented Jan 25, 2023

Description of changes

Builds of aiohttpd have been failing on powerpc64le since 4553e62 with:

python3.10-aiohttp>   File "/nix/store/mhw4rs4xb20acvn55iw5i5k5rhaqidnq-python3-3.10.9/lib/python3.10/importlib/__init__.py", line 126, in import_module
python3.10-aiohttp>     return _bootstrap._gcd_import(name[level:], package, level)
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
python3.10-aiohttp> ModuleNotFoundError: No module named 'setuptools'
python3.10-aiohttp> note: keeping build directory '/nix/tmp/nix-build-python3.10-aiohttp-3.8.3.drv-5'

This commit adds python3Packages.setuptools as a propagatedBuildInput, which many other python packages seem to do. It fixes the build. I don't know much python, so if there is a better/preferred way to address this please let me know.

Things done

@ghost
Copy link
Author

ghost commented Jan 25, 2023

Let me know if this should go to staging instead. It isn't urgent.

@FRidh
Copy link
Member

FRidh commented Jan 25, 2023

Please share the full trace. It is not obvious here where it fails. I think it fails during a test and not the build but cannot be sure from this.

@ghost
Copy link
Author

ghost commented Jan 25, 2023

Please share the full trace.

python3.10-aiohttp> Sourcing python-remove-tests-dir-hook
python3.10-aiohttp> Sourcing python-catch-conflicts-hook.sh
python3.10-aiohttp> Sourcing python-remove-bin-bytecode-hook.sh
python3.10-aiohttp> Sourcing pip-build-hook
python3.10-aiohttp> Using pipBuildPhase
python3.10-aiohttp> Using pipShellHook
python3.10-aiohttp> Sourcing pip-install-hook
python3.10-aiohttp> Using pipInstallPhase
python3.10-aiohttp> Sourcing python-imports-check-hook.sh
python3.10-aiohttp> Using pythonImportsCheckPhase
python3.10-aiohttp> Sourcing python-namespaces-hook
python3.10-aiohttp> Sourcing python-catch-conflicts-hook.sh
python3.10-aiohttp> unpacking sources
python3.10-aiohttp> unpacking source archive /nix/store/i4j1w19y8nkdwjz0pa7491gzwyr3pydi-aiohttp-3.8.3.tar.gz
python3.10-aiohttp> source root is aiohttp-3.8.3
python3.10-aiohttp> setting SOURCE_DATE_EPOCH to timestamp 1663765788 of file aiohttp-3.8.3/setup.cfg
python3.10-aiohttp> patching sources
python3.10-aiohttp> updateAutotoolsGnuConfigScriptsPhase
python3.10-aiohttp> configuring
python3.10-aiohttp> no configure script, doing nothing
python3.10-aiohttp> building
python3.10-aiohttp> Executing pipBuildPhase
python3.10-aiohttp> Creating a wheel...
python3.10-aiohttp> WARNING: The directory '/homeless-shelter/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
python3.10-aiohttp> Processing /build/aiohttp-3.8.3
python3.10-aiohttp>   Running command Preparing metadata (pyproject.toml)
python3.10-aiohttp>   Preparing metadata (pyproject.toml) ... done
python3.10-aiohttp> ERROR: Exception:
python3.10-aiohttp> Traceback (most recent call last):
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 160, in exc_logging_wrapper
python3.10-aiohttp>     status = run_func(*args)
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
python3.10-aiohttp>     return func(self, options, args)
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/commands/wheel.py", line 170, in run
python3.10-aiohttp>     requirement_set = resolver.resolve(reqs, check_supported_wheels=True)
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 73, in resolve
python3.10-aiohttp>     collected = self.factory.collect_root_requirements(root_reqs)
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 491, in collect_root_requirements
python3.10-aiohttp>     req = self._make_requirement_from_install_req(
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 453, in _make_requirement_from_install_req
python3.10-aiohttp>     cand = self._make_candidate_from_link(
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
python3.10-aiohttp>     self._link_candidate_cache[link] = LinkCandidate(
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 297, in __init__
python3.10-aiohttp>     super().__init__(
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 162, in __init__
python3.10-aiohttp>     self.dist = self._prepare()
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 231, in _prepare
python3.10-aiohttp>     dist = self._prepare_distribution()
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 308, in _prepare_distribution
python3.10-aiohttp>     return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 491, in prepare_linked_requirement
python3.10-aiohttp>     return self._prepare_linked_requirement(req, parallel_builds)
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 577, in _prepare_linked_requirement
python3.10-aiohttp>     dist = _get_prepared_distribution(
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 69, in _get_prepared_distribution
python3.10-aiohttp>     abstract_dist.prepare_distribution_metadata(
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/distributions/sdist.py", line 61, in prepare_distribution_metadata
python3.10-aiohttp>     self.req.prepare_metadata()
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/req/req_install.py", line 535, in prepare_metadata
python3.10-aiohttp>     self.metadata_directory = generate_metadata(
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/operations/build/metadata.py", line 35, in generate_metadata
python3.10-aiohttp>     distinfo_dir = backend.prepare_metadata_for_build_wheel(metadata_dir)
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_internal/utils/misc.py", line 706, in prepare_metadata_for_build_wheel
python3.10-aiohttp>     return super().prepare_metadata_for_build_wheel(
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_vendor/pep517/wrappers.py", line 189, in prepare_metadata_for_build_wheel
python3.10-aiohttp>     return self._call_hook('prepare_metadata_for_build_wheel', {
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_vendor/pep517/wrappers.py", line 319, in _call_hook
python3.10-aiohttp>     raise BackendUnavailable(data.get('traceback', ''))
python3.10-aiohttp> pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
python3.10-aiohttp>   File "/nix/store/f2qkcpfq4bm1pdjld4yrbrbwnqsihpdl-python3.10-pip-22.3.1/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 77, in _build_backend
python3.10-aiohttp>     obj = import_module(mod_path)
python3.10-aiohttp>   File "/nix/store/mhw4rs4xb20acvn55iw5i5k5rhaqidnq-python3-3.10.9/lib/python3.10/importlib/__init__.py", line 126, in import_module
python3.10-aiohttp>     return _bootstrap._gcd_import(name[level:], package, level)
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
python3.10-aiohttp> ModuleNotFoundError: No module named 'setuptools'
python3.10-aiohttp> note: keeping build directory '/nix/tmp/nix-build-python3.10-aiohttp-3.8.3.drv-7'
error: build of '/nix/store/8smpkz73w7aa50gkkrpdcxk51ic33whd-python3.10-aiohttp-3.8.3.drv' on 'ssh://nix@stallman' failed: error: builder for '/nix/store/8smpkz73w7aa50gkkrpdcxk51ic33whd-python3.10-aiohttp-3.8.3.drv' failed with exit code 2;
              last 10 log lines:
              >   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
              >   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
              >   File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
              >   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
              >   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
              >   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
              >   File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
              > ModuleNotFoundError: No module named 'setuptools'
              >
              >
              For full logs, run 'nix log /nix/store/8smpkz73w7aa50gkkrpdcxk51ic33whd-python3.10-aiohttp-3.8.3.drv'.
error: builder for '/nix/store/8smpkz73w7aa50gkkrpdcxk51ic33whd-python3.10-aiohttp-3.8.3.drv' failed with exit code 1;
       last 10 log lines:
       >   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
       >   File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
       >   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
       >   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
       >   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
       >   File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
       > ModuleNotFoundError: No module named 'setuptools'
       >
       >
       > note: keeping build directory '/nix/tmp/nix-build-python3.10-aiohttp-3.8.3.drv-7'
       For full logs, run 'nix log /nix/store/8smpkz73w7aa50gkkrpdcxk51ic33whd-python3.10-aiohttp-3.8.3.drv'.

@FRidh
Copy link
Member

FRidh commented Jan 25, 2023

Thanks. Checking the trace, it seems to fail during wheel building. Looking at the expression, it seems the build system is not added. It does manage to get access to it via the runtime dependencies (propagatedBuildInputs), at least on other platforms, but that's not how it is supposed to be.

You can add instead of your current fix

nativeBuildInputs = [
  setuptools
  wheel
];

to solve this.

Also please follow the contribution guidelines when it comes to the commit title.

@dotlambda dotlambda changed the title python-modules/aiohttpd: add missing setuptools buildinput python310Packages.aiohttpd: add missing setuptools build input Jan 26, 2023
@dotlambda dotlambda changed the title python310Packages.aiohttpd: add missing setuptools build input python310Packages.aiohttp: add missing setuptools build input Jan 26, 2023
Copy link
Member

@dotlambda dotlambda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also please follow the contribution guidelines when it comes to the commit title.

See PR title.

@@ -46,6 +48,11 @@ buildPythonPackage rec {
--replace "charset-normalizer >=2.0, < 3.0" "charset-normalizer >=2.0, < 4.0"
'';

nativeBuildInputs = [
setuptools
wheel
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@FRidh Why wheel? It's not specified in upstream's pyproject.toml.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The wheel package is always needed to make a wheel. Maybe we get away with it because pip that we instruct to build the wheel brings it in. I don't know anymore.

@ghost ghost changed the title python310Packages.aiohttp: add missing setuptools build input aiohttp: add missing build inputs: setuptools, wheel Jan 26, 2023
@ghost ghost requested review from dotlambda and removed request for FRidh and jonringer January 26, 2023 12:52
Copy link
Member

@dotlambda dotlambda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

commit message should start with e.g. python310Packages.. Please target staging.

Builds of aiohttpd have been failing on powerpc64le since
4553e62 with:

```
python3.10-aiohttp>   File "/nix/store/mhw4rs4xb20acvn55iw5i5k5rhaqidnq-python3-3.10.9/lib/python3.10/importlib/__init__.py", line 126, in import_module
python3.10-aiohttp>     return _bootstrap._gcd_import(name[level:], package, level)
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
python3.10-aiohttp>   File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
python3.10-aiohttp> ModuleNotFoundError: No module named 'setuptools'
python3.10-aiohttp> note: keeping build directory '/nix/tmp/nix-build-python3.10-aiohttp-3.8.3.drv-5'
```

This commit adds `setuptools` and `wheel` as `nativeBuildInputs` to
fix this.
@ghost ghost changed the title aiohttp: add missing build inputs: setuptools, wheel python310Packages.aiohttp: add missing build inputs: setuptools, wheel Jan 27, 2023
@ghost ghost marked this pull request as draft January 27, 2023 02:25
@ghost ghost changed the base branch from master to staging January 27, 2023 02:26
@ghost ghost marked this pull request as ready for review January 27, 2023 02:26
@ghost ghost requested a review from dotlambda January 27, 2023 02:26
Copy link
Contributor

@lopsided98 lopsided98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I confirmed that this fixes cross-compiling to armv6l-linux. The build still succeeds without wheel, but I don't know why it should or shouldn't be included.

@ghost
Copy link
Author

ghost commented Apr 24, 2023

This can wait until after ZHF.

@ghost ghost marked this pull request as draft April 24, 2023 05:27
Copy link
Member

@FRidh FRidh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not tested but I don't see any reason why this would break anything.

@FRidh
Copy link
Member

FRidh commented Aug 11, 2023

Fixed with #247310

@FRidh FRidh closed this Aug 11, 2023
@ghost ghost deleted the pr/python/aiohttpd/fix branch January 23, 2024 06:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants