pythonPackages: ensure all derivations provide python modules #194205
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
13 tasks
|
Wondering if the eval failure is the result of the one remaining application package: openrazer-daemon. If so, this is pretty hard to glean from the trace. |
|
Ha, no, this test fails because I forgot about setuphooks. In this case the I wonder how to exempt those... We really need types in Nixpkgs so we can check them and filter based on them. |
|
Is there an eval penalty to this? and if so how high would it be? |
Additional eval load is negligible for this.
Other solutions:
Some packages to consider as well:
|
|
|
FRidh
force-pushed
the
ensure
branch
4 times, most recently
from
7e97f7e
to
9566aa4
Compare
ofborg
bot
added
10.rebuild-darwin: 1-10
10.rebuild-linux: 11-100
and removed
10.rebuild-darwin: 0
10.rebuild-linux: 0
labels
This adds a test to ensure no new uses of `buildPythonApplication` can be added to `python-packages.nix`. Python packages can be grouped into two groups: 1) applications and 2) packages providing importable modules. In `python-packages.nix` we only want to have 2). 1) should be in the top-level package set. To achieve this, all setup hooks need to be marked as being a setup hook. For the setup hooks in the Python packages set this is done by creating a new builder, `makePythonHook`. Because there were issues with splicing, the file importing all the hooks is converted to an extension. All non-packages were moved out of `python-packages.nix` into `python-packages-base.nix`. The `keep` argument to `makeScopeWithSplicing was cleaned up as well; there is no need to keep this one manually in sync reducing the risk of breaking cross-compilation.
|
Everything seems to be OK now. I am surprised to see rebuilds though. |
|
Some packages now get a duplicate package in a closure. I guess an incorrect hook is being used. |
- Don't call from a different scope - Disable catching of conflicts because we do get a (installation-time only) conflict due to overrides in awscli.
This is the recommended way. It won't actually matter here though.
|
Bytecode has differences in filenames, but that would be fixed if we would build wheels as intermediate step and use CA then. |
|
This seems to have broken |
|
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/i-cant-update-my-system-due-to-buildpythonpackage/22886/7 |
gador
pushed a commit
to gador/nixpkgs
that referenced
this pull request
Nov 4, 2022
|
this broke the fix i did in #196052 (comment) before 33d12e5: nix-repl> pkgsCross.aarch64-multiplatform.python3Packages.xpybutil.nativeBuildInputs
[ «derivation /nix/store/hvb9yxgv1133cfhxxd869sibldvv2vdx-python3-3.10.7.drv» «derivation /nix/store/v880cnh4ml7czmivfbk3cdh93hz9yvbn-hook.drv» «derivation /nix/store/f243ab7wv92gqsmc9h7gr0qcnj0xcgdb-hook.drv» «derivation /nix/store/880lf8895bzn8d94lrr2y7ilgkxq0lc4-python-remove-tests-dir-hook.drv» «derivation /nix/store/fzjnhawfs1wpw58hcd1vxd9y750dc08y-python-remove-bin-bytecode-hook.drv» «derivation /nix/store/n8l59iparx98yfw8g5ydqmzmk3fdic75-setuptools-setup-hook.drv» «derivation /nix/store/7vyhynlaxlva1rpvfl0hpdl3qywi9qmc-pip-install-hook.drv» «derivation /nix/store/0l76r5pf69smrl631qa801sjkyb8isk6-python-namespaces-hook.sh.drv» «derivation /nix/store/zipz2sj6ncyllb2z53p0fmsirq601rj7-python-output-dist-hook.drv» ]after 33d12e5: nix-repl> pkgsCross.aarch64-multiplatform.python3Packages.xpybutil.nativeBuildInputs
[ «derivation /nix/store/bhz39ds4v02hn6x4py4mzjyilw4a589h-python3-aarch64-unknown-linux-gnu-3.10.7.drv» «derivation /nix/store/q0pdz5z18bfra7841qwdqq9i3g5na3w7-hook.drv» «derivation /nix/store/f243ab7wv92gqsmc9h7gr0qcnj0xcgdb-hook.drv» «derivation /nix/store/880lf8895bzn8d94lrr2y7ilgkxq0lc4-python-remove-tests-dir-hook.drv» «derivation /nix/store/fzjnhawfs1wpw58hcd1vxd9y750dc08y-python-remove-bin-bytecode-hook.drv» «derivation /nix/store/x9b2q51spifxvzsxj90if1h4nb617pys-setuptools-setup-hook.drv» «derivation /nix/store/zsh90vnqf3zblmrlyh8py5hxky2bk78a-pip-install-hook.drv» «derivation /nix/store/a4cyp1fcsnlppzrwwbk1ka5xhbj52hnm-python-namespaces-hook.sh.drv» «derivation /nix/store/2hc97nsdi375cv13014c4lardpxk2mgw-python-output-dist-hook.drv» ] |
Artturin
reviewed
Nov 17, 2022
| inherit toPythonModule toPythonApplication; | ||
| inherit buildSetupcfg; | ||
|
|
||
| python = toPythonModule python; |
There was a problem hiding this comment.
toPythonModule is probably discarding the splicing and that's why the fix is broken
Will test later
12 tasks
12 tasks
Artturin
added a commit
to Artturin/nixpkgs
that referenced
this pull request
Jul 26, 2023
`lib.extends hooks pythonPackagesFun` includes python which we want to splice what this fixes: ``` nix-repl> lib.elemAt pkgsCross.aarch64-multiplatform.python3Packages.xpybutil.nativeBuildInputs 0 «derivation /nix/store/39dkb51rciw6zwg0c2c44gpmpjapddxc-python3-aarch64-unknown-linux-gnu-3.10.12.drv» ``` to ``` nix-repl> lib.elemAt pkgsCross.aarch64-multiplatform.python3Packages.xpybutil.nativeBuildInputs 0 «derivation /nix/store/21ldw2dp26xvv9iyxn9x77a8yh4waqz5-python3-3.10.12.drv» ``` Before NixOS#194205 The keep python used to work by accident because self was passed from __splicedPackages https://github.com/NixOS/nixpkgs/pull/196052/files#diff-44ce3495c4f983ce64dd47c86a9d3e77bad210b2709c098a3806998dcd9b000bR213 But now it does not work because overrideAttrs is used on python in python-packages-base.nix Preferably we would [ splice the hooks too ](NixOS#228139) but we cannot do that until [Package sets within derivations (i.e. python3.pkgs) are not spliced](NixOS#211340) is fixed, because people often use `python3.pkgs.wrapPython` in `nativeBuildInputs` (it's correct but python3.pkgs should be python3Packages to get splicing.
rtimush
pushed a commit
to rtimush/nixpkgs
that referenced
this pull request
Sep 21, 2023
13 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds a test to ensure no new uses of
buildPythonApplicationcan be added topython-packages.nix.Python packages can be grouped into two groups: 1) applications and 2) packages providing importable modules. In
python-packages.nixwe only want to have 2). 1) should be in the top-level package set.Description of changes
Things done
sandbox = trueset innix.conf? (See Nix manual)nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)nixos/doc/manual/md-to-db.shto update generated release notes