-
-
Notifications
You must be signed in to change notification settings - Fork 14.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
stdenv, gcc: rename postPostInstall -> preFixup #223634
Commits on Mar 17, 2023
-
Revert "nixos/lib/testing: set default timeout for VM tests"
This reverts commit 254426e.
Adam Joseph committedMar 17, 2023 Configuration menu - View commit details
-
Copy full SHA for 8b5d412 - Browse repository at this point
Copy the full SHA 8b5d412View commit details
Commits on Mar 19, 2023
-
trivial-builders/test/references.nix: fix eval
The command ``` nix-build -A tests.trivial-builders.references --show-trace ``` fails eval with ``` in job ‘nixpkgs.tests.trivial-builders.references’: error: The option `meta.description' does not exist. Definition values: - In `makeTest parameters': "Run the Nixpkgs trivial builders tests" ``` because `meta.description` and `meta.license` are not valid for `nixosTest`s (they are valid for `mkDerivation` of course). This has been causing Hydra eval failures: https://hydra.nixos.org/jobset/nixos/pr-209870-gcc-external-bootstrap#tabs-errors Let's fix eval by removing these attributes.
Adam Joseph committedMar 19, 2023 Configuration menu - View commit details
-
Copy full SHA for e33f2d1 - Browse repository at this point
Copy the full SHA e33f2d1View commit details -
test-defaultPkgConfigPackages.nix: filter out recurseForDerivations
The command at the top of this file fails to evaluate: ``` $ nix-build -A tests.pkg-config.defaultPkgConfigPackages in job ‘nixpkgs.tests.pkg-config.defaultPkgConfigPackages.tests-combined.x86_64-linux’: error: pkg-config module `recurseForDerivations` is not defined to be a derivation. Please check the attribute value for `recurseForDerivations` in `pkgs/top-level/pkg-config-packages.nix` in Nixpkgs. ``` This is also causing eval errors on Hydra: https://hydra.nixos.org/jobset/nixos/pr-209870-gcc-external-bootstrap#tabs-errors Let's filter out `recurseForDerivations=true` from the attrset, since it exists mainly as a flag to signal special handling when recursing.
Adam Joseph committedMar 19, 2023 Configuration menu - View commit details
-
Copy full SHA for 44137a7 - Browse repository at this point
Copy the full SHA 44137a7View commit details -
cc-wrapper: if isClang, add -L${gccForLibs.libgcc}/lib
When wrapping `clang` and using a `gccForLibs` whose `libgcc` is in its own output (rather than the `lib` output), this commit will adds `-L${gccForLibs.libgcc}/lib` to `cc-ldflags`. If that flag is not added, `firefox` will fail to compile because it invokes `clang-wrapper` with `-fuse-ld=lld` and passes `-lgcc_s` to `lld`, but does not tell `lld` where to find `libgcc_s.so`. In that situation, firefox will fail to link.
Adam Joseph committedMar 19, 2023 Configuration menu - View commit details
-
Copy full SHA for dc2a208 - Browse repository at this point
Copy the full SHA dc2a208View commit details -
ccache-links: inherit version to keep cc-wrapper happy
The Nix-driven bootstrap of gcc expects that `stdenv.cc.cc` has a `version` attribute, except if `stdenv.cc` is the `bootstrapFiles`. This commit causes `ccache-links`'s `cc-wrapper` to `inherit version` in order to satisfy that requirement. Without this commit, ofborg CI will fail. Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 3cddb6a - Browse repository at this point
Copy the full SHA 3cddb6aView commit details -
stdenv/linux: factor out commonGccOverrides
This commit has no effect on eval. It simply factors out a common subexpression.
Adam Joseph committedMar 19, 2023 Configuration menu - View commit details
-
Copy full SHA for 0f8923c - Browse repository at this point
Copy the full SHA 0f8923cView commit details -
This commit has no effect on eval. It simply reorganizes the `gcc11` and `gcc12` expressions so they apply a list of `overrideAttrs`. The list is currently empty.
Adam Joseph committedMar 19, 2023 Configuration menu - View commit details
-
Copy full SHA for f0be5a9 - Browse repository at this point
Copy the full SHA f0be5a9View commit details
Commits on Mar 28, 2023
-
stdenv: Nix-driven bootstrap of gcc
#### Summary By default, when you type `make`, GCC will compile itself three times. This PR inhibits that behavior by configuring GCC with `--disable-bootstrap`, and reimplements the triple-rebuild using Nix rather than `make`/`sh`. #### Immediate Benefits - Allow `gcc11` and `gcc12` on `aarch64` (without needing new `bootstrapFiles`) - Faster stdenv rebuilds: the third compilation of gcc (i.e. stageCompare) is no longer a `drvInput` of the final stdenv. This allows Nix to build stageCompare in parallel with the rest of nixpkgs instead of in series. - No more copying `libgcc_s` out of the bootstrap-files or other derivations - No more Frankenstein compiler: the final gcc and the libraries it links against (mpfr, mpc, isl, glibc) are all built by the same compiler (xgcc) instead of a mixture of the bootstrapFiles' compiler and xgcc. - No more [static lib{mpfr,mpc,gmp,isl}.a hack] - Many other small `stdenv` hacks eliminated - `gcc` and `clang` share the same codepath for more of `cc-wrapper`. #### Future Benefits - This should allow using a [foreign] `bootstrap-files` so long as `hostPlatform.canExecute bootstrapFiles`. - This should allow each of the libraries that ship with `gcc` (lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty, offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be built in separate (one-liner) derivations which `inherit src;` from `gcc`, much like #132343 #### Incorporates - #210004 - #36948 (unreverted) - #210325 - #210118 - #210132 - #210109 - #213909 - #216136 - #216237 - #210019 - #216232 - #216016 - #217977 - #217995 #### Closes - Closes #108305 - Closes #108111 - Closes #201254 - Closes #208412 #### Credits This project was made possible by three important insights, none of which were mine: 1. @Ericson2314 was the first to advocate for this change, and probably the first to appreciate its advantages. Nix-driven (external) bootstrap is "cross by default". 2. @trofi has figured out a lot about how to get gcc to not mix up the copy of `libstdc++` that it depends on with the copy that it builds, by moving the `bootstrapFiles`' `libstdc++` into a [versioned directory]. This allows a Nix-driven bootstrap of gcc without the final gcc would still having references to the `bootstrapFiles`. 3. Using the undocumented variable [`user-defined-trusted-dirs`] when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it uses a completely different and totally special set of rules for finding `libgcc_s.so`. This trick is the only way we can put `libgcc_s.so` in its own separate outpath without creating circular dependencies or dependencies on the bootstrapFiles. I would never have guessed to use this (or that it existed!) if it were not for a [comment in guix] which @Mic92 [mentioned]. My own role in this PR was basically: being available to go on a coding binge at an opportune moment, so we wouldn't waste a [crisis]. [aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938 [amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857 [nonexistent sysroot]: #210004 [versioned directory]: #209054 [`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html [comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253 [mentioned]: #210112 (comment) [crisis]: #108305 [foreign]: #170857 (comment) [static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
Adam Joseph committedMar 28, 2023 Configuration menu - View commit details
-
Copy full SHA for 131aa21 - Browse repository at this point
Copy the full SHA 131aa21View commit details -
This commit adds `gcc/common/checksum.nix`, which contains code common to both gcc11 and gcc12, implementing the `enableChecksum` feature. When gcc's built-in bootstrap (`--enable-bootstrap`) is used, gcc compiles itself three times and compares a hash of the unlinked `.o` files from the second and third compilation. The `enableChecksum=true` parameter performs the same comparison as part of the `postInstall` phase. Notably, `enableChecksum=true` can be used with `enableBootstrap=false`. Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for d193b8a - Browse repository at this point
Copy the full SHA d193b8aView commit details -
pkgs/test/stdenv/default.nix: add gcc-stageCompare
This commit adds a derivation `gcc-stageCompare` to `pkgs/test/stdenv/default.nix`. It is important to always build this derivation whenever building `stdenv`! Because we are using a Nix-driven bootstrap instead of gcc's built-in `--enable-bootstrap`, the `gcc` derivation no longer performs the post-self-compilation sanity check. You must build this derivation in order to perform that sanity check. The major benefit of this new approach is that the sanity check (which involves a third compilation of gcc) can be performed *concurrently* with all packages that depend on `stdenv`, rather than serially. Since `stdenv` has very little derivation-level parallelism it cannot take advantage of more than one or perhaps two builders. If you have three or more builders this commit will reduce the time-to-rebuild-stdenv by around 20% (one of three gcc rebuilds is removed from the critical path, and stdenv's build time is dominated by roughly 3*gcc + 1*binutils + 1*bison-test-suite). Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 09d06cc - Browse repository at this point
Copy the full SHA 09d06ccView commit details -
emacs: path fixes resulting from libgccjit changes
The Nix-driven bootstrap of gcc resulted in some changes to the structure of the `libgccjit` outpaths, and also added an additional output (`libgcc`) to `gcc`. This commit makes the corresponding changes in the `emacs` derivation in order to not break emacs. Emacs is the only user of `libgccjit` in nixpkgs at the moment.
Adam Joseph committedMar 28, 2023 Configuration menu - View commit details
-
Copy full SHA for bf99025 - Browse repository at this point
Copy the full SHA bf99025View commit details -
make-bootstrap-tools.nix: follow libgcc_s symlink
Adam Joseph committedMar 28, 2023 Configuration menu - View commit details
-
Copy full SHA for 4b2dc7f - Browse repository at this point
Copy the full SHA 4b2dc7fView commit details -
stdenv, gcc: rename postPostInstall -> preFixup
This commit implements @SuperSandro2000's suggestion here: #209870 (review) I am submitting it as a separate PR in order to avoid a full rebuild on Hydra.
Adam Joseph committedMar 28, 2023 Configuration menu - View commit details
-
Copy full SHA for 132e7d1 - Browse repository at this point
Copy the full SHA 132e7d1View commit details