minimal-bootstrap: component packages for bootstrap-tools

[emilytrau]

Description of changes

Packages many components of bootstrap-tools as statically linked binaries. The only missing tool is a gcc that targets glibc, I'm still working on understanding how compiler/libc split works 😅

Related #227914

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 23.11 Release Notes (or backporting 23.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

[06kellyjac]

Very cool. Some of this looks a decent amount simpler e.g. binutils vs binutils-static.

Thoughts are do we need to keep the non-static version? or are these sufficiently divergent that we can't do an enableStatic ? true type thing? (supporting both static and non-static is more work but that's also what we have ATM with the packages here that have both copies)

[emilytrau]

The main pain point i've found with doing something like an enableStatic variant is long cyclic dependency chains that we can avoid by maintaining a second copy. At the cost of some duplication the development cycle is much nicer

[06kellyjac]

cool cool. any thoughts on dropping some of the non-static copies that aren't used?

[Ericson2314]

Yeah I agree it would be good to drop the non-static versions of things if possible.

Also the current (non-mimimal, not your work) bootstrap tools design of throwing everything together in one FHS tree is bad and we should break with it.

[Ericson2314]

The only missing tool is a gcc that targets glibc, I'm still working on understanding how compiler/libc split works 😅

It is not yet good enough for GCC. If we can make it better and a stronger split rather than try to understand what is there, that is good with me.

[emilytrau]

cool cool. any thoughts on dropping some of the non-static copies that aren't used?

This is good feedback and something that could use better clarity. The -static suffix isn't properly descriptive of the purpose, which is to rebuild a non-hacky version with gcc that is suitable to be used "externally" to minimal-bootstrap. I'm not sure what to call this.

[Ericson2314]

So you are saying regardless of the naming we currently need all these duplicated package definitions for the next round of bootstrapping?

[emilytrau]

Indeed, to provide a stronger assurance that minimal-bootstrap can be a self-contained abstraction and that the tools provided as the interface don't leak any unexpected behaviour/bugs. Would love to take ideas for alternate names or ways to organise the files so that the intent is clearer

[SuperSandro2000]

Can this be deduped with the musl variant?

[emilytrau]

There's a motivation to prevent dependency cycles and rebuild loops, and favouring simpler code over deduplication. Important to the "trust" factor will be the the ability for bootstrap to be readable and understandable by end users who wish to, and duplicating code so as to not need to jump between files helps with that I think.

Hope this explanation makes sense? It's something that I need to write up as part of the documentation deliverable for the STF fund.

[Ericson2314]

There's a motivation to prevent dependency cycles and rebuild loops, and favouring simpler code over deduplication.

Per what I wrote in #260193 (comment), I rather have duplication than simpler code.

My guess is that code I wrote was getting within @emilytrau's way, which is understandable. So I think it would be good if newer deduplication we had was driven by @emilytrau so the burden of untangling on first readthrough will be on the rest of us, not her.

In previous PRs I was saying this can wait to the end, but now I that we are getting nearer a combinatorial explosion of stuff, I am getting wary about continuing to wait.

[Ericson2314]

FWIW I think we can do better than the rest of Nixpkgs by preferring Nix-language-level than shell-level abstractions. The benefit is then the final produced code that is in the derivation (try nix derivation show | jq ...) will have all those abstractions removed, and what is left is simple (but verbose) commands! Best of both worlds!

[SuperSandro2000]

I am not sure how much this is in scope but maybe applying patches would be a candidate to move into some wrapper around runCommand to reduce duplication?

[emilytrau]

This is a code style choice where we don't create abstractions over the run command semantics so that it can be easily understood when reading the code

[Ericson2314]

I would rather have simple forest and complex trees, than simple trees and complex forest, if that makes sense.

If you do anything at all, no matter how baroque, to deduplicate these packages, I will probably prefer it. I find it easier to read.

[ghost]

Congratulations on getting near the finish line!

Maybe an English nitpick, but is there a proof somewhere that no lesser bootstrap is possible? That's what minimal means (an element of a partial order such that any other element less-than-or-equal-to it is equal to it).

• https://en.wikipedia.org/wiki/Minimal_element

I think you might have meant minimalist or minimized or something else.

I didn't bring this up earlier since until now there appeared to be more pressing problems, but it looks like those have all been conquered!

I only point it out because, although this is quite an achievement, the entire nix binary and all the libraries it links are still part of the not-source-code bootstrap input. The name of this project seems to imply that isn't possible, but in fact guix has already shown that it is.

If we were to someday have a bootstrap starting from only a scheme/forth interpreter written in hex0, would that be less-than-minimal-bootstrap?

[mohe2015]

You could also argue that this simply means the minimal bootstrap upstreamed in nixpkgs.

[philiptaron]

@emilytrau -- would you mind if I restarted some work here?

[emilytrau]

@philiptaron please and thank you