stdenv: delete the NIX_COREFOUNDATION_RPATH sledgehammer

[eliasnaur]

Similar to PR #223861 and the issue #221350, NIX_COREFOUNDATION_RPATH is set unconditionally in the darwin stdenv. This is wrong for cross-compiles and results in rpaths depending on the builder platform.

This change makes another bold move and deletes the NIX_COREFOUNDATION_RPATH facility completely, in the hope it is no longer needed, or that any breakage is manageable.

As an added bonus we can delete a darwin specific hack in glibc.

Description of changes

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 23.05 Release Notes (or backporting 22.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

[eliasnaur]

Gentle ping, and CC'ing @trofi who had great insights on the darwin subtleties in a related change, #226290.

[trofi]

Afraid I know nothing about native builds or macos frameworks. Let's ask @NixOS/darwin-maintainers

[toonn]

I think this warrants a Hydra jobset.

[eliasnaur]

I agree this PR needs wide testing before merging. What's the procedure for a hydra test run?

[winterqt]

Paging @vcunat, can we get a jobset? :)

[veprbl]

See also #111385

[vcunat]

Risky changes to core packages are not allowed now. I don't know darwin stuff really, but at least it's my understanding that this PR can't be merged for 23.05 anymore, according to the schedule: #223562

So perhaps wait a couple of weeks? Either way, for the jobset it's better to put the tested commits atop some nixpkgs version that's known to be OK-ish (e.g. master), so that it doesn't mix failures coming from various sources. For example, even stdenv won't build for x86_64-darwin on the current staging-next branch (and staging probably as well).

[eliasnaur]

Risky changes to core packages are not allowed now. I don't know darwin stuff really, but at least it's my understanding that this PR can't be merged for 23.05 anymore, according to the schedule: #223562

I don't mind waiting a month to get this merged, but I'd like to shake out problems with the PR before that.

As you can tell from the PR description and commit, I'm surprised this change works at all. So my goal with wider testing is to pinpoint the need for NIX_COREFOUNDATION_RPATH, if any. A Hydra run seems perfect for that. I'm also happy with a reviewer pointing out the folly of my ways, or if you can provide me with (a set of) packages that would likely break with this change.

So perhaps wait a couple of weeks?

For merging, no problem.

Either way, for the jobset it's better to put the tested commits atop some nixpkgs version that's known to be OK-ish (e.g. master), so that it doesn't mix failures coming from various sources. For example, even stdenv won't build for x86_64-darwin on the current staging-next branch (and staging probably as well).

Rebased to master.

[vcunat]

OK, so let's do just aarch64-darwin in the first step, I guess: https://hydra.nixos.org/eval/1793921?compare=1793915

[eliasnaur]

Thank you.

I'm unsure how to interpret the results: there are thousands of "newly failing" jobs, but they seem to be timeouts. I used "compare to..." to compare with "aarch64-darwin", giving me

https://hydra.nixos.org/eval/1793921?compare=aarch64-darwin&full=0

The result is 130 "newly failing", but the few I sampled were also timeouts.

[toonn]

@eliasnaur, you should compare to an evaluation of the specific commit you based your changes on. If you base a branch on master you'll sift through evaluation of nixpkgs:trunk to find that.

And yes, sampling a couple failures is pretty much what you do and then follow the propagated from links usually.

I'm not sure how expected the timeouts are or what to do about them.

Due to the PR veprbl linked I'm unsure whether we would actually see failures from this change. I think it might just link to system frameworks impurely?

[vcunat]

you should compare to an evaluation of the specific commit you based your changes on.

That's the link I posted.

Such timeouts are normal. The darwin builders are just less reliable than linux ones 🤷🏽 But this time it was unlucky to happen during stdenv bootstrapping, so everything got failed. I'll make sure it gets built and then restart everything.

[vcunat]

Uh, another unexplained timeout causing mass failure, another restart of everything. Well, just ping me if it repeats.

[eliasnaur]

I rebuilt a handful of the "newly failing" python packages and kitty, and they all succeeded. I also enabled sandbox and rebuilt another handful, no difference. I rebased to master; can we try again or is there some other way for me to reproduce the failing builds?

My reference: https://hydra.nixos.org/eval/1793921?compare=1793915#tabs-now-fail

[toonn]

Normally jobsets are evaluated on a schedule and when there's changes to the branch a full evaluation gets run. So an evaluation should be triggered eventually. The only way to speed that up is by getting the jobset owner to do so manually.

[eliasnaur]

Normally jobsets are evaluated on a schedule and when there's changes to the branch a full evaluation gets run. So an evaluation should be triggered eventually. The only way to speed that up is by getting the jobset owner to do so manually.

Thanks. I don't mind waiting.

[vcunat]

Well, I could've restarted the failures, but your push makes everything rebuild from scratch (even successes).

[eliasnaur]

Well, I could've restarted the failures, but your push makes everything rebuild from scratch (even successes).

Yes, I see that now. Sorry for exposing you to my learning curve.

Due to the PR veprbl linked I'm unsure whether we would actually see failures from this change. I think it might just link to system frameworks impurely?

For some reason I didn't see this comment before, or I didn't understand its implications. Indeed, it does seem that @veprbl's #111385 is a better PR. I shall close this for now and push forward in that PR.