enable RPATH linking by default

[branfosj]

I've marked this as draft while we discuss changing this default in EB5.

[Micket]

1. we are keeping LD_LIBRARY_PATH
2. this just affects stuff we built (no attempts to elf-patch binaries)
3. should be fine to introduce in existing software trees (it of course won't affect things retroactively, but new things will build with rpath enabled).
4. No attempts to change how users link against modules when building their own code.

Positives:
https://docs.easybuild.io/rpath-support/#rpath_support_why
though primarily i think its main advantage is to built libraries and binaries more robust against users fiddling with LD_LIBRARY_PATH.

Downsides:

1. Higher burden when making new easyconfigs due to the code that ensures things did get rpath'ed, contributions test builds will non-rpath'ed linking has occurred. e.g. eb git-lfs-3.2.0.eb --rpath fails on RPATH sanity check easybuild-easyconfigs#17516 . One could argue we would rather catch these sooner, but it would also affect every novice user of EB building local easyconfigs.
2. Maybe we actually rather would want a different approach like: also support using full path soname for shared libraries as alternative to using RPATH #4122 in which case it might hurt to set this new default if there is a better option around the corner.

[jfgrimm]

I'd support this.

For reference, we enabled rpath on our new cluster (without filtering LD_LIBRARY_PATH), and have zero negative feedback from users.
EasyBuild already skips the rpath check when using the Binary easyblock, so I haven't run into many issues at all (2 failed on the rpath check out of ~1000 unique software, ~4000 total modules).

I don't think we should worry about holding back to wait for #4122, that can always be added later (and would probably get more attention with rpath being on by default)

[ocaisa]

I'm also in general supportive, but I do think there needs to be space in some cases to be able to override libraries with an ABI compatible alternative (MPI in particular). Current rpath support does make this possible (and we use that in EESSI). If we do switch to #4122 at some point, we should still be able to retain this possibility.

[boegel]

I'm fine with this, as long as we add a very clear section to the documentation that explains our motivation for doing this, outline the (mostly positive) impact this has. That section should include clear instructions on how to switch back to the old behavior of not using RPATH linking, or how to disable it for specific easyconfigs (by setting the rpath toolchain option to False).

We should also only have it enabled by default on Linux, and keep it disabled on macOS (where it doesn't work at all).

[branfosj]

• 612b1c0 enables / disables this for not-Mac / Mac
• RPATH is enabled by default in EasyBuild v5.0 easybuild-docs#241 adds documentation

[boegel]

lgtm