Let --remote_download_symlink_template use hard links #12566
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In an attempt to achieve 'Builds without the Bytes' without losing
access to build outputs, I am experimenting with a FUSE file system that
gives access to objects stored in the CAS. In PR #11703, I added a
command line flag to let Bazel emit symbolic links pointing into this
FUSE file system, as opposed to downloading files into the exec root.
Though this change has allowed me to get quite a lot of stuff working,
there are also many build actions that break. For example, Python calls
realpath(argv[0]) to figure out its installation path. Because the FUSE
file system does not mimic the execroot, Python won't be able to find
its site packages. Similar problems hold with shared library resolution
in general.
This is why I think the only proper way we can get this to work is by
using hard links instead of symbolic links. That way the usual file
hierarchy remains intact. This change renames the
--remote_download_symlink_template flag to
--remote_download_hard_link_template and changes the code to create hard
links instead.
When used in combination with --exec_root_base (#12558), it's now
possible to let Bazel construct an exec root that does not have any
additional indirection through symbolic links, thereby keeping programs
that do symlink expansion happy.