-
Notifications
You must be signed in to change notification settings - Fork 4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Find out how many users want lazy downloading of outputs, but can't use a daemon or FUSE / NFS #13669
Comments
FYI @brentleyjones as I noticed you commented on a related PR recently. |
We speculated on the need for (2) when BWtB was being conceived, but I have yet to come across a motivating use-case or user who would actually want it...most usage of BWtB I see where some outputs are desired is easily enough handled by using So my 2c is that (2) is reasonable in the abstract, but doesn't seem particularly important to users in practice (at least the ones I've talked to), and so doesn't seem terribly worth prioritizing. (1) would be much more valuable though, and I think get a lot of usage if it existed. I'd vote to focus on just that. |
This is probably out of scope but: @philwo is this hypothetical external process that handles presenting a fake This could compound if we could embed metadata into this content to generate a chain of trust. Like: this |
Prefer 1. lazy loading via a FUSE style daemon |
We use remote execution. Currently we hack |
BwoB is now enabled by default in Bazel@HEAD and we have flag |
Dear Bazel users,
I need to collect your feedback to judge which variants of lazy downloading of output files Bazel should implement.
One proposal is to implement lazy downloading of outputs via an external daemon process that would present the full output tree as a virtual filesystem that you can mount via FUSE or localhost loopback NFS. Basically, your
bazel-out
folder would not actually exist on disk, but instead be a FUSE / NFS mount that shows all files and then lazily download them as you access them.We're also considering extending Bazel's Builds-without-the-Bytes (
--remote_download_outputs=minimal
) feature with a command-line interface and API to support browsing and downloading of artifacts after a build.We currently don't know whether we should implement just the first or both of the proposals. We don't want to add unnecessary complexity to Bazel and its code, but we also want to offer lazy downloading to all users who need it.
Basically, I'm trying to find out how many users we have who
Thoughts? :)
The text was updated successfully, but these errors were encountered: