Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only invalidate repositories when an environment variables changes that they read #20951

Open
lberki opened this issue Jan 19, 2024 · 2 comments
Labels
P2 We'll consider working on this in future. (Assignee optional) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: feature request

Comments

@lberki
Copy link
Contributor

lberki commented Jan 19, 2024

Description of the feature request:

Currently, external repositories are dirtied in Skyframe when any variable in --repo_env changes.

This means that there is a lot of unnecessary re-hashing to establish that the thus dirtied repositories are in fact up to date. This is because currently every repository depends on PRECOMPUTED:repo_env, which is just a bag of every such environment variable.

Ideally, repositories should only depend on the variables that they actually access.

There is some prior art in this area:

  • c230e39 adds the ability to gather the accessed environment variables (not strictly necessary, but is a good pleprint)
  • 80a4a14 makes repository rules not depend on PRECOMPUTED:repo_env if they don't read any environment variable. It's very crude, though.

Which category does this issue belong to?

No response

What underlying problem are you trying to solve with this feature?

No response

Which operating system are you running Bazel on?

No response

What is the output of bazel info release?

No response

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

@lberki lberki added type: feature request P2 We'll consider working on this in future. (Assignee optional) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. labels Jan 19, 2024
@555vedant
Copy link

Respected sir please can you assign this issuue to me ?

@lberki
Copy link
Contributor Author

lberki commented Feb 23, 2024

@555vedant are you volunteering to fix this issue? (Unless @Wyverald disagrees, I'd be delighted.)

nya3jp pushed a commit to nya3jp/cros-bazel that referenced this issue Apr 5, 2024
…g environment variables.

This should help us mitigate the effects of github.com/bazelbuild/bazel/issues/20951

BUG=None
TEST=portage/tools/run_tests.sh

Change-Id: I2e6c55e508b75945d0e5f6f1e4e912cc0a369b34
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/bazel/+/5420211
Auto-Submit: Matt Stark <msta@google.com>
Tested-by: Matt Stark <msta@google.com>
Reviewed-by: Shuhei Takahashi <nya@chromium.org>
Commit-Queue: Matt Stark <msta@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P2 We'll consider working on this in future. (Assignee optional) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: feature request
Projects
None yet
Development

No branches or pull requests

5 participants