experimental_strict_action_env doesn't do the right thing with remote builds

[EdSchouten]

BazelRuleClassProvider.java contains the following TODO:

// TODO(ulfjack): The default PATH should be set from the exec platform, which may be different
// from the local machine. For now, this can be overridden with --action_env=PATH=<value>, so
// at least there's a workaround.

This issue causes Bazel to set PATH to sane default for, say, Windows, even when making use of a Linux-based remote build cluster. Users must currently pass --action_env=PATH=/bin:/usr/bin in addition to --experimental_strict_action_env to make Bazel functional when doing builds on Linux from a Windows host.

In #6263, it was suggested that a separate bug for this is filed.

[buchgr]

@katre is this something where platforms can help? It seems that for cross compilation we'll need separate environment variables for host and execution platform?

[katre]

Yes, as @ulfjack said in the original TODO, eventually the PATH needs to be set from either the execution platform or the toolchain, so that it is correctly dependent on the actual execution system's properties.

I'm not sure anyone has had any good thoughts on how to accomplish this, however.

[ulfjack]

@EdSchouten is there anything specific you want to achieve here? At this time, Bazel cannot automatically detect the exec platform (it only talks to remote execution after creating the configuration). The best we can do, right now, is to have the user set a flag saying what the exec platform is, which is largely equivalent to setting --action_env=PATH=<value>.

[EdSchouten]

In our case we want to do builds of software on Linux workers from Windows hosts. We currently see that if we pass in --action_env=PATH=<value>, remote builds work. Unfortunately, actions run as part of repository rules (e.g., http_archive()'s patches attribute) fail. These now also use the UNIX specific --action_env as well.

One idea might be to split up --action_env into --local_action_env vs --remote_action_env?

[ulfjack]

I see. Well, it's definitely problematic to mix up these things, but the proposal doesn't sound quite right either - there is no such distinction between local and remote (there is no spoon!). Are you sure that params passed to --action_env are passed to repository execute calls? That sounds completely wrong.

[buchgr]

I'm not sure anyone has had any good thoughts on how to accomplish this, however.

@katre could you elaborate on this a bit please? Why is this hard to do?

[katre]

@buchgr The main steps that would be needed:

1. Figure out how to tie the PATH to a specific execution platform. New attribute on platform? Express this as part of the (incompletely implemented) shell toolchain? Express this as a new toolchain?
2. Update existing platforms with the new data.
3. Plumb through the data everywhere that a PATH is needed.

I'm not even sure what the set of places to change in the last step are, much less whether they each have access to an execution platform.

There's a lot of design work needed for this feature.

[philwo]

Assigning this to the Configurability team - this is nothing where Execution can help. (We just take whatever PATH the Spawn has in its environment and use that.)

[github-actions]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 3 years. It will be closed in the next 14 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

[brentleyjones]

I think this is still valid, right @katre @coeuvre @tjgq?

[katre]

It's still a valid problem, but we also still don't have a solution and no one is looking into it.

[brentleyjones]

@bazelbuild/triage not stale

[Silic0nS0ldier]

Toolchains seem quite close to what is needed here, and would offer greater extensibility vs setting this on execution platforms.

My thinking is...

The thinking is such a toolchain would be added automatically to exec groups + the default, actions will automatically pick this up accordingly.

• Automatic toolchain inclusion (rule) acts as a replacement for existing starlark API related to env vars.
• Automatic toolchain selection (action) keeps busy work to a minimum.
• Most existing API surfaces can be left as-is. (would need to define ordering for replacement)