Add optional tool_path_origin to Tools in C++ crosstool

[Yannic]

This change adds an optional field to the C++ crosstool proto that
allows configuring the origin tool_path is relative to.

For now, the origin can be one of the following:

• CROSSTOOL_PACKAGE: If tool_path is a relative path, it's assumed to
  be relative to the package of the crosstool top.
• FILESYSTEM_ROOT: tool_path is an absolute path. This is checked by
  Bazel.
• WORKSPACE_ROOT: tool_path must be a relative path and is assumed to
  be relative to the exec-root of the workspace (similar to other
  executables).

Updates #8438

[lberki]

@oquenchil , can you take a look?

[oquenchil]

Hi Yannic, this #7746 (comment) from scentini and a few comments above from hlopko is the last context that I can find about how we should go about this problem and how to allow tools from other repositories.

I don't exactly see how your change is going in that direction but perhaps it's because this is all kind of new to me and I might be missing something.

Could you please clarify?

[Riatre]

Not original author but this bite me hard so I'll try to explain:

Currently, tool_path must either be an absolute path, or a path relative to the package defining cc_toolchain_config, and it must be canonical (cannot contains ../), thus it is impossible to reference files from other repositories.

For example, say, you have a cc_toolchain_config rule //tools/toolchain:config, you want to use a gcc binary from @gcc_archive//:bin/gcc, currently it is not possible because @gcc_archive//:bin/gcc would be located at <execroot>/external/gcc_archive/bin/gcc, and there is no canonical relative path from <execroot>/tools/toolchain to <execroot>/external/gcc_archive_bin/gcc.

People currently solve this with one-liner wrapper scripts located under //tools/toolchain, it exploits the fact that when we execute tools, $PWD is set to exec-root, so we can exec external/gcc_archive/bin/gcc "$@" at that time. Even official tutorial leverages this pattern, though it has other legitimate reasons to wrap the compiler. But this is less than ideal for at least two reasons:

1. User must construct the exec-path manually.
2. It is not obvious how I can refer to an artifact instead of raw file.
3. As per toolchain tool_paths should accept labels #8438 (comment), this hack is not compatible with rules_foreign_cc. (I haven't verified this though)

From what I can see, the patch adds a new attribute tool to toolFromSkylark as scentini suggested, other changes (tool_path_origin etc.) are required because Bazel enforces that tool_path must be an relative path without ../.

Edit: further clarify.

[Yannic]

The use-case I had in mind is primarily to remove the need for the checked-in wrapper-scripts @Riatre mentioned. It's also a step in the right direction for tools from external repositories (although, AFAICT, a bit more work is needed there).

The way CC toolchains/crosstools currently work is that, for relative paths, tool.path is expected to be relative to the cc_toolchain (cc_toolchain_suite?) target, which is what's saved in the proto crosstool. This toolchain-relative path is later converted to an exec-path [1] so Bazel can actually run the compile/link/... action. This change allows doing this exec-path convertion earlier (i.e. when generating the crosstool proto), which removes the need for all of these toolchain-relative wrapper scripts.

It's still neccessary to provide all tool artifacts a second time to cc_toolchain.*_files (although that is probably not to hard to fix in a follow-up).

Usage example:

load("@rules_cc//cc:cc_toolchain_config_lib.bzl", "tool")

def _my_toolchain_impl(ctx):
    # ...

    # Use system compiler
    compiler = tool(path = "/bin/gcc")

    # Use custom linker (new)
    linker = tool(tool = ctx.executable.ld)

    # ...

[1] https://source.bazel.build/bazel/+/master:src/main/java/com/google/devtools/build/lib/rules/cpp/CcToolchainFeatures.java;l=950;drc=f5262c54f0609a7c40a57d7dc70e298068b06b0e

[oquenchil]

LGTM