toolchain tool_paths should accept labels

[guibou]

Description of the problem / feature request:

toolchain confiuration cc_common.create_cc_toolchain_config_info attributes tool_paths/path should accept bazel labels.

Feature requests: what underlying problem are you trying to solve with this feature?

The current status (in bazel 0.25) is that toolchain configuration info attributes for tool paths only accept local or absolute paths, but not bazel label.

That's not convenient if your tools comes from another (possibly repository) rule because you need to generate the path by yourself (I don't even know how bazel correctly knows it had to build the dependent rule before).

Example of what I'd like to do:

cc_common.create_cc_toolchain_config_info(
     ...
        tool_paths = [
            tool_path(
                name = "gcc",
                path = "@gcc//:bin/gcc",
...

What operating system are you running Bazel on?

Linux

What's the output of bazel info release?

release bazel-0.25-2

[katre]

This is not a configuration issue, it is part of cc toolchain definition.

[guibou]

Note: It should also provide a way to reference directories (for include paths for example).

[hlopko]

CC @scentini

[guibou]

May I add to this issue that a lot of rule have string argument in place which may be label. For example, I'm currently limited by http_archive patch_tool attribute which is a string and not a label, meaning that I cannot build my local patch tool and depend on it.

[katre]

You should file that as a separate issue, so that it can be separately triaged and tracked.

[guibou]

@katre done. Considering that I've opened this issue 6 months and that nobody seems interested to at least discuss it, I'm wondering if I may not be doing something totally wrong.

If we stay on the topic of setting create_cc_toolchain_config_info with path for the compiler, how do people write bazel toolchain using compilers which are them self provided by bazel?

Right now I'm cheating this way:

• My "external repository rules" for compilers are actually generating output in a directory that I manage, outside of bazel. They are also generating BUILD files with variables containing theses paths.
• I can then load theses BUILD files and use the variables as argument for my toolchain path.

This is painful because I need to ensure that the first step is reproducible by myself, a job I'd like bazel to do for me.

[katre]

I'm sorry this got missed and wasn't handled. I actually don't know the details of how to specify cc toolchains, so I've re-marked this as "untriaged" to get the attention of the CPP rules team. Sorry this has been frustrating.

You might consider asking on bazel-discuss@ how other users have handled this problem.

[emusand]

@guibou Did you find out how other users handle this problem?

We have the same problem. For one of our toolchains, we want to download the SDK and use the compiler and linker paths in the toolchain definition. We know the labels to these tools, but how do we convert them to paths?

[elklein]

This seems at least somewhat related to #7746. The source code says that tool_path is deprecated, and instead action_config / tool should be used. tool() of course has the same issue, but it's quite a bit easier to modify the tool/ToolInfo and the underlying java code to accept a File/Artifact object. I played around with this a little the last day or so and I have something working that works for me to accept labels in action_config/tool settings. It needs some work to play nicely with the protobuf serialization/deserialization, but it's something at least. Hoping to get some guidance in #7746 and then I'll submit a pull request.

[Riatre]

There are two possible partial workarounds to @emusand 's problem as of now. They only work for files in the repo so they are not solutions for the original issue.

1. The official cc-toolchain-config tutorial uses a shell script to workaround this. tl;dr point your tool_path to a shell script in config repository, the shell script will be invoked with CWD set to execroot, you can then access files in external repo via external/<repo_name>/....
2. Allow tool_path to be unnormalized. #5809 makes it possible to use relative paths in tool_path. With that patch it is possible to generate a relative path in Starlark as tool_path:

def _fspath_split(path):
    return [segment for segment in path.split("/") if segment]

def _fspath_relative(src_dir, dst):
    src = _fspath_split(src_dir)
    dst = _fspath_split(dst)
    lcp = min(len(src), len(dst))
    for i in range(lcp):
        if src[i] != dst[i]:
            lcp = i
            break
    return "/".join([".."] * (len(src) - lcp) + dst)

def resolve_label_like_tool_path(ctx, label_string):
    if not (label_string.startswith('@') or label_string.startswith('//')):
        return label_string
    tool_label = Label(label_string)
    rule_label = ctx.label

    # Paths relative to execroot
    tool_execroot_path = tool_label.workspace_root + "/" + tool_label.package + "/" + tool_label.name
    rule_execroot_path = rule_label.workspace_root + "/" + rule_label.package
    return _fspath_relative(rule_execroot_path, tool_execroot_path)

[otisonoza]

I believe the first workaround breaks the configure_make rule: https://github.com/bazelbuild/rules_foreign_cc/blob/master/tools/build_defs/configure.bzl

configure is not invoked from execroot, as I see it's invoked from a temp directory under /tmp/.
Then when it invokes the wrapper script from the tool_path, it will not be able to find the executable (for example gcc) when checking if the compiler works.

[guibou]

@emusand there is a mapping between label and path for external rules. For example, if your gcc is in @gcc//:bin/gcc, you will be able to find it under the path external/gcc/bin/gcc. That's the solution I'm using since 2 years.

[guibou]

@emusand and when it does not work because your package is not loaded in the sandbox, I'm writing the absolute path of my binary. I wrote a few automated process which are generating on the fly paths.bzl files which contains variables with the absolute paths.

[jheaff1]

I believe the first workaround breaks the configure_make rule: https://github.com/bazelbuild/rules_foreign_cc/blob/master/tools/build_defs/configure.bzl

configure is not invoked from execroot, as I see it's invoked from a temp directory under /tmp/.
Then when it invokes the wrapper script from the tool_path, it will not be able to find the executable (for example gcc) when checking if the compiler works.

Is there a resolution for the "first workaround", i.e using wrappers around the tools.
I have defined a custom cc toolchain whereby the tools are downloaded from https://musl.cc. I'm using the wrapper workaround and the Bazel documentation states that create_cc_toolchain_config_info requires tool_paths. These tool_paths must be of type ToolPathInfo and must be normalized (i.e. contain no ../ characters) otherwise an error occurs.

Indeed, when building zlib using cmake with rules_foreign_cc and my musl gcc toolchain, the following error occurs:

external/gcc_x86_64_linux_musl/bin/gcc: No such file or directory

I can successfully build simple cc_library and cc_binary targets using my musl gcc toolchain, i.e. without rules_foreign_cc involved

[nishanthkarthik]

I hit the same error while building with rules_foreign_cc and my custom gcc toolchain. This example from https://github.com/grailbio/bazel-toolchain might help you.

https://github.com/grailbio/bazel-toolchain/blob/32d7596dd99c608015ba9a20c24b4fd5d03fdad2/toolchain/cc_wrapper.sh.tpl#L36

[oquenchil]

tool_path is deprecated in favor of action_configs. The problem described in this issue can be solved with action_configs already without any additional Bazel changes. Let's say you had:

def _impl(ctx):
    tool_paths = [
        tool_path(
            name = "gcc",
            path = "clang42", # binary living directly under your workspace root
        ),
        tool_path(
            name = "ld",
            path = "clang42",
        ),
    ]
    ....
    return cc_common.create_cc_toolchain_config_info(
        ctx = ctx,
        tool_paths = tool_paths,
        ...
    )

cc_toolchain_config = rule(
    implementation = _impl,
    attrs = {},
    provides = [CcToolchainConfigInfo],
)

That can be converted to:

def _impl(ctx):
    compile_action = action_config(
        action_name = ACTION_NAMES.cpp_compile,
        tools = [
            struct(
                type_name = "tool",
                tool = ctx.file.clang,
            ),
        ],
    )
    link_action = action_config(
        action_name = ACTION_NAMES.cpp_link_executable,
        tools = [
            struct(
                type_name = "tool",
                tool = ctx.file.clang,
            ),
        ],
    )
    return cc_common.create_cc_toolchain_config_info(
        ctx = ctx,
        action_configs = [compile_action, link_action],
        ...
    )

cc_toolchain_config = rule(
    implementation = _impl,
    attrs = {
        "clang": attr.label(default=":clang42", allow_single_file=True),
    },
    provides = [CcToolchainConfigInfo],
)

Please let me know if there is anything about that will make it not work for your use case.

[bjacklyn]

@oquenchil Also running into this issue trying to use an executable from a different external repository than the one my toolchain is configured in. And trying to follow your advice about using action_configs instead of tool_paths:

I have a custom rule which uses the CcToolchainInfo to find various executables in the toolchain. But it seems like the only way to populate these values is by using tool_path()? I'm looking at the bottom of src/main/starlark/builtins_bzl/common/cc/cc_toolchain_provider_helper.bzl and for example objcopy, it ends up being hard-coded to external/my_toolchain/objcopy when tool_paths is not provided.

Is there a way to access the action_configs of a toolchain from a custom rule / is there another way to access the tools in a toolchain?

Example custom rule:

load("@bazel_tools//tools/cpp:toolchain_utils.bzl", "find_cpp_toolchain")

def _my_custom_rule_impl(ctx):
    cc_toolchain = find_cpp_toolchain(ctx)
    cc_objcopy_executable = cc_toolchain.objcopy_executable    <-- has wrong value
    // ctx.actions.run(  ...do something with objcopy_executable...  )

my_custom_rule = rule(
    implementation = _my_custom_rule_impl,
    fragments = ["cpp"],
    toolchains = ["@bazel_tools//tools/cpp:toolchain_type"],
),

[oquenchil]

Not sure I understand the problem completely. Have you seen get_tool_for_action? If you specify the right action config in the toolchain that method should give you the correct path.

[bjacklyn]

Thanks @oquenchil, get_tool_for_action works. Unfortunately OBJ_COPY_ACTION_NAME is not present in action_names.bzl in bazel 6.3.0 (or the upcoming 6.4.0), but I did see it work in 7.

[oquenchil]

Do you have a custom toolchain? The presence of the constant in action_names.bzl shouldn't matter since you can just use the string directly but the action_config needs to be configured in the toolchain so if you are using the default toolchain you are out of luck because it was introduced after the 6.0 cut, see here.

If this is important to you, you can ask for that commit to be cherrypicked into 6.4.0. It won't break anything.

[bjacklyn]

Oh interesting, I do have a custom toolchain. I ended up creating wrapper scripts, but I plan to revisit this when bazel 7 is released.

JFYI the docs indicate that the string has to be in the bzl file, but sounds like this doesn't apply to custom toolchains then:

[oquenchil]

That's correct, I'd expect this to work as long as the action_config is registered by your custom toolchain. When you revisit this, please give it a try and report back if it doesn't.