From 1f12af93871e9f1cc23ed0d1e2b48c90f4d28f2b Mon Sep 17 00:00:00 2001 From: Tony Allevato Date: Tue, 17 Jan 2023 11:16:43 -0800 Subject: [PATCH] Allow the Swift toolchain to be associated with an execution group and pass an optional execution group name down to actions that use it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This updates the following APIs: - `swift_common.get_toolchain` now takes an `exec_group` argument, which is will look up the toolchain under `ctx.exec_groups[NAME]` instead of `ctx`. - `swift_common.{compile,compile_module_interface,precompile_clang_module}` now take an `exec_group` argument, which is passed down to the underlying `ctx.actions.run` call for actions using the toolchain. In all cases the default value of the argument is `None`, so omitting it retains the current behavior. Most importantly, this lets us declare the Swift toolchain somewhere other than the rule/aspect's `ctx`, which ensures that in a multiple toolchain scenario, the Swift toolchain doesn't fully decide the execution platform for all actions (even non-Swift actions). Since execution groups are somewhat in flux right now, this may not be the final form—it would be nice to have the coupling between the execution group and the toolchain automated. I considered just having each action be its own execution group (like C++ link actions use `cpp_link`), but that seems worse; Swift actions should almost never be split across different toolchains/platforms, and it would force the user to have to declare multiple exec groups if they wanted to use the feature with multiple actions. This lets them share something like a single "swift" execution group across multiple actions, with the only caveat being that they have to pass that name explicitly. PiperOrigin-RevId: 502638394 (cherry picked from commit aeee2bb27d1a640705f155038cbdb8c50eeace9d) Signed-off-by: Brentley Jones --- doc/api.md | 31 +++++++++++++++++------------- swift/internal/actions.bzl | 5 +++++ swift/internal/compiling.bzl | 17 ++++++++++++++++ swift/internal/toolchain_utils.bzl | 16 ++++++++++++--- 4 files changed, 53 insertions(+), 16 deletions(-) diff --git a/doc/api.md b/doc/api.md index 6ca045dfa..0e4c29848 100644 --- a/doc/api.md +++ b/doc/api.md @@ -86,10 +86,11 @@ A new attribute dictionary that can be added to the attributes of a ## swift_common.compile 
-swift_common.compile(actions, additional_inputs, cc_infos, copts, defines, extra_swift_infos,
-                     feature_configuration, generated_header_name, is_test, include_dev_srch_paths,
-                     module_name, objc_infos, package_name, plugins, private_swift_infos, srcs,
-                     swift_infos, swift_toolchain, target_name, workspace_name)
+swift_common.compile(actions, additional_inputs, cc_infos, copts, defines, exec_group,
+                     extra_swift_infos, feature_configuration, generated_header_name, is_test,
+                     include_dev_srch_paths, module_name, objc_infos, package_name, plugins,
+                     private_swift_infos, srcs, swift_infos, swift_toolchain, target_name,
+                     workspace_name)
Compiles a Swift module. @@ -104,6 +105,7 @@ Compiles a Swift module. | cc_infos | A list of `CcInfo` providers that represent C/Objective-C requirements of the target being compiled, such as Swift-compatible preprocessor defines, header search paths, and so forth. These are typically retrieved from a target's dependencies. | none | | copts | A list of compiler flags that apply to the target being built. These flags, along with those from Bazel's Swift configuration fragment (i.e., `--swiftcopt` command line flags) are scanned to determine whether whole module optimization is being requested, which affects the nature of the output files. | `[]` | | defines | Symbols that should be defined by passing `-D` to the compiler. | `[]` | +| exec_group | Runs the Swift compilation action under the given execution group's context. If `None`, the default execution group is used. | `None` | | extra_swift_infos | Extra `SwiftInfo` providers that aren't contained by the `deps` of the target being compiled but are required for compilation. | `[]` | | feature_configuration | A feature configuration obtained from `swift_common.configure_features`. | none | | generated_header_name | The name of the Objective-C generated header that should be generated for this module. If omitted, no header will be generated. | `None` | @@ -161,9 +163,9 @@ A `struct` with the following fields: ## swift_common.compile_module_interface 
-swift_common.compile_module_interface(actions, compilation_contexts, feature_configuration,
-                                      module_name, swiftinterface_file, swift_infos, swift_toolchain,
-                                      target_name)
+swift_common.compile_module_interface(actions, compilation_contexts, exec_group,
+                                      feature_configuration, module_name, swiftinterface_file,
+                                      swift_infos, swift_toolchain, target_name)
Compiles a Swift module interface. @@ -175,6 +177,7 @@ Compiles a Swift module interface. | :------------- | :------------- | :------------- | | actions | The context's `actions` object. | none | | compilation_contexts | A list of `CcCompilationContext`s that represent C/Objective-C requirements of the target being compiled, such as Swift-compatible preprocessor defines, header search paths, and so forth. These are typically retrieved from the `CcInfo` providers of a target's dependencies. | none | +| exec_group | Runs the Swift compilation action under the given execution group's context. If `None`, the default execution group is used. | `None` | | feature_configuration | A feature configuration obtained from `swift_common.configure_features`. | none | | module_name | The name of the Swift module being compiled. This must be present and valid; use `swift_common.derive_module_name` to generate a default from the target's label if needed. | none | | swiftinterface_file | The Swift module interface file to compile. | none | @@ -578,7 +581,7 @@ Extracts the symbol graph from a Swift module. ## swift_common.get_toolchain 
-swift_common.get_toolchain(ctx, attr)
+swift_common.get_toolchain(ctx, exec_group, attr)
Gets the Swift toolchain associated with the rule or aspect. @@ -589,6 +592,7 @@ Gets the Swift toolchain associated with the rule or aspect. | Name | Description | Default Value | | :------------- | :------------- | :------------- | | ctx | The rule or aspect context. | none | +| exec_group | The name of the execution group that should contain the toolchain. If this is provided and the toolchain is not declared in that execution group, it will be looked up from `ctx` as a fallback instead. If this argument is `None` (the default), then the toolchain will only be looked up from `ctx.` | `None` | | attr | The name of the attribute on the calling rule or aspect that should be used to retrieve the toolchain if it is not provided by the `toolchains` argument of the rule/aspect. Note that this is only supported for legacy/migration purposes and will be removed once migration to toolchains is complete. | `"_toolchain"` | **RETURNS** @@ -680,9 +684,9 @@ A new attribute dictionary that can be added to the attributes of a ## swift_common.precompile_clang_module 
-swift_common.precompile_clang_module(actions, cc_compilation_context, feature_configuration,
-                                     module_map_file, module_name, swift_toolchain, target_name,
-                                     swift_infos)
+swift_common.precompile_clang_module(actions, cc_compilation_context, exec_group,
+                                     feature_configuration, module_map_file, module_name,
+                                     swift_toolchain, target_name, swift_infos)
Precompiles an explicit Clang module that is compatible with Swift. @@ -694,6 +698,7 @@ Precompiles an explicit Clang module that is compatible with Swift. | :------------- | :------------- | :------------- | | actions | The context's `actions` object. | none | | cc_compilation_context | A `CcCompilationContext` that contains headers and other information needed to compile this module. This compilation context should contain all headers required to compile the module, which includes the headers for the module itself *and* any others that must be present on the file system/in the sandbox for compilation to succeed. The latter typically refers to the set of headers of the direct dependencies of the module being compiled, which Clang needs to be physically present before it detects that they belong to one of the precompiled module dependencies. | none | +| exec_group | Runs the Swift compilation action under the given execution group's context. If `None`, the default execution group is used. | `None` | | feature_configuration | A feature configuration obtained from `swift_common.configure_features`. | none | | module_map_file | A textual module map file that defines the Clang module to be compiled. | none | | module_name | The name of the top-level module in the module map that will be compiled. | none | @@ -775,7 +780,7 @@ toolchains = swift_common.use_toolchain() + [other toolchains...] **RETURNS** -A list of toolchain types that should be passed to `rule()` or - `aspect()`. +A list of toolchain types that should be passed to `rule()`, `aspect()`, + or `exec_group`. diff --git a/swift/internal/actions.bzl b/swift/internal/actions.bzl index 302af77e0..8aed56ae2 100644 --- a/swift/internal/actions.bzl +++ b/swift/internal/actions.bzl @@ -120,8 +120,10 @@ def is_action_enabled(action_name, swift_toolchain): return bool(tool_config) def run_toolchain_action( + *, actions, action_name, + exec_group = None, feature_configuration, prerequisites, swift_toolchain, @@ -133,6 +135,8 @@ def run_toolchain_action( actions: The rule context's `Actions` object, which will be used to create `Args` objects. action_name: The name of the action that should be run. + exec_group: Runs the Swift compilation action under the given execution + group's context. If `None`, the default execution group is used. feature_configuration: A feature configuration obtained from `swift_common.configure_features`. mnemonic: The mnemonic to associate with the action. If not provided, @@ -206,6 +210,7 @@ def run_toolchain_action( actions.run( arguments = [tool_executable_args, args], env = tool_config.env, + exec_group = exec_group, executable = executable, execution_requirements = execution_requirements, inputs = depset( diff --git a/swift/internal/compiling.bzl b/swift/internal/compiling.bzl index dab4b870f..ba68f8f99 100644 --- a/swift/internal/compiling.bzl +++ b/swift/internal/compiling.bzl @@ -204,6 +204,7 @@ def compile_module_interface( *, actions, compilation_contexts, + exec_group = None, feature_configuration, module_name, swiftinterface_file, @@ -219,6 +220,8 @@ def compile_module_interface( Swift-compatible preprocessor defines, header search paths, and so forth. These are typically retrieved from the `CcInfo` providers of a target's dependencies. + exec_group: Runs the Swift compilation action under the given execution + group's context. If `None`, the default execution group is used. feature_configuration: A feature configuration obtained from `swift_common.configure_features`. module_name: The name of the Swift module being compiled. This must be @@ -327,6 +330,7 @@ def compile_module_interface( run_toolchain_action( actions = actions, action_name = SWIFT_ACTION_COMPILE_MODULE_INTERFACE, + exec_group = exec_group, feature_configuration = feature_configuration, outputs = [swiftmodule_file], prerequisites = prerequisites, @@ -360,6 +364,7 @@ def compile( cc_infos, copts = [], defines = [], + exec_group = None, extra_swift_infos = [], feature_configuration, generated_header_name = None, @@ -392,6 +397,8 @@ def compile( determine whether whole module optimization is being requested, which affects the nature of the output files. defines: Symbols that should be defined by passing `-D` to the compiler. + exec_group: Runs the Swift compilation action under the given execution + group's context. If `None`, the default execution group is used. extra_swift_infos: Extra `SwiftInfo` providers that aren't contained by the `deps` of the target being compiled but are required for compilation. @@ -708,6 +715,7 @@ to use swift_common.compile(include_dev_srch_paths = ...) instead.\ run_toolchain_action( actions = actions, action_name = SWIFT_ACTION_COMPILE, + exec_group = exec_group, feature_configuration = feature_configuration, outputs = all_compile_outputs, prerequisites = prerequisites, @@ -753,6 +761,7 @@ to use swift_common.compile(include_dev_srch_paths = ...) instead.\ precompiled_module = _precompile_clang_module( actions = actions, cc_compilation_context = compilation_context_to_compile, + exec_group = exec_group, feature_configuration = feature_configuration, is_swift_generated_header = True, module_map_file = compile_outputs.generated_module_map_file, @@ -836,6 +845,7 @@ def precompile_clang_module( *, actions, cc_compilation_context, + exec_group = None, feature_configuration, module_map_file, module_name, @@ -855,6 +865,8 @@ def precompile_clang_module( of headers of the direct dependencies of the module being compiled, which Clang needs to be physically present before it detects that they belong to one of the precompiled module dependencies. + exec_group: Runs the Swift compilation action under the given execution + group's context. If `None`, the default execution group is used. feature_configuration: A feature configuration obtained from `swift_common.configure_features`. module_map_file: A textual module map file that defines the Clang module @@ -875,6 +887,7 @@ def precompile_clang_module( return _precompile_clang_module( actions = actions, cc_compilation_context = cc_compilation_context, + exec_group = exec_group, feature_configuration = feature_configuration, is_swift_generated_header = False, module_map_file = module_map_file, @@ -888,6 +901,7 @@ def _precompile_clang_module( *, actions, cc_compilation_context, + exec_group = None, feature_configuration, is_swift_generated_header, module_map_file, @@ -908,6 +922,8 @@ def _precompile_clang_module( of headers of the direct dependencies of the module being compiled, which Clang needs to be physically present before it detects that they belong to one of the precompiled module dependencies. + exec_group: Runs the Swift compilation action under the given execution + group's context. If `None`, the default execution group is used. feature_configuration: A feature configuration obtained from `swift_common.configure_features`. is_swift_generated_header: If True, the action is compiling the @@ -989,6 +1005,7 @@ def _precompile_clang_module( run_toolchain_action( actions = actions, action_name = SWIFT_ACTION_PRECOMPILE_C_MODULE, + exec_group = exec_group, feature_configuration = feature_configuration, outputs = [precompiled_module], prerequisites = prerequisites, diff --git a/swift/internal/toolchain_utils.bzl b/swift/internal/toolchain_utils.bzl index cf24ba614..d2d830dbe 100644 --- a/swift/internal/toolchain_utils.bzl +++ b/swift/internal/toolchain_utils.bzl @@ -16,11 +16,16 @@ SWIFT_TOOLCHAIN_TYPE = "@build_bazel_rules_swift//toolchains:toolchain_type" -def get_swift_toolchain(ctx, attr = "_toolchain"): +def get_swift_toolchain(ctx, *, exec_group = None, attr = "_toolchain"): """Gets the Swift toolchain associated with the rule or aspect. Args: ctx: The rule or aspect context. + exec_group: The name of the execution group that should contain the + toolchain. If this is provided and the toolchain is not declared in + that execution group, it will be looked up from `ctx` as a fallback + instead. If this argument is `None` (the default), then the + toolchain will only be looked up from `ctx.` attr: The name of the attribute on the calling rule or aspect that should be used to retrieve the toolchain if it is not provided by the `toolchains` argument of the rule/aspect. Note that this is only @@ -30,6 +35,11 @@ def get_swift_toolchain(ctx, attr = "_toolchain"): Returns: A `SwiftToolchainInfo` provider. """ + if exec_group: + group = ctx.exec_groups[exec_group] + if group and SWIFT_TOOLCHAIN_TYPE in group.toolchains: + return group.toolchains[SWIFT_TOOLCHAIN_TYPE].swift_toolchain + if SWIFT_TOOLCHAIN_TYPE in ctx.toolchains: return ctx.toolchains[SWIFT_TOOLCHAIN_TYPE].swift_toolchain @@ -55,8 +65,8 @@ def use_swift_toolchain(): ``` Returns: - A list of toolchain types that should be passed to `rule()` or - `aspect()`. + A list of toolchain types that should be passed to `rule()`, `aspect()`, + or `exec_group`. """ # TODO(b/205018581): Intentionally empty for now so that rule definitions