Register subsystems and target types by backend/plugin. #12623

benjyw · 2021-08-22T05:15:31Z

We store, on BuildConfiguration, a map from each subsystem/
target type to the list of backends/plugins that can provide it.

A future change can use this to add this information to
help pages.

[ci skip-rust]

[ci skip-build-wheels]

We store, on BuildConfiguration, a map from each subsystem/ target type to the list of backends/plugins that can provide it. A future change can use this to add this information to help pages. [ci skip-rust] [ci skip-build-wheels]

benjyw · 2021-08-22T05:19:35Z

src/python/pants/build_graph/build_configuration.py



 @dataclass(frozen=True)
 class BuildConfiguration:
    """Stores the types and helper functions exposed to BUILD files."""

    registered_aliases: BuildFileAliases
-    subsystems: FrozenOrderedSet[Type[Subsystem]]
+    subsystem_to_providers: FrozenDict[Type[Subsystem], Tuple[str, ...]]


AFAICT the ordering of subsystems and target_types here was never important or significant, but please correct me if I'm wrong about that.

It's not clear to me that the rules/union_rules need to be ordered either. Or at least, the current order is registration order, which seems not significant and subject to change. If we need rules in a consistent order (perhaps to ensure consistent graph construction?) then maybe that should be alphabetical order of rule name or order of rule signature? Still, this is all future musing, and not for this change.

union_rules ordering matters currently: their registration order is preserved, and ends up being their order in the UnionMembership, which drives what order formatters are applied in, which could hypothetically matter. I think that's the only place where ordering might matter currently though.

rule ordering shouldn't matter as long as it is stable with a given set of backends (to avoid error messages changing their rendering, etc).

Where order matters, we might want to determine an order more meaningful than "registration order", which is fairly arbitrary. Nowhere as far as I can tell do we ensure that registration happens in a certain useful order.

# Rust tests and lints will be skipped. Delete if not intended. [ci skip-rust] # Building wheels and fs_util will be skipped. Delete if not intended. [ci skip-build-wheels]

stuhood

Thanks!

Eric-Arellano

Yay!

* Clean up the config parsing code. ([pantsbuild#12678](pantsbuild#12678)) * [internal] `--generate-lockfiles-resolve` does not determine `PythonLockfileRequest` for unspecified tools ([pantsbuild#12692](pantsbuild#12692)) * [internal] Add missing `resources` targets for default tool lockfiles ([pantsbuild#12670](pantsbuild#12670)) * [internal] Reorganize lockfile code to no longer be in `backend/python/experimental` ([pantsbuild#12669](pantsbuild#12669)) * [internal] Test that default tool lockfiles work on macOS ([pantsbuild#12666](pantsbuild#12666)) * [internal] Improve the error message for stale/invalid lockfiles ([pantsbuild#12618](pantsbuild#12618)) * [internal] Rename `./pants lock` to `./pants generate-user-lockfile` and `./pants tool-lock` to `./pants generate-lockfiles` ([pantsbuild#12641](pantsbuild#12641)) * [internal] Only run macOS tests in CI annotated with `@pytest.mark.platform_specific_behavior` ([pantsbuild#12665](pantsbuild#12665)) * [internal] Test that Python tools work with all expected interpreter versions ([pantsbuild#12656](pantsbuild#12656)) * [internal] Factor out a helper rule for setting up `--resolve-all-constraints` ([pantsbuild#12630](pantsbuild#12630)) * [internal] Add links to changelog and Twitter in Pants PyPI project page. ([pantsbuild#12636](pantsbuild#12636)) * [internal] Update CoC with proper markdown + https links. ([pantsbuild#12637](pantsbuild#12637)) * GitHub issue templates ([pantsbuild#12617](pantsbuild#12617)) * Register subsystems and target types by backend/plugin. ([pantsbuild#12623](pantsbuild#12623)) * [internal] Sort input requirements for `LockfileMetadata` ([pantsbuild#12615](pantsbuild#12615)) * [internal] Expect lockfile metadata to be defined ([pantsbuild#12616](pantsbuild#12616)) * Simplify subsystem registration. ([pantsbuild#12620](pantsbuild#12620)) * [internal] Use constants for magic strings `<none>` and `<default>` for tool lockfiles ([pantsbuild#12613](pantsbuild#12613)) * [internal] Refactor `lockfile_metadata.py` to be more class-based ([pantsbuild#12611](pantsbuild#12611)) * Embrace that help strings are markdown. ([pantsbuild#12606](pantsbuild#12606)) * Stop fetching a lockfile request just to figure out the requirements digest ([pantsbuild#12607](pantsbuild#12607)) * [internal] Refactor MyPy first party plugins ([pantsbuild#12599](pantsbuild#12599)) * [internal] Refactor MyPy config file setup ([pantsbuild#12593](pantsbuild#12593)) * [internal] use `is_union()` rather than `union.is_instance()`. ([pantsbuild#12595](pantsbuild#12595)) * fs_util can use mTLS and ignore certificates ([pantsbuild#12586](pantsbuild#12586)) * [internal] Fix Pylint lockfile to actually be wired up ([pantsbuild#12590](pantsbuild#12590)) * [internal] Update lockfile TODOs with current state of project ([pantsbuild#12592](pantsbuild#12592)) * [internal] Check for lockfile staleness by using context's interpreter constraints, rather than global constraints ([pantsbuild#12566](pantsbuild#12566)) * [internal] Refactor Pylint first-party plugins ([pantsbuild#12583](pantsbuild#12583)) * [internal] Fix Pytest and IPython to check for stale lockfiles ([pantsbuild#12582](pantsbuild#12582)) * [internal] Do not cache lockfile generation ([pantsbuild#12674](pantsbuild#12674))

* Clean up the config parsing code. ([#12678](#12678)) * [internal] `--generate-lockfiles-resolve` does not determine `PythonLockfileRequest` for unspecified tools ([#12692](#12692)) * [internal] Add missing `resources` targets for default tool lockfiles ([#12670](#12670)) * [internal] Reorganize lockfile code to no longer be in `backend/python/experimental` ([#12669](#12669)) * [internal] Test that default tool lockfiles work on macOS ([#12666](#12666)) * [internal] Improve the error message for stale/invalid lockfiles ([#12618](#12618)) * [internal] Rename `./pants lock` to `./pants generate-user-lockfile` and `./pants tool-lock` to `./pants generate-lockfiles` ([#12641](#12641)) * [internal] Only run macOS tests in CI annotated with `@pytest.mark.platform_specific_behavior` ([#12665](#12665)) * [internal] Test that Python tools work with all expected interpreter versions ([#12656](#12656)) * [internal] Factor out a helper rule for setting up `--resolve-all-constraints` ([#12630](#12630)) * [internal] Add links to changelog and Twitter in Pants PyPI project page. ([#12636](#12636)) * [internal] Update CoC with proper markdown + https links. ([#12637](#12637)) * GitHub issue templates ([#12617](#12617)) * Register subsystems and target types by backend/plugin. ([#12623](#12623)) * [internal] Sort input requirements for `LockfileMetadata` ([#12615](#12615)) * [internal] Expect lockfile metadata to be defined ([#12616](#12616)) * Simplify subsystem registration. ([#12620](#12620)) * [internal] Use constants for magic strings `<none>` and `<default>` for tool lockfiles ([#12613](#12613)) * [internal] Refactor `lockfile_metadata.py` to be more class-based ([#12611](#12611)) * Embrace that help strings are markdown. ([#12606](#12606)) * Stop fetching a lockfile request just to figure out the requirements digest ([#12607](#12607)) * [internal] Refactor MyPy first party plugins ([#12599](#12599)) * [internal] Refactor MyPy config file setup ([#12593](#12593)) * [internal] use `is_union()` rather than `union.is_instance()`. ([#12595](#12595)) * fs_util can use mTLS and ignore certificates ([#12586](#12586)) * [internal] Fix Pylint lockfile to actually be wired up ([#12590](#12590)) * [internal] Update lockfile TODOs with current state of project ([#12592](#12592)) * [internal] Check for lockfile staleness by using context's interpreter constraints, rather than global constraints ([#12566](#12566)) * [internal] Refactor Pylint first-party plugins ([#12583](#12583)) * [internal] Fix Pytest and IPython to check for stale lockfiles ([#12582](#12582)) * [internal] Do not cache lockfile generation ([#12674](#12674))

Register subsystems and target types by backend/plugin.

d587aed

We store, on BuildConfiguration, a map from each subsystem/ target type to the list of backends/plugins that can provide it. A future change can use this to add this information to help pages. [ci skip-rust] [ci skip-build-wheels]

benjyw requested review from Eric-Arellano and stuhood August 22, 2021 05:15

benjyw commented Aug 22, 2021

View reviewed changes

benjyw added 2 commits August 22, 2021 07:38

Fix test

f575874

# Rust tests and lints will be skipped. Delete if not intended. [ci skip-rust] # Building wheels and fs_util will be skipped. Delete if not intended. [ci skip-build-wheels]

Fix test again

b5dcbe2

# Rust tests and lints will be skipped. Delete if not intended. [ci skip-rust] # Building wheels and fs_util will be skipped. Delete if not intended. [ci skip-build-wheels]

benjyw mentioned this pull request Aug 22, 2021

Subsystems not labeled with their backend on the docsite #12523

Closed

stuhood approved these changes Aug 23, 2021

View reviewed changes

Eric-Arellano approved these changes Aug 23, 2021

View reviewed changes

benjyw merged commit 63ab1e8 into pantsbuild:main Aug 23, 2021

benjyw deleted the register_by_backend branch August 23, 2021 21:17

illicitonion mentioned this pull request Aug 30, 2021

Prep for 2.7.0rc0 #12695

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Register subsystems and target types by backend/plugin. #12623

Register subsystems and target types by backend/plugin. #12623

benjyw commented Aug 22, 2021

benjyw Aug 22, 2021

stuhood Aug 23, 2021 •

edited

Loading

benjyw Aug 23, 2021

stuhood left a comment

Eric-Arellano left a comment

Register subsystems and target types by backend/plugin. #12623

Register subsystems and target types by backend/plugin. #12623

Conversation

benjyw commented Aug 22, 2021

benjyw Aug 22, 2021

Choose a reason for hiding this comment

stuhood Aug 23, 2021 • edited Loading

Choose a reason for hiding this comment

benjyw Aug 23, 2021

Choose a reason for hiding this comment

stuhood left a comment

Choose a reason for hiding this comment

Eric-Arellano left a comment

Choose a reason for hiding this comment

stuhood Aug 23, 2021 •

edited

Loading