Add support for --override.
#2431
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This allows overriding transitive requirements in a resolve when the user knows a thrid party project's requirements are too restrictive. Also plumb the existing `--exclude` through to locks (along with `--override`) to allow sealing dependency manipulation configuration into the lock. Closes pex-tool#2425
jsirois
commented
Jul 7, 2024
tests/integration/test_overrides.py
Outdated
Comment on lines
66
to
117
| @skip_unless_compatible_with_requests_2_31_0 | ||
| def test_override( | ||
| tmpdir, # type: Any | ||
| complex_requests_override_pex, # type: str | ||
| ): | ||
| # type: (...) -> None | ||
|
|
||
| pex = os.path.join(str(tmpdir), "pex") | ||
| run_pex_command( | ||
| args=[ | ||
| "--python", | ||
| sys.executable, | ||
| "--python", | ||
| ensure_python_interpreter(PY310 if PY_VER < (3, 10) else PY39), | ||
| "requests==2.31.0", | ||
| "--override", | ||
| "urllib3==1.21; python_version >= '3.10'", | ||
| "--override", | ||
| "urllib3==1.20; python_version < '3.10'", | ||
| "--override", | ||
| "idna<2.5", | ||
| "-o", | ||
| pex, | ||
| ] | ||
| ).assert_success() | ||
|
|
||
| dists = [ | ||
| dist for dist in PEX(pex).resolve() if ProjectName("requests") == dist.metadata.project_name | ||
| ] | ||
| assert 1 == len(dists) | ||
| requests = dists[0] | ||
| request_deps_by_project_name = {dep.project_name: dep for dep in requests.requires()} | ||
| assert ( | ||
| Requirement.parse("urllib3<3,>=1.21.1") | ||
| == request_deps_by_project_name[ProjectName("urllib3")] | ||
| ), ( | ||
| "Our multiple override test requires requests normally have an urllib3 lower bound of " | ||
| "1.21.1." | ||
| ) | ||
| assert ( | ||
| Requirement.parse("idna<4,>=2.5") == request_deps_by_project_name[ProjectName("idna")] | ||
| ), "Our single override test requires requests normally have an idna lower bound of 2.5." | ||
|
|
||
| assert_overrides( | ||
| pex, | ||
| expected_overrides=[ | ||
| 'urllib3==1.21; python_version >= "3.10"', | ||
| 'urllib3==1.20; python_version < "3.10"', | ||
| "idna<2.5", | ||
| ], | ||
| expected_overridden_dists={"urllib3": ["1.20", "1.21"], "idna": ["2.4"]}, | ||
| ) |
There was a problem hiding this comment.
@cognifloyd you can do pretty crazy stuff now between --exclude and --override. Use the power wisely.
There was a problem hiding this comment.
Thank you! That turned out to be very elegant.
kaos
approved these changes
Jul 7, 2024
jsirois
commented
Nov 26, 2024
Comment on lines
+127
to
+159
| def register(parser): | ||
| # type: (_ActionsContainer) -> None | ||
| """Register dependency configuration options with the given parser. | ||
|
|
||
| :param parser: The parser to register dependency configuration options with. | ||
| """ | ||
| parser.add_argument( | ||
| "--exclude", | ||
| dest="excluded", | ||
| default=[], | ||
| type=str, | ||
| action="append", | ||
| help=( | ||
| "Specifies a requirement to exclude from the built PEX. Any distribution included in " | ||
| "the PEX's resolve that matches the requirement is excluded from the built PEX along " | ||
| "with all of its transitive dependencies that are not also required by other " | ||
| "non-excluded distributions. At runtime, the PEX will boot without checking the " | ||
| "excluded dependencies are available (say, via `--inherit-path`). This option can be " | ||
| "used multiple times." | ||
| ), | ||
| ) | ||
| parser.add_argument( | ||
| "--override", | ||
| dest="overridden", | ||
| default=[], | ||
| type=str, | ||
| action="append", | ||
| help=( | ||
| "Specifies a transitive requirement to override when resolving. Any distribution " | ||
| "requirement in the PEX's resolve that matches the override project name is replaced " | ||
| "with the given override requirement. This option can be used multiple times." | ||
| ), | ||
| ) |
There was a problem hiding this comment.
@benjyw I wanted to draw your attention to this change retroactively. The --exclude and --override options are very different from --constraints. With --constraints you can only narrow the requirements of some transitive dependency. With --exclude and --override you can actually edit the requirements of a transitive dependency.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This allows overriding transitive requirements in a resolve when the
user knows a thrid party project's requirements are too restrictive.
Also plumb the existing
--excludethrough to locks (along with--override) to allow sealing dependency manipulation configurationinto the lock.
Closes #2425