Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Eliminate warnings for default use. #2188

Merged
merged 5 commits into from
Jul 24, 2023
Merged

Conversation

jsirois
Copy link
Member

@jsirois jsirois commented Jul 23, 2023

Previously pex would warn for default usage scenarios under both
Python 3.11 and Python 3.12. For Python 3.12, the --pip-version
defaulted to 23.2 but a warning about vendored Pip 20.3.4 was issued
anyway; this is now fixed. For both Python 3.11 and Python 3.12, the
vendored version of packaging used the deprecated sre_constants
stdlib module; this is now fixed with packaging 23.1 being vendored
and selected for Python >= 3.7.

Fixes #2186

Previously `pex` would warn for default usage scenarios under both
Python 3.11 and Python 3.12. For Python 3.12, the `--pip-version`
defaulted to 23.2 but a warning about vendored Pip 20.3.4 was issued
anyway; this is now fixed. For both Python 3.11 and Python 3.12, the
vendored version of packaging used the deprecated `sre_constants`
stdlib module; this is now fixed with packaging 23.1 being vendored
and selected for Python >= 3.7.

Fixes pex-tool#2186
# N.B.: The pyparsing constraint is needed because our import re-writer (RedBaron) chokes on
# newer versions.
yield VendorSpec.pinned(
"packaging", "21.3", import_path="packaging_21_3", constraints=("pyparsing<3",)
)
if not filter_requires_python or sys.version_info[:2] >= (3, 7):
Copy link
Member Author

@jsirois jsirois Jul 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This 3rd vendored version of packaging increases Pex size ~100k from 4103623 bytes to 4205804 bytes.

@@ -886,8 +886,8 @@ def main(args=None):
except target_configuration.InterpreterConstraintsNotSatisfied as e:
die(str(e), exit_code=CANNOT_SETUP_INTERPRETER)

resolver_configuration = finalize_resolve_config(
resolver_configuration, targets, context="PEX building"
resolver_configuration = try_(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new test caught this old bug.

@@ -59,7 +59,7 @@ def build_pex_pex(output_file: PurePath, verbosity: int = 0) -> None:

def describe_git_rev() -> str:
git_describe = subprocess.run(
["git", "describe"], check=True, capture_output=True, encoding="utf-8"
["git", "describe"], check=True, stdout=subprocess.PIPE, encoding="utf-8"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing the packaging and pytest Python 3.6 edge case (tox -epy36-integration) caught this bug: capture_output was only added in 3.7. CI only runs unit tests for 3.5, ITs start at 3.7.

@jsirois
Copy link
Member Author

jsirois commented Jul 23, 2023

Hrm, so packaging 22.0+ drop support for LegacyVersion (an internal type really), but that means support for arbitrary equality === in version specifiers is also dropped. As such, this is a backwards incompatible change which Pex doesn't do. I'll back out this portion of the change for now while I think about a path forward for the packaging warnings.

... I guess I can just only use packaging 23.1 for Python>=3.12. The Python 3.11 folks have been suffering the warning already and the only folks not broken are 3.12 users mod a few hours.

Aha - ok, this is a test artifact. The === operator is still supported.

When the operator is `===`, we now perform the comparison on our own
since modern packaging errors. We also emit `===` in PEX-INFO when an
embedded distribution has a non PEP-440 compliant version.
@@ -21,11 +21,14 @@ def test_pip_2020_resolver_engaged():

# The Pip legacy resolver cannot solve the following requirements but the 2020 resolver can.
# Use this fact to prove we're plumbing Pip resolver version arguments correctly.
pex_args = ["boto3==1.15.6", "botocore>1.17<1.18.7", "--", "-c", "import boto3"]
pex_args = ["boto3==1.15.6", "botocore>1.17,<1.20", "--", "-c", "import boto3"]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modern packaging switched from pyparsing to a be-spoke parser and fixed the parsing bug that allowed botocore>1.17<1.18.7 previously (missing comma / AND). The <1.18.7 -> <1.20 is to force a naive conflict that pip-legacy-resolver can't handle - the actual point of the test. That version, 1.19.63, is now encoded in the test assertions.

@jsirois jsirois merged commit de45991 into pex-tool:main Jul 24, 2023
30 checks passed
@jsirois jsirois deleted the issues/2186 branch July 24, 2023 02:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Pex warns by default under Python 3.12
2 participants