Breaking change in handling of some version constraints from poetry-core 1.0.0 to 1.1.0

[gtrog]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Mac OS X 11.6 (BigSur)
• Poetry version: 1.1.14 -> 1.2.0
• Link of a Gist with the contents of your pyproject.toml file:

Issue

I ran into an interesting bug (I'm not sure if it's the root cause of some other issues, I had trouble finding a matching issue) where when poetry was using poetry-core==1.0.0 it was correctly resolving the version constraint of a somewhat non-standard semver version, but once poetry started using version poetry-core==1.1.0, it's no longer correctly resolving the same version constraint, resulting in dependency resolution issues because the resolved constraint is never satisfiable.

For more context, in my current project, I'm pulling in a library:

opentelemetry-instrumentation-fastapi = "^0.32b0"

Where the next available version is 0.33b0.

In a version of poetry that uses poetry-core==1.0.0 , it correctly resolves this constraint:

# pip install poetry-core==1.0.0
# python <<EOF
from poetry.core.semver import parse_constraint
print(parse_constraint("^0.32b0"))
EOF

This will output >=0.32b0,<0.33, which is what ultimately ends up in setup.py when publishing with poetry.

Now, using poetry-core==1.1.0:

# pip install poetry-core==1.1.0
# python <<EOF
from poetry.core.semver.version import Version 
print(Version.parse("0.32b0").next_breaking())
EOF

This will output 0.32, which ultimately will get translated to a constraint that's unsolvable >=0.32b0,<0.32 in setup.py:

install_requires = \
['fastapi-versioning>=0.10.0,<0.11.0',
 'fastapi>=0.79.0,<0.80.0',
 'gprof2dot>=2022.7.29,<2023.0.0',
 'opentelemetry-instrumentation-fastapi>=0.32b0,<0.32',  # <-- here
 'opentelemetry-instrumentation-logging>=0.32b0,<0.32',  # <-- and here
 'requests>=2.27.1,<3.0.0',
 'splunk-opentelemetry[all]>=1.7.0,<2.0.0',
 'starlette-exporter>=0.14.0,<0.15.0',
 'uvicorn[standard]>=0.18.2,<0.19.0']

Ultimately, after upgrading to poetry version 1.2.0 and publishing my package, it could no longer be installed because this version constraint could no longer be satisfied, resulting in a breaking change. I've worked around this for now by specifying an explicit version constraint, but, wanted to see if this is something that anyone else has noticed, if this is a "won't fix" type situation, since the version is somewhat non-standard, etc.

[dimbleby]

feels like a bug in Version.next_breaking() to me.

I guess it's correct that "0.32b0".next_minor() is "0.32", but next_breaking() should give "0.33"

If you care enough to put together a fix: suggest that next_breaking() wants to start by going stable = self.stable and then consider stable everywhere it currently considers self.

And add a testcase in here - next_breaking() has no coverage at the moment so the more the merrier, cf the tests for next_major() and next_minor() and suchlike.

[dimbleby]

though fwiw >=0.32b0,<0.32 is not unsolvable: it allows 0.32b0 (and 0.32b1, etc)

[mazinesy]

👋 Just adding on top of this, >0.0.2-alpha.21 yields >0.0.2-alpha.21,<0.0.2 on 1.2.0 but works fine on 1.1.15

[dimbleby]

should be straightforward to fix and test, see my comment a couple up if this is important to you

[mazinesy]

@dimbleby Opened python-poetry/poetry-core#475

[mazinesy]

@dimbleby Looking at the PR, is that really the problem ? The issue I had was:

We have a package with sqlalchemy2-stubs=^0.0.2-alpha.21 and when building the wheel, it resolved into the following constraint:

# Poetry 1.2.0
>=0.0.2-alpha.21,<0.0.2

which seems fine since 0.0.2-alpha.21 < 0.0.2. But poetry 1.2.0 was unable to resolve it.

On poetry 1.1.15, building the wheel resulted in the following constraint:

# Poetry 1.1.15
>=0.02-alpha.21,<0.0.3

I believe that the correct upper limit after a prerelease is the release itself which is 0.0.2. This would imply that the bug is not in this next_breaking() but rather somewhere where the constraints is actually computed. 🤔

[dimbleby]

I just tried it and poetry successfully resolves sqlalchemy2-stubs = "^0.0.2-alpha.21" to sqlalchemy2_stubs-0.0.2a27.

Please provide a repro of whatever your problem is!

[dimbleby]

Are you reporting that when you try to pip install the wheel after poetry build, pip does not agree with poetry about version ordering?

$ pip install dist/foo-0.1.0-py3-none-any.whl
Processing ./dist/foo-0.1.0-py3-none-any.whl
ERROR: Could not find a version that satisfies the requirement sqlalchemy2-stubs<0.0.2,>=0.0.2-alpha.21 (from foo) (from versions: 0.0.1a1, 0.0.1a2, 0.0.1a3, 0.0.1a4, 0.0.1a5, 0.0.1a6, 0.0.1a7, 0.0.1a8, 0.0.1a9, 0.0.1a10, 0.0.1a11, 0.0.2a1, 0.0.2a2, 0.0.2a3, 0.0.2a4, 0.0.2a5, 0.0.2a6, 0.0.2a7, 0.0.2a8, 0.0.2a9, 0.0.2a10, 0.0.2a11, 0.0.2a12, 0.0.2a13, 0.0.2a14, 0.0.2a15, 0.0.2a16, 0.0.2a17, 0.0.2a18, 0.0.2a19, 0.0.2a20, 0.0.2a21, 0.0.2a22, 0.0.2a23, 0.0.2a24, 0.0.2a25, 0.0.2a26, 0.0.2a27)
ERROR: No matching distribution found for sqlalchemy2-stubs<0.0.2,>=0.0.2-alpha.21

it seems to me that pip is wrong and poetry is right here:

• 0.0.2-alpha.21 should normalize to 0.0.2a21
• then that requirement should be satisfied by 0.0.2a21, 0.0.2a22, etc

so if that is what you are talking about, I'd suggest taking it to the pip issue tracker for an opinion

[dimbleby]

although I could imagine that poetry was supposed to normalize the version before writing it

edit: that doesn't help, you still get

ERROR: No matching distribution found for sqlalchemy2-stubs<0.0.2,>=0.0.2a21

so this really does want taking to pip IMO

[dimbleby]

hmm, beginning to change my mind...

PEP440 says "The exclusive ordered comparison <V MUST NOT allow a pre-release of the specified version unless the specified version is itself a pre-release"

So "<0.0.2" does indeed exclude all those prereleases...

possibly the bottom line here is that no-one really knows what "^0.0.2a21" is supposed to mean, the caret notation is a thing that poetry introduces and so this is a problem of its own making.

python-poetry/poetry-core#475 seems like the right answer

[mazinesy]

Alright! But if you want to replicate the issue, here's where to start:

1. Build a wheel of a package that depends on sqlalchemy2-stubs^0.0.2-alpha.21 like this:

mkdir mypkg
cd mypkg
poetry init  # go through all with no dependencies
# edit pyproject.toml to manually add sqlalchemy2-stubs="^0.0.2-alpha.21" in the dependencies

poetry lock --no-update
# create the empty package
mkdir mypkg
touch mypkg/__init__.py

poetry build  # generates the wheel
poetry run pip install pkginfo
poetry run pkginfo dist/path-to.whl
# here you'll see that the sqlalchemy2-stubs has this constraint: >=0.0.2-alpha.21,<0.0.2

From that state, you need another poetry project that depends on mypkg. This yields an error that basically says:

unable to resolve sqlalchemy2-stubs <empty>

[dimbleby]

yep, I think we're all on the same page now.

If you can get python-poetry/poetry-core#475 over the line everything should be good

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.