Packages appear in export twice

[AndrewGuenther]

I've found that it's possible for packages can be listed in the export twice if they exist as a dependency with and without extras. Here's an example pyproject.toml:

[tool.poetry]
name = "pyjwt-test"
version = "0.1.0"
description = ""
authors = ["Andrew Guenther <guenther.andrew.j@gmail.com>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.8"
pyjwt = {extras = ["crypto"], version = "^2.1.0"}
djangorestframework-simplejwt = "^5.3.1"


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

djangorestframework-simplejwt also depends on pyjwt, but without extras. If you export the resulting lockfile, you'll see:

pyjwt==2.8.0 ; python_version >= "3.8" and python_version < "4.0" \
    --hash=sha256:57e28d156e3d5c10088e0c68abb90bfac3df82b40a71bd0daa20c65ccd5c23de \
    --hash=sha256:59127c392cc44c2da5bb3192169a91f429924e17aff6534d70fdc02ab3e04320
pyjwt[crypto]==2.8.0 ; python_version >= "3.8" and python_version < "4.0" \
    --hash=sha256:57e28d156e3d5c10088e0c68abb90bfac3df82b40a71bd0daa20c65ccd5c23de \
    --hash=sha256:59127c392cc44c2da5bb3192169a91f429924e17aff6534d70fdc02ab3e04320

In the output. It's the same package, listed twice, once with and once without extras. pip is smart enough not to barf when this happens, but other tools (in my case Bazel) are not as smart about it and will throw errors.

[dimbleby]

please search for duplicates next time

per #108, #125, #126 this is working as intended

[AndrewGuenther]

I did search, but the connection to those issues was not clear to me. Appreciate the links, I'll read up.

[AndrewGuenther]

Actually, reading those other issues, I'm not so sure we're describing exactly the same thing, so I want to make sure.

In this instance, the markers between the two deps are identical whereas the referenced issue seems to center around transitive dependencies which had different constraints being merged and constraints being lost. That's not the case here, these are completely safe to combine.

So is the argument that since it isn't safe to combine sometimes that export will simply never combine because pip appears to handle this correctly regardless? I still don't quite track after reading those issues how two dependencies with different constraints in the lock file is resolved by pip, presumably it combines those constraints internally and does the right thing?

@dimbleby would appreciate some insight if you know more about the internals there.

[dimbleby]

export will simply never combine because pip appears to handle this correctly regardless?

Yes

two dependencies with different constraints in the lock file is resolved by pip

Pip just evaluates exported requirements against the environment and either installs them or doesn't

If the constraints are contradictory then poetry would not have locked them that way in the first place