Clarifiaction on install from git+ssh URL

[yoanisgil]

Hi there,

I followed the instructions as described in the README.md and things work pretty well as long as I use poetry related commands to build/install my package. However I'm when I try this:

pip install --no-clean git+ssh://git@github.com/org/repo.git@v1.0.0-pre

the actual package version gets set to 0.0.0. I'm using pip==22.1.1 and python 3.8.12. I suspect it has something to do with that caveat you mention in the README that relates to PEP-517, but I was under the impression that this tools should work provided that a recent version of pip is used.

Any ideas, what I might be doing wrong?

[mtkennerly]

Could you share the pyproject.toml, or at least the [tool.poetry-dynamic-versioning] and [build-system] sections?

[yoanisgil]

Yes. Looks like this:

[build-system]
requires = ["poetry>=1.2.0b1", "poetry-dynamic-versioning-plugin"]
build-backend = "poetry.masonry.api"

[tool.poetry-dynamic-versioning]
enable = true
vcs = "git"
format = "v{base}"

I also came across these issues:

• Support finding project version from sources python-poetry/poetry#4971
• Make version in pyproject.toml optional, or at least validate it *after* the plugins are run python-poetry/poetry#4299

which seems to suggest that this plugin would not work for my use case. Apparently the developers from poetry seem to have a rather strong opinion on supporting this particular use case :(.

[mtkennerly]

Well, those tickets are just talking about whether Poetry should support it natively, but that's definitely a use case that this plugin tries to enable.

Looking at the code for poetry.masonry.api, it just re-exports some items from Poetry Core, which wouldn't have access to Poetry's PluginManager:

https://github.com/python-poetry/poetry/blob/f945937c3a01de380f70a3cbbf3bdbaa902a8c34/src/poetry/masonry/api.py

from poetry.core.masonry.api import build_sdist
from poetry.core.masonry.api import build_wheel
from poetry.core.masonry.api import get_requires_for_build_sdist
from poetry.core.masonry.api import get_requires_for_build_wheel
from poetry.core.masonry.api import prepare_metadata_for_build_wheel

I'm surprised it doesn't have some plugin-aware wrappers, and I'm not sure if that's intentional or just outstanding work. Could you try with poetry-core + poetry-dynamic-versioning and see if it works for you? I may need to remove/change the note about poetry + poetry-dynamic-versioning-plugin.

[build-system]
requires = ["poetry-core>=1.0.0", "poetry-dynamic-versioning"]
build-backend = "poetry.core.masonry.api"

[yoanisgil]

Let me give it a try and I'll let you know.

[yoanisgil]

Yup, that did the trick.

pip install --no-clean git+ssh://git@github.com/org/orchestra.git@v1.0.1-pre

which eventually leads to:

Building wheels for collected packages: orchestra
  Building wheel for orchestra (pyproject.toml) ... done
  Created wheel for orchestra: filename=orchestra-1.0.1-py3-none-any.whl size=17710 sha256=4ff2bc76a9b197929f59a6774ea04c94af37b6c2ad7c8efe9b5da9f74a13eaa2
  Stored in directory: /private/var/folders/8w/88wymn8j5fbfnrb9jmfdvj2w0000gn/T/pip-ephem-wheel-cache-v_9ykipe/wheels/f2/29/71/fed5b2d873c5aa9375c29b5f7d6408c3b103ad102e6e8133cc
Successfully built orchestra
Installing collected packages: orchestra
Successfully installed orchestra-1.0.1

and because this is a CLI tool (build on top of click), I can now see the version coming from the tag:

orchestra --version
orchestra, version 1.0.1

Pretty neat stuff! Out of curiosity, what was the issue here?

[yoanisgil]

I still want/need to do some more testing to ensure everything installs properly when starting from a clean environment but I think I'm good now. Big thanks!!!

[mtkennerly]

Great! I'm glad that helped.

Out of curiosity, what was the issue here?

There's some more info here: python-poetry/poetry#5701 . Basically, poetry.masonry.api bypasses the new Poetry 1.2 plugin system altogether, so poetry-dynamic-versioning-plugin isn't used in that context.

[Dom-TC]

I've just ran into the same issue. I've tried with poetry-dynamic-versioning and poetry-dynamic-versioning-plugin each with the following combinations of build settings:

[build-system]
requires = ["poetry>=1.0.0", "poetry-dynamic-versioning-plugin"]
build-backend = "poetry.masonry.api"

[build-system]
requires = ["poetry>=1.0.0", "poetry-dynamic-versioning"]
build-backend = "poetry.masonry.api"

[build-system]
requires = ["poetry-core>=1.0.0", "poetry-dynamic-versioning"]
build-backend = "poetry.core.masonry.api"

[build-system]
requires = ["poetry-core>=1.0.0", "poetry-dynamic-versioning-plugin"]
build-backend = "poetry.core.masonry.api"

and can't get versioning to be set correctly when installing via pip install git+ssh<...>.

I'm using Poetry version 1.2.0b3 and poetry-dynamic-versioning version 0.3.2.

My entire pyproject.toml is:

[tool.poetry]
name = "gitbackup"
version = "0.0.0"
description = ""
authors = ["<...>"]
include = ["log.yaml"]

[tool.poetry.dependencies]
python = "^3.10"
GitPython = "^3.1.27"
PyYAML = "^6.0"
poetry = "^1.1.14"

[tool.poetry.dev-dependencies]

[tool.poetry.scripts]
git-backup = 'gitbackup:main'

[tool.poetry-dynamic-versioning]
enable = true
pattern = "^(?P<base>\\d+(\\.\\d+)*)$"

[build-system]
requires = ["poetry-core>=1.0.0", "poetry-dynamic-versioning-plugin"]
build-backend = "poetry.core.masonry.api"

[tool.isort]
<...>

[tool.black]
<...>

Do you have any ideas as to what else to try?

[mtkennerly]

What version is it generating? Exactly 0.0.0 or something like 0.0.0.post10.dev0?

This one should work:

[build-system]
requires = ["poetry-core>=1.0.0", "poetry-dynamic-versioning"]
build-backend = "poetry.core.masonry.api"

What output do you get for the following commands?

pip install git+ssh://git@github.com/mtkennerly/playground.git
python -c "import dummy_python_project"

(future reference: https://github.com/mtkennerly/playground/tree/eb35d0da7978e806144e17ff60102a3332645a27)

For me, I get:

$ pip install git+ssh://git@github.com/mtkennerly/playground.git
[...]
Successfully installed dummy-python-project-0.1.0.post3.dev0+eb35d0d

$ python -c "import dummy_python_project"
loaded dummy project version 0.1.0.post3.dev0+eb35d0d

...where 0.1.0.post3.dev0 comes from the tag v0.1.0.

[Dom-TC]

What version is it generating? Exactly 0.0.0 or something like 0.0.0.post10.dev0?

Exactly 0.0.0

This one should work:

[build-system]
requires = ["poetry-core>=1.0.0", "poetry-dynamic-versioning"]
build-backend = "poetry.core.masonry.api"

I've just retried with these build settings and have the same issue. I get the following output, where gitbackup is the package in question:

$ pip install git+ssh://git@github.com/Dom-TC/git-backup-source.git#build-system
[...]
Successfully installed GitPython-3.1.27 PyYAML-6.0 gitbackup-0.0.0 gitdb-4.0.9 smmap-5.0.0

What output do you get for the following commands?

pip install git+ssh://git@github.com/mtkennerly/playground.git
python -c "import dummy_python_project"

The same as you:

$ pip install git+ssh://git@github.com/mtkennerly/playground.git
[...]
Installing collected packages: dummy-python-project
Successfully installed dummy-python-project-0.1.0.post3.dev0+eb35d0d
$ python -c "import dummy_python_project"
loaded dummy project version 0.1.0.post3.dev0+eb35d0d

[mtkennerly]

Well, that's bizarre. That narrows it down to just a difference between the two repositories, but I don't know what it could be. Is it a private repo? I'd like to try installing it on my end, if you wouldn't mind temporarily giving me view-only access.

[Dom-TC]

Of course. You should have an invite. There's a build-system branch which is what I've been playing around on as I was investigating the issue, but you can build off either branch.

[mtkennerly]

Ah, so that's what the #build-system part of your URL was. You just need to specify it as @build-system instead (source):

$ pip install git+ssh://git@github.com/Dom-TC/git-backup-source.git@build-system
Successfully installed gitbackup-1.2.1.post14.dev0+9b3de55

Pip was ignoring #build-system and defaulting to the main branch.

[Dom-TC]

🤦 That's entirely a typo I should have spotted! Thanks!!