venv created with the wrong python version

[asafalon1]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: macOS Big Sur (11.6)

• Poetry version: 1.1.11

Issue

I have both versions 3.7.2 and 3.8.0 of python installed on my mac.
My .toml states the python version should be 3.7:
...
[tool.poetry.dependencies]
python = "~3.7"
...

When i run "poetry update" (I use zsh terminal, don't know if it matters or not) a venv is created for my project (good), its name is myproj-DbOwq67j-py3.7 (also good), but the actual files created are python 3.8 (not good!). If you navigate to that folder you can clearly see the folder name is 3.7 but the file names are 3.8. Also I had issues installing packages that can only be installed on 3.7, so no doubt this is not just a naming issue.

This is bad as it is, because that venv is forever broken unless I patch the poetry code that creates venvs, but even after patching I get another issue:
Every time I close the terminal and reopen it, I run "poetry shell" to run commands for that venv, and i get this:

➜ myproj git:(master) ✗ poetry shell
The currently activated Python version 3.8.0 is not supported by the project (~3.7).
Trying to find and use a compatible version.
Using python3 (3.7.2)
The virtual environment found in /Library/Frameworks/Python.framework/Versions/3.8 seems to be broken.
Recreating virtualenv myproj-DbOwq67j-py3.7 in /Users/[my user]/Library/Caches/pypoetry/virtualenvs/myproj-DbOwq67j-py3.7
Spawning shell within /Users/[my user]/Library/Caches/pypoetry/virtualenvs/myproj-DbOwq67j-py3.7
➜ myproj git:(master) ✗ . /Users/[my user]/Library/Caches/pypoetry/virtualenvs/myproj-DbOwq67j-py3.7/bin/activate
(myproj-DbOwq67j-py3.7) ➜ myproj git:(master) ✗

Which means that the venv will be recreated once more with the wrong python version and override my patching. Also I need to reinstall all packages which in this case takes like 800s. Not fun.
By the way, the version in /Library/Frameworks/Python.framework/Versions/3.8 is definitely not broken, so i don't know what that is about.

[billmers]

I have the same issue on MacOS, Poetry 1.2.0a2, and pyenv 2.1.0.

[greatvovan]

I am experiencing the same issue.
Also, a workaround here.

pyenv: 2.2.0
poetry: 1.1.11
MacOS 11.6

Relevant Python versions are different from @asafalon1 , but the gist is basically the same: Python binaries of the created env. are of wrong version:

% poetry shell
The currently activated Python version 3.8.6 is not supported by the project (^3.10.0).
Trying to find and use a compatible version.
Using python3 (3.10.0)
Creating virtualenv my-project-ZyKUAaQE-py3.10 in /Users/me/Library/Caches/pypoetry/virtualenvs
Spawning shell within /Users/me/Library/Caches/pypoetry/virtualenvs/my-project-ZyKUAaQE-py3.10
% . /Users/me/Library/Caches/pypoetry/virtualenvs/my-project-ZyKUAaQE-py3.10/bin/activate
(my-project-ZyKUAaQE-py3.10) % python
Python 3.8.6 (default, Dec  8 2020, 13:48:43)
[Clang 12.0.0 (clang-1200.0.32.27)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

1. The "activated" version is determined incorrectly. I tried to set both pyenv global and pyenv local to another version, but it did not change anything, and "activated" version still was 3.8.6.
2. I made poetry env use 3.10 and it fixed the problem.

The concept of activated version is not quite clear, poetry must use the version specified in the .toml file for purposes of creating environment. What is activated version? I understand, it might make sense to select between variants of the same version (e.g. Python 3.9.7 from brew and Python 3.9.7 from pyenv), but why poetry may be using completeyl different version? Documentation was not very helpful for me.

[Kurt-von-Laven]

I recommend closing this issue as a duplicate of #1888 (with same workaround).

[greatvovan]

@Kurt-von-Laven this issue is different in the gist.

The problem #1888 you are referring to, is about creating a new environment, where the behaviour is not ideal as well and not perfectly documented (in terms of which version of Python to expect in pyproject.toml).

This issue however is about a different situation when pyproject.toml already exists and has a desired version of Python in it.

[Kurt-von-Laven]

I realize the scenarios aren't all exactly identical, but I believe in all cases a new virtual environment is being created, which is the salient point here. They do, after all, have precisely the same workaround that you and I arrived at independently. #651 and #4187 are related as well. In the former, the creator of Poetry explains that the wrong Python version being used in these virtual environments is an unfortunate side effect of an intentional change that they would like to keep for its maintenance and security benefits.

[asafalon1]

From what i'm reading the other issues describe other symptoms of the same change, but the bug described here is different, so connecting is them is fine, but I dont think they are all the same issue, or should be closed as one.

The workaround is nice, by the way, but its impractical on my end, I have over 30 repos i work on, with different python versions, ranging from 3.6 to 3.10.
It takes longer than doing what Im doing right now, which is create the venvs correctly once by patching the poetry code, and then disable poetry's ability to recreate my venvs. I wouldn't recommend this to anyone else though.

[castagninojose]

This has proven to be a complete nightmare, at least for me. I cannot, by any means, get poetry to stop using 3.8.10 for any new project I start. I maintain dozens of projects and was able to use a different version for every single one with poetry+pyenv for almost 3 years and experienced zero issues until now.-

• Uninstalled and re-installed poetry using both pipx (original way I installed it) and the new installer
• poetry env use (all py3.9 executables I could find), delete poetry.lock, environment and try installing again
• uninstalled any and all instances of python 3.8.10 I could find from my system (which is of course very bad)

pyenv versions
  system
  3.10.1
  3.8.5
* 3.9.9 (set by /my-repo/.python-version)

A plain python call uses the correct version

python
Python 3.9.9 (main, Feb 10 2022, 15:12:42) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 

In the toml project file I have python = "3.9.9" but when I try poetry install I get

Current Python version (3.8.10) is not allowed by the project (3.9.9).
Please change python executable via the "env use" command.

I have no idea what else to try.

[neersighted]

poetry config virtualenvs.prefer-active-python true -- this is documented as part of the 1.2 release, and while experimental, should provide the same experience as the old installer (which determined the active Python using a shebang).

[neersighted]

Closing this for now as it overlaps with multiple feature requests (automatic Pyenv version selection) as well as fixes that have gone in (respect the activated python3 in PATH).

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.