Fix support for folders in python.pythonPath setting

[meakbiyik]

Environment data

• VS Code version: 1.54.3
• Extension version (available under the Extensions sidebar): v2021.3.680753044
• OS and version: Windows 10.0.19041
• Python version (& distribution if applicable, e.g. Anaconda): 3.8.6
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): venv (via poetry)
• Value of the python.languageServer setting: Pylance

Expected behaviour

For "python.pythonPath": "${workspaceFolder}/.venv" workspace setting, extension is expected to find the virtualenv in the given directory.

Actual behaviour

The extension is unable to find the virtual env with the latest update (rolling back to the previous version works fine).

Steps to reproduce:

1. Create a virtualenv in the workspace possibly using poetry, and point to the workspace with ${workspaceFolder}/{workspace_name}

IMPORTANT NOTE: ".venv\\Scripts\\python.exe" works fine, but "${workspaceFolder}\\.venv", "${workspaceFolder}/.venv" does not. All of them works in the previous version.

Possibly related to #15781 but I do not have the error described in that issue in the output below.

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

User belongs to experiment group 'pythonaa'
User belongs to experiment group 'pythonJoinMailingListVar2'
User belongs to experiment group 'pythonJediLSP'
User belongs to experiment group 'pythonSendEntireLineToREPL'
User belongs to experiment group 'pythonNotDisplayLinterPrompt'
User belongs to experiment group 'pythonDiscoveryModuleWithoutWatcher'
User belongs to experiment group 'pythonFindInterpreter'
User belongs to experiment group 'ShowExtensionSurveyPrompt - enabled'
User belongs to experiment group 'CollectLSRequestTiming - control'
> c:\Users\erena\OneDrive\Desktop\SNLab\soccerbars/.venv c:\Users\erena\.vscode\extensions\ms-python.python-2021.3.680753044\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> c:\Users\erena\OneDrive\Desktop\SNLab\soccerbars/.venv c:\Users\erena\.vscode\extensions\ms-python.python-2021.3.680753044\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> c:\Users\erena\OneDrive\Desktop\SNLab\soccerbars/.venv c:\Users\erena\.vscode\extensions\ms-python.python-2021.3.680753044\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> c:\Users\erena\OneDrive\Desktop\SNLab\soccerbars/.venv c:\Users\erena\.vscode\extensions\ms-python.python-2021.3.680753044\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
Starting Pylance language server.
> c:\Users\erena\OneDrive\Desktop\SNLab\soccerbars/.venv c:\Users\erena\.vscode\extensions\ms-python.python-2021.3.680753044\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> c:\Users\erena\OneDrive\Desktop\SNLab\soccerbars/.venv c:\Users\erena\.vscode\extensions\ms-python.python-2021.3.680753044\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> c:\Users\erena\OneDrive\Desktop\SNLab\soccerbars/.venv c:\Users\erena\.vscode\extensions\ms-python.python-2021.3.680753044\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> c:\Users\erena\OneDrive\Desktop\SNLab\soccerbars/.venv c:\Users\erena\.vscode\extensions\ms-python.python-2021.3.680753044\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> c:\Users\erena\OneDrive\Desktop\SNLab\soccerbars/.venv c:\Users\erena\.vscode\extensions\ms-python.python-2021.3.680753044\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> c:\Users\erena\OneDrive\Desktop\SNLab\soccerbars/.venv c:\Users\erena\.vscode\extensions\ms-python.python-2021.3.680753044\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
Error 2021-03-25 12:24:27: Diagnostic Code: InvalidPythonInterpreterDiagnostic, Message: No Python interpreter is selected. You need to select a Python interpreter to enable features such as IntelliSense, linting, and debugging.
Diagnostic Code: InvalidPythonInterpreterDiagnostic, Message: No Python interpreter is selected. You need to select a Python interpreter to enable features such as IntelliSense, linting, and debugging.

[Mills-2]

On MacOS downgrading from 1.54.3 to 1.54.2 did not work for me. When I select the interpreter path through vscode to be in my venv path pythonPath is instead changed to an Xcode path. This is very unexpected to me.

Edit: if you have this issue simply performing a restart seemed to clear this issue after the downgrade. I still have the unexpected behavior however.

[angelitoTalens]

From #15781

it seems the detection of interpreter happen in a different shell and not using my venv.
weird thing is I can run unittest without the python interpreter detected since my pythonPath is set. So that means the unittest runs on my venv

Workaround:
the detection seems to work only if I populate my {$workspaceFolder}/.env file with the required LD_LIBRARY_PATH

[karrtikr]

Thanks for the bug report! We just wanted to quickly acknowledge we received it and we will triage this as soon as we can.

[karrtikr]

@meakbiyik The bug was introduced with #15726, but we plan to move away from supporting interpreter paths that way now, python.pythonPath can now only point to the binary itself (python.exe/python). We have multiple reasons for that:

• Inferring the actual python path using just the environment folder name required us to search for python executables within the directory itself by guessing which of

  vscode-python/src/client/common/configSettings.ts

  Lines 732 to 741 in c91b414

  	'python',
  	'python4',
  	'python3.6',
  	'python3.5',
  	'python3',
  	'python2.7',
  	'python2',
  	'python3.7',
  	'python3.8',
  	'python3.9',

  
  could be, and then verify whether it's a valid executable, which is observed to cause high CPU loads.
• We're likely deprecating the setting itself due to reasons mentioned in Details on the A/B experiment removing python.pythonPath support #12313. So sharing settings that way won't be possible.

You can try to workaround this using #12313 (comment).

[karrtikr]

I can temporarily fix this with #15806, but do note that we eventually plan to remove the support with #12313 🙂

[meakbiyik]

@karrtikr thanks a lot! Just to note my use case: I use poetry in all my projects with with the poetry.toml:

[virtualenvs]
create = true
in-project = true

looking like this, and "python.pythonPath": "${workspaceFolder}/.venv". So, when someone clones the repository and just runs poetry install, vscode uses the created environment and everything runs perfectly, with zero OS dependence. If vscode decides that we have to explicitly write the binary, though, then the OS independence would be lost since what vscode is automatically doing right now would have to be done manually by the user (and less meta-shenanigans the better especially for people who are not familiar with vscode and/or python). If you can take this use case into consideration that would be really amazing (this might be the single most important setting that has to be shared between the developers/users of a python package for consistency).

[karrtikr]

Yes.. thanks for bringing this up and we're well aware of this. This has been thoroughly discussed with #12313, but unfortunately for now the decision is made. You can follow the thread for updates we make. We try our best to keep everyone happy but sometimes it's not possible.

I think the workaround mentioned in #12313 (comment) should work for your case.

[meakbiyik]

That's unfortunately not the case, as the assumption of not having any settings is just extreme. I take it that there won't be any solutions to this. Eh, we'll pass that bridge then, if we can. Thanks anyways 👍

[karrtikr]

the assumption of not having any settings is just extreme

Actually the assumption is to not have any python.pythonPath setting set, not that you shouldn't have "any" settings. I have updated the comment.

[meakbiyik]

Oh that makes much more sense. Thanks, I will try that out!

[imba-tjd]

@karrtikr I think the folder support is still broken.

I installed from https://github.com/microsoft/vscode-python/wiki/Contributing. Don't know if it contains your commits.

OS: Win. Python 3.9.2 from MS Store.

[karrtikr]

The fix has not been released yet. You can use the Insiders build of the extension by using Switch to Insiders weekly command which has the fix.

[imba-tjd]

I think I already have the newest build. As I said I installed this extension from wiki page. The version number is v2021.4.690665134-dev. It doesn't have update after I set "python.insidersChannel": "daily". But I still have this issue. It works only if I set .venv\\Scripts\\python.exe

[karrtikr]

@imba-tjd Due to bug introduced with #15776 (comment), development build is no longer up to date, hence it doesn't have the fix yet. We'll fix this first thing next week, thanks for bringing this to our attention.

[imba-tjd]

I just installed the build from GitHub Actions artifacts and I can confirm "python.pythonPath": "${workspaceFolder}/.venv" works. The version is v2021.4.709858994-dev

Though it still can't detect venv. If I don't set python.pythonPath, it will use system-wide python.

---

I don't have the issue now. v2021.4.765268190

[karrtikr]

Interesting, can you please open up a separate issue for this? Also please upload the logs as mentioned in the isse template.

[karrtikr]

FYI Insiders build should now have the fix.