Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Terminal environment variables loaded by extension are corrupted #23604

Open
aatle opened this issue Jun 13, 2024 · 9 comments
Open

Terminal environment variables loaded by extension are corrupted #23604

aatle opened this issue Jun 13, 2024 · 9 comments
Assignees
Labels
area-terminal info-needed Issue requires more information from poster

Comments

@aatle
Copy link

aatle commented Jun 13, 2024

Type: Bug

Behaviour

In the terminal under specific circumstances, some environment variables are loaded with invalid values because of backslashes, causing certain commands to fail.

(project-py3.12)
abcde@MY-PC MINGW64 C:/Users/abcde/AppData/Local/Programs/Microsoft VS Code
$ git --help
fatal: unable to access 'C:\Users?bcde/.config/git/config': Invalid argument
(project-py3.12) 
abcde@MY-PC MINGW64 C:/Users/abcde/AppData/Local/Programs/Microsoft VS Code
$ echo $HOME
C:\Usersbcde

It is expected that the environment variable values would not have special escape sequences from backslashes in Windows paths.

Steps to reproduce:

VS Code Python extension loading poetry virtual environment on Git Bash on Windows

  1. Use Windows (11). The name of your User Account, the name seen in C:\Users, should start with lowercase a, e.g. "abcde". (Some other characters probably also work for this: b,e,f,n,r,t,v.)
  2. Have Git installed. Have python installed (I am using 3.12.4). Have poetry installed (1.8.3 used, but probably any version) using pip.
  3. Create a new folder. Open it with VS Code and create a pyproject.toml file with the following contents:
[tool.poetry]
name = "project"
version = "0.1.0"
description = ""
authors = []

[tool.poetry.dependencies]
python = "^3.10"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
  1. Open the terminal, use Git Bash as the default (instead of Command Prompt, PowerShell).
  2. Exit VS Code and reopen the project. Kill the terminal and reopen it. The PS1 prompt will now have something like (project-py3.12), see above.
  3. Git commands such as git --help fail. Possibly some other specific commands also fail. The HOME, PATH, and EXEPATH environment variables have invalid values.

Diagnostic data

Output for Python in the Output panel

(User account name replaced with abcde.)

2024-06-12 18:06:58.366 [info] Experiment 'pythonaacf' is active
2024-06-12 18:06:58.366 [info] Experiment 'pythonCreateEnvOnPipInstall' is active
2024-06-12 18:06:58.366 [info] Experiment 'pythonRecommendTensorboardExt' is active
2024-06-12 18:06:58.366 [info] Experiment 'pythonTerminalEnvVarActivation' is active
2024-06-12 18:06:58.366 [info] Experiment 'pythonTestAdapter' is active
2024-06-12 18:06:58.366 [info] Default formatter is set to charliermarsh.ruff for workspace c:\Users\abcde\development\reproduce
2024-06-12 18:06:58.366 [info] Starting Environment refresh
2024-06-12 18:06:58.366 [info] Starting Environment refresh
2024-06-12 18:06:58.366 [info] > hatch env show --json
2024-06-12 18:06:58.366 [info] cwd: .
2024-06-12 18:06:58.366 [info] > poetry env list --full-path
2024-06-12 18:06:58.366 [info] cwd: .
2024-06-12 18:06:59.770 [info] Environments refresh finished (event): 1467 milliseconds
2024-06-12 18:06:59.770 [info] Environments refresh paths discovered: 1467 milliseconds
2024-06-12 18:06:59.771 [info] Environments refresh finished (event): 1467 milliseconds
2024-06-12 18:06:59.771 [info] Environments refresh paths discovered: 1467 milliseconds
2024-06-12 18:06:59.772 [info] Environment refresh took 1468 milliseconds
2024-06-12 18:06:59.772 [info] Environment refresh took 1469 milliseconds
2024-06-12 18:06:59.780 [info] > pyenv which python
2024-06-12 18:06:59.780 [info] cwd: .
2024-06-12 18:06:59.826 [info] Running installed packages checker:  {
  id: 'C:\\USERS\\ABCDE\\APPDATA\\LOCAL\\PYPOETRY\\CACHE\\VIRTUALENVS\\PROJECT-3JZSXIOL-PY3.12\\SCRIPTS\\PYTHON.EXE',
  sysPrefix: 'C:\\Users\\abcde\\AppData\\Local\\pypoetry\\Cache\\virtualenvs\\project-3JzsXiol-py3.12',
  envType: 'Poetry',
  envName: 'project-3JzsXiol-py3.12',
  envPath: 'C:\\Users\\abcde\\AppData\\Local\\pypoetry\\Cache\\virtualenvs\\project-3JzsXiol-py3.12',
  path: 'C:\\Users\\abcde\\AppData\\Local\\pypoetry\\Cache\\virtualenvs\\project-3JzsXiol-py3.12\\Scripts\\python.exe',
  architecture: 1,
  sysVersion: undefined,
  version: {
    raw: '3.12.4',
    major: 3,
    minor: 12,
    patch: 4,
    build: [],
    prerelease: [ 'final', '0' ]
  },
  displayName: "Python 3.12.4 ('project-3JzsXiol-py3.12')",
  detailedDisplayName: "Python 3.12.4 ('project-3JzsXiol-py3.12': Poetry)",
  type: 'Virtual'
} c:\Users\abcde\.vscode\extensions\ms-python.python-2024.8.0\python_files\installed_check.py c:\Users\abcde\development\reproduce\pyproject.toml
2024-06-12 18:06:59.826 [info] > ~\AppData\Local\pypoetry\Cache\virtualenvs\project-3JzsXiol-py3.12\Scripts\python.exe ~\.vscode\extensions\ms-python.python-2024.8.0\python_files\installed_check.py .\pyproject.toml
2024-06-12 18:06:59.828 [info] Python interpreter path: ~\AppData\Local\pypoetry\Cache\virtualenvs\project-3JzsXiol-py3.12\Scripts\python.exe
2024-06-12 18:06:59.856 [info] > . ~/AppData/Local/pypoetry/Cache/virtualenvs/project-3JzsXiol-py3.12/Scripts/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2024.8.0/python_files/printEnvVariables.py
2024-06-12 18:06:59.856 [info] shell: bash
2024-06-12 18:07:00.080 [info] > ~\AppData\Local\Programs\Python\Python312\python.exe ~/.vscode/extensions/ms-python.python-2024.8.0/python_files/printEnvVariables.py
2024-06-12 18:07:00.080 [info] shell: bash
2024-06-12 18:07:00.137 [info] Setting environment variable EXEPATH in collection to C:\Program Files\Git\bin {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-06-12 18:07:00.137 [info] Setting environment variable HOME in collection to C:\Users\abcde {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-06-12 18:07:00.137 [info] Setting environment variable MSYS in collection to disable_pcon {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-06-12 18:07:00.137 [info] Setting environment variable MSYSTEM in collection to MINGW64 {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-06-12 18:07:00.137 [info] Prepending environment variable PATH in collection to c:\Users\abcde\.vscode\extensions\ms-python.python-2024.8.0\python_files\deactivate\bash;C:\Users\abcde\AppData\Local\pypoetry\Cache\virtualenvs\project-3JzsXiol-py3.12\Scripts;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\Users\abcde\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0;C:\WINDOWS\System32\OpenSSH;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Program Files\Git\cmd;C:\Users\abcde\AppData\Local\Programs\Python\Python312\Scripts;C:\Users\abcde\AppData\Local\Programs\Python\Python312;C:\Users\abcde\AppData\Local\Microsoft\WindowsApps;C:\Users\abcde\AppData\Local\Programs\Microsoft VS Code\bin;C:\users\abcde\.local\bin; {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-06-12 18:07:00.138 [info] Setting environment variable PLINK_PROTOCOL in collection to ssh {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-06-12 18:07:00.138 [info] Prepending environment variable PS1 in collection with (project-py3.12)  {"applyAtShellIntegration":true,"applyAtProcessCreation":false}
2024-06-12 18:07:00.138 [info] Setting environment variable PWD in collection to C:/Users/abcde/AppData/Local/Programs/Microsoft VS Code {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-06-12 18:07:00.138 [info] Setting environment variable PYTHONIOENCODING in collection to utf-8 {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-06-12 18:07:00.138 [info] Setting environment variable PYTHONUNBUFFERED in collection to 1 {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-06-12 18:07:00.138 [info] Setting environment variable TERM in collection to xterm-256color {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-06-12 18:07:00.138 [info] Setting environment variable VIRTUAL_ENV in collection to C:/Users/abcde/AppData/Local/pypoetry/Cache/virtualenvs/project-3JzsXiol-py3.12 {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-06-12 18:07:00.138 [info] Setting environment variable VIRTUAL_ENV_PROMPT in collection to project-py3.12 {"applyAtShellIntegration":true,"applyAtProcessCreation":true}
2024-06-12 18:07:00.138 [info] Send text to terminal: C:/Users/abcde/AppData/Local/Programs/Python/Python312/python.exe c:\Users\abcde\.vscode\extensions\ms-python.python-2024.8.0\python_files\printEnvVariablesToFile.py c:\Users\abcde\.vscode\extensions\ms-python.python-2024.8.0\python_files\deactivate\bash\envVars.txt
2024-06-12 18:07:00.797 [info] Starting Jedi language server for reproduce.
2024-06-12 18:07:30.143 [error] Failed to initialize deactivate script C:\Program Files\Git\bin\bash.exe [Error: "c:\Users\abcde\.vscode\extensions\ms-python.python-2024.8.0\python_files\deactivate\bash\envVars.txt" file not created
	at Timeout.<anonymous> (c:\Users\abcde\.vscode\extensions\ms-python.python-2024.8.0\out\client\extension.js:2:273846)
	at listOnTimeout (node:internal/timers:573:17)
	at process.processTimers (node:internal/timers:514:7)]

Terminal Environment Changes data (open terminal, hover over bash - Microsoft VS Code, click Show environment contributions)

Note that the git extension was able to load its environment variables correctly.

# Terminal Environment Changes

## Extension: vscode.git

Enables the following features: git auth provider

- `GIT_ASKPASS=c:\Users\abcde\AppData\Local\Programs\Microsoft VS Code\resources\app\extensions\git\dist\askpass.sh`
- `VSCODE_GIT_ASKPASS_NODE=C:\Users\abcde\AppData\Local\Programs\Microsoft VS Code\Code.exe`
- `VSCODE_GIT_ASKPASS_EXTRA_ARGS=`
- `VSCODE_GIT_ASKPASS_MAIN=c:\Users\abcde\AppData\Local\Programs\Microsoft VS Code\resources\app\extensions\git\dist\askpass-main.js`
- `VSCODE_GIT_IPC_HANDLE=\\.\pipe\vscode-git-c39b414ee2-sock`

## Extension: ms-python.python

Activated environment for `~\AppData\Local\pypoetry\Cache\virtualenvs\project-3JzsXiol-py3.12\Scripts\python.exe`

- `EXEPATH=C:\Program Files\Git\bin`
- `HOME=C:\Users\abcde`
- `MSYS=disable_pcon`
- `MSYSTEM=MINGW64`
- `PATH=c:\Users\abcde\.vscode\extensions\ms-python.python-2024.8.0\python_files\deactivate\bash;C:\Users\abcde\AppData\Local\pypoetry\Cache\virtualenvs\project-3JzsXiol-py3.12\Scripts;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\Users\abcde\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0;C:\WINDOWS\System32\OpenSSH;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Program Files\Git\cmd;C:\Users\abcde\AppData\Local\Programs\Python\Python312\Scripts;C:\Users\abcde\AppData\Local\Programs\Python\Python312;C:\Users\abcde\AppData\Local\Microsoft\WindowsApps;C:\Users\abcde\AppData\Local\Programs\Microsoft VS Code\bin;C:\users\abcde\.local\bin;${env:PATH}`
- `PLINK_PROTOCOL=ssh`
- `PS1=(project-py3.12) ${env:PS1}`
- `PWD=C:/Users/abcde/AppData/Local/Programs/Microsoft VS Code`
- `PYTHONIOENCODING=utf-8`
- `PYTHONUNBUFFERED=1`
- `TERM=xterm-256color`
- `VIRTUAL_ENV=C:/Users/abcde/AppData/Local/pypoetry/Cache/virtualenvs/project-3JzsXiol-py3.12`
- `VIRTUAL_ENV_PROMPT=project-py3.12`

Three corrupted environment variables

Bash

$ echo $HOME
C:\Usersbcde

$ echo $PATH
c:\Usersbcde\.vscodetensions\ms-python.python-2024.8.0\python_files\deactivatash;C:\Usersbcde\AppData\Local\pypoetry\Cache
irtualenvs\project-3JzsXiol-py3.12\Scripts;C:\Program Files\Git\mingw6in;C:\Program Files\Git\usin;C:\Usersnthin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell
1.0;C:\WINDOWS\System32\OpenSSH;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Program Files\Git/c/Users/abcde/bin:/mingw64/bin:/usr/local/bin:/usr/bin:/bin:/mingw64/bin:/usr/bin:/c/Users/abcde/bin:/c/Users/abcde/.vscode/extensions/ms-python.python-2024.8.0/python_files/deactivate/bash:/c/Users/abcde/AppData/Local/pypoetry/Cache/virtualenvs/project-3JzsXiol-py3.12/Scripts:/mingw64/bin:/usr/bin:/c/Users/abcde/bin:/c/WINDOWS/system32:/c/WINDOWS:/c/WINDOWS/System32/Wbem:/c/WINDOWS/System32/WindowsPowerShell/v1.0:/c/WINDOWS/System32/OpenSSH:/c/Program Files (x86)/Windows Kits/10/Windows Performance Toolkit:/cmd:/c/Users/abcde/AppData/Local/Programs/Python/Python312/Scripts:/c/Users/abcde/AppData/Local/Programs/Python/Python312:/c/Users/abcde/AppData/Local/Microsoft/WindowsApps:/c/Users/abcde/AppData/Local/Programs/Microsoft VS Code/bin:/c/users/abcde/.local/bin:/c/WINDOWS/system32:/c/WINDOWS:/c/WINDOWS/System32/Wbem:/c/WINDOWS/System32/WindowsPowerShell/v1.0:/c/WINDOWS/System32/OpenSSH:/c/Program Files (x86)/Windows Kits/10/Windows Performance Toolkit:/cmd:/c/Users/abcde/AppData/Local/Programs/Python/Python312/Scripts:/c/Users/abcde/AppData/Local/Programs/Python/Python312:/c/Users/abcde/AppData/Local/Microsoft/WindowsApps:/c/Users/abcde/AppData/Local/Programs/Microsoft VS Code/bin:/c/users/abcde/.local/bin:/usr/bin/vendor_perl:/usr/bin/core_perl

$ echo $EXEPATH
C:\Program Files\Giin

Python (shows escape sequences)

$ python
Python 3.12.4 (tags/v3.12.4:8e8a4ba, Jun  6 2024, 19:30:16) [MSC v.1940 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.environ["HOME"]
'C:\\Users\x07bcde'
>>> os.environ["EXEPATH"]
'C:\\Program Files\\Git\x08in'
>>> os.environ["PATH"]
'c;C:\\Users\x07bcde\\.vscode\x1bxtensions\\ms-python.python-2024.8.0\\python_files\\deactivate\x08ash;C;C:\\Users\x07bcde\\AppData\\Local\\pypoetry\\Cache\x0birtualenvs\\project-3JzsXiol-py3.12\\Scripts;C;C:\\Program Files\\Git\\mingw64\x08in;C;C:\\Program Files\\Git\\usr\x08in;C;C:\\Users\x07bcde\x08in;C;C:\\WINDOWS\\system32;C;C:\\WINDOWS;C;C:\\WINDOWS\\System32\\Wbem;C;C:\\WINDOWS\\System32\\WindowsPowerShell\x0b1.0;C;C:\\WINDOWS\\System32\\OpenSSH;C;C:\\Program Files (x86)\\Windows Kits\\10\\Windows Performance Toolkit;C;C:\\Program Files\\Git\\c\\Users\\abcde\\bin;C:\\Program Files\\Git\\mingw64\\bin;C:\\Program Files\\Git\\usr\\local\\bin;C:\\Program Files\\Git\\usr\\bin;C:\\Program Files\\Git\\usr\\bin;C:\\Program Files\\Git\\mingw64\\bin;C:\\Program Files\\Git\\usr\\bin;C:\\Users\\abcde\\bin;C:\\Users\\abcde\\.vscode\\extensions\\ms-python.python-2024.8.0\\python_files\\deactivate\\bash;C:\\Users\\abcde\\AppData\\Local\\pypoetry\\Cache\\virtualenvs\\project-3JzsXiol-py3.12\\Scripts;C:\\Program Files\\Git\\mingw64\\bin;C:\\Program Files\\Git\\usr\\bin;C:\\Users\\abcde\\bin;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0;C:\\WINDOWS\\System32\\OpenSSH;C:\\Program Files (x86)\\Windows Kits\\10\\Windows Performance Toolkit;C:\\Program Files\\Git\\cmd;C:\\Users\\abcde\\AppData\\Local\\Programs\\Python\\Python312\\Scripts;C:\\Users\\abcde\\AppData\\Local\\Programs\\Python\\Python312;C:\\Users\\abcde\\AppData\\Local\\Microsoft\\WindowsApps;C:\\Users\\abcde\\AppData\\Local\\Programs\\Microsoft VS Code\\bin;C:\\users\\abcde\\.local\\bin;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0;C:\\WINDOWS\\System32\\OpenSSH;C:\\Program Files (x86)\\Windows Kits\\10\\Windows Performance Toolkit;C:\\Program Files\\Git\\cmd;C:\\Users\\abcde\\AppData\\Local\\Programs\\Python\\Python312\\Scripts;C:\\Users\\abcde\\AppData\\Local\\Programs\\Python\\Python312;C:\\Users\\abcde\\AppData\\Local\\Microsoft\\WindowsApps;C:\\Users\\abcde\\AppData\\Local\\Programs\\Microsoft VS Code\\bin;C:\\users\\abcde\\.local\\bin;C:\\Program Files\\Git\\usr\\bin\\vendor_perl;C:\\Program Files\\Git\\usr\\bin\\core_perl'

Notes

It would seem that the Python extension is attempting to load environment variables with values that include backslashes because of Windows paths.
But somewhere in the process, a few of the backslashes are being interpreted as escape sequences. For example \v becomes a vertical tab and \b becomes a backspace.
The interpretation aligns with how a .env file interprets backslashes. See C Escape Sequences.
If the user does not have a user account name that starts with a valid escape sequence character, only PATH is corrupted, and not enough to be immediately noticeable.\

A workaround is adding "python.experiments.optOutFrom": ["pythonTerminalEnvVarActivation"] to settings.json.

Extension version: 2024.8.0
VS Code version: Code 1.90.0 (89de5a8d4d6205e5b11647eb6a74844ca23d2573, 2024-06-04T19:33:54.889Z)
OS version: Windows_NT x64 10.0.22631
Modes:

  • Python version (& distribution if applicable, e.g. Anaconda): 3.12.4
  • Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Poetry
  • Value of the python.languageServer setting: Default
User Settings


languageServer: "Jedi"

experiments
• optOutFrom: ["pythonTerminalEnvVarActivation"]

Installed Extensions
Extension Name Extension Id Version
Azure Repos ms-vscode.azure-repos 0.38.0
Even Better TOML tamasfe.even-better-toml 0.19.2
GitHub Pull Requests GitHub.vscode-pull-request-github 0.88.1
GitHub Repositories GitHub.remotehub 0.62.0
JavaScript Debugger ms-vscode.js-debug 1.90.0
JavaScript Debugger Companion Extension ms-vscode.js-debug-companion 1.1.2
Mypy matangover.mypy 0.3.1
Python ms-python.python 2024.8.0
Python Debugger ms-python.debugpy 2024.6.0
Remote Repositories ms-vscode.remote-repositories 0.40.0
Ruff charliermarsh.ruff 2024.26.0
Table Visualizer for JavaScript Profiles ms-vscode.vscode-js-profile-table 1.0.9
System Info
Item Value
CPUs 11th Gen Intel(R) Core(TM) i7-11700F @ 2.50GHz (16 x 2496)
GPU Status 2d_canvas: enabled
canvas_oop_rasterization: enabled_on
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: enabled
Load (avg) undefined
Memory (System) 15.77GB (6.93GB free)
Process Argv --crash-reporter-id 937dcfed-06f3-46d6-bf2d-b1f57e7b00a9
Screen Reader no
VM 0%
A/B Experiments
vsliv368cf:30146710
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscorecescf:30445987
vscod805cf:30301675
binariesv615:30325510
vsaa593:30376534
py29gd2263:31024239
c4g48928:30535728
azure-dev_surveyone:30548225
962ge761:30959799
pythongtdpath:30769146
welcomedialog:30910333
pythonidxpt:30866567
pythonnoceb:30805159
asynctok:30898717
pythontestfixt:30902429
pythonregdiag2:30936856
pythonmypyd1:30879173
h48ei257:31000450
pythontbext0:30879054
accentitlementsc:30995553
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
jchc7451:31067544
chatpanelc:31048052
dsvsc021:30996838
f3je6385:31013174
pythoncenvpt:31062603
a69g1124:31058053
dvdeprecation:31068756
pythonprt:31056678
dwnewjupytercf:31046870
newcmakeconfigv2:31071590
26j00206:31048877

@github-actions github-actions bot added the triage-needed Needs assignment to the proper sub-team label Jun 13, 2024
@anthonykim1
Copy link

Hi @aatle Thanks for filing this issue.

Can you opt out from the new terminal environment variable experiment for now and see if the problem persist?
You can do this by adding "python.experiments.optOutFrom": ["pythonTerminalEnvVarActivation"], into your settings.json

@anthonykim1 anthonykim1 added area-terminal and removed triage-needed Needs assignment to the proper sub-team labels Jun 20, 2024
@github-actions github-actions bot added the info-needed Issue requires more information from poster label Jun 20, 2024
@aatle
Copy link
Author

aatle commented Jun 20, 2024

As I have said in the original post, that setting is indeed a workaround and fixes the problem. Unfortunately, it is actually not a perfect solution because it also introduces a different problem.

(Unrelated to the setting), the Python extension will sometimes still activate the poetry virtual environment but decide to only edit (and corrupt) the PATH variable, instead of editing a bunch of variables. Not sure why.

The Environment changes show the differences:

Project 1: Only PATH corrupted, and the prompt (PS1) stays the same:

## Extension: ms-python.python

Activated environment for `~\AppData\Local\pypoetry\Cache\virtualenvs\pyriak-WneeXSsL-py3.12\Scripts\python.exe`

- `PATH=c:\Users\abcde\.vscode\extensions\ms-python.python-2024.8.1\python_files\deactivate\bash;C:\Users\abcde\AppData\Local\pypoetry\Cache\virtualenvs\pyriak-WneeXSsL-py3.12\Scripts;${env:PATH}`

Project 2: reproducible example, prompt changes to show virtual environment, more variables corrupted:

## Extension: ms-python.python

Activated environment for `~\AppData\Local\pypoetry\Cache\virtualenvs\project-3JzsXiol-py3.12\Scripts\python.exe`

- `EXEPATH=C:\Program Files\Git\bin`
- `HOME=C:\Users\abcde`
- `MSYS=disable_pcon`
- `MSYSTEM=MINGW64`
- `PATH=c:\Users\abcde\.vscode\extensions\ms-python.python-2024.8.1\python_files\deactivate\bash;C:\Users\abcde\AppData\Local\pypoetry\Cache\virtualenvs\project-3JzsXiol-py3.12\Scripts;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\Users\abcde\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0;C:\WINDOWS\System32\OpenSSH;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Program Files\Git\cmd;C:\Users\abcde\AppData\Local\Programs\Python\Python312\Scripts;C:\Users\abcde\AppData\Local\Programs\Python\Python312;C:\Users\abcde\AppData\Local\Microsoft\WindowsApps;C:\Users\abcde\AppData\Local\Programs\Microsoft VS Code\bin;C:\users\abcde\.local\bin;${env:PATH}`
- `PLINK_PROTOCOL=ssh`
- `PS1=(project-py3.12) ${env:PS1}`
- `PWD=C:/Users/abcde/AppData/Local/Programs/Microsoft VS Code`
- `PYTHONIOENCODING=utf-8`
- `PYTHONUNBUFFERED=1`
- `TERM=xterm-256color`
- `VIRTUAL_ENV=C:/Users/abcde/AppData/Local/pypoetry/Cache/virtualenvs/project-3JzsXiol-py3.12`
- `VIRTUAL_ENV_PROMPT=project-py3.12`

For the latter Project 2, the setting indeed fixes things, no environment variables are corrupted, and the extension does not contribute to terminal environment.

For the former, the setting seems to corrupt PATH even more, so sed and other commands are not found, rendering the terminal almost useless. The environment changes show that the Python extension is still contributing PATH, but has a different value.

Project 1 terminal environment changes, with setting enabled
## Extension: ms-python.python

- `PATH=C:\Users\abcde\AppData\Local\pypoetry\Cache\virtualenvs\pyriak-WneeXSsL-py3.12\Scripts;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Program Files\Git\cmd;C:\Users\abcde\AppData\Local\Programs\Python\Python312\Scripts\;C:\Users\abcde\AppData\Local\Programs\Python\Python312\;C:\Users\abcde\AppData\Local\Microsoft\WindowsApps;C:\Users\abcde\AppData\Local\Programs\Microsoft VS Code\bin;c:\users\abcde\.local\bin`

Copy link

Hey @anthonykim1, this issue might need further attention.

@aatle, you can help us out by closing this issue if the problem no longer exists, or adding more information.

@aatle
Copy link
Author

aatle commented Aug 1, 2024

This issue still exists, and makes all git commands fail on some of my poetry projects.

@anthonykim1
Copy link

but decide to only edit (and corrupt) the PATH variable, instead of editing a bunch of variables. Not sure why.

I believe this is part of the Terminal Env Var experiment as we prepend PATH and calculate other environment variables from user. #22905

When you say "corrupted" PATH, are you looking the prepending pattern?

@aatle
Copy link
Author

aatle commented Aug 12, 2024

Yes, in PATH, I am mainly looking at the prepended paths that have backslashes in them.

My original post isn't entirely clear, I'll clarify and summarize what I think is happening:

I am using Windows, poetry virtual environment, and Git Bash terminal. From my understanding, the opt-out experimental feature edits and calculates certain environment variables.
The feature has one problem: in the environment variables edited, some backslash literals (often present in Windows paths) become special escape sequences e.g. \v, \n. Only certain characters such as a and n can form escape sequences with backslash, so not all backslashes are corrupted.

So, paths in the edited environment variables such as C:\Program Files\Git\bin become C:\Program Files\Giin where there is a \b backspace character in it. This obviously renders those paths useless/corrupted.
This problem is worse if the user's $HOME variable is also affected. C:\Users\abcde (calculated) -> C:\Usersbcde with special \a character. (Without Python extension env var experiment, it would be /c/Users/abcde, using forward slashes.)

So, I think that there is a backslash issue somewhere in the process of parsing, editing, and exporting environment variable strings.

If I opt out of this feature using the setting, then some of my other environments, which look slightly different for some reason, get even more corrupted such that basic commands like sed are not found and PATH is corrupted.

@anthonykim1
Copy link

anthonykim1 commented Sep 3, 2024

@aatle Apologies for late response and thanks for further explanation here.

paths in the edited environment variables such as C:\Program Files\Git\bin become C:\Program Files\Giin where there is a \b backspace character in it.

This is interesting. Are you able to click on show environment contribution and see if WHO (for example Python extension specifically) is contributing to your corrupted environment?
Screenshot 2024-09-02 at 11 58 31 PM

Screenshot 2024-09-02 at 11 58 39 PM

Also note that GitBash is currently not officially supported by Python extension at this moment: #23382

If I opt out of this feature using the setting, then some of my other environments, which look slightly different for some reason, get even more corrupted such that basic commands like sed are not found and PATH is corrupted.

If this is the case, then the problem may reside outside of Python extension.
What the Terminal Env Variable experiment does is, it tries to "activate" your environment using environment variables instead of explicitly running activate script on your terminal.
You will see the "activate" line of command in terminal if you opt out of this experiment, but if that STILL causes a problem then it becomes very hard to tell because Python extension would not mingle around with your PATH if you opt-out of the experiment.

Are you able to "correctly" activate your environment outside of VS Code environment? (External terminal for example)

@aatle
Copy link
Author

aatle commented Sep 4, 2024

@anthonykim1 No worries, my workflow has not been inconvenienced by this.
(Also, I did not know that GitBash is not officially supported currently, I use it as default on Windows. Hopefully it is officially supported soon.)

I am mostly certain that this bug is caused by the Microsoft Python extension, in certain conditions.
The other shells I tested, powershell, cmd, GitBash outside of VS Code, they seemed to work fine when using poetry shell venv.
Checking the Environment Contributors -> Show Environment Contributions, I see this file:

Environment Changes
# Terminal Environment Changes

## Extension: vscode.git

Enables the following features: git auth provider

- `GIT_ASKPASS=c:\Users\abcde\AppData\Local\Programs\Microsoft VS Code\resources\app\extensions\git\dist\askpass.sh`
- `VSCODE_GIT_ASKPASS_NODE=C:\Users\abcde\AppData\Local\Programs\Microsoft VS Code\Code.exe`
- `VSCODE_GIT_ASKPASS_EXTRA_ARGS=`
- `VSCODE_GIT_ASKPASS_MAIN=c:\Users\abcde\AppData\Local\Programs\Microsoft VS Code\resources\app\extensions\git\dist\askpass-main.js`
- `VSCODE_GIT_IPC_HANDLE=\\.\pipe\vscode-git-b1ae1b85c2-sock`

## Extension: ms-python.python

Activated environment for `~\AppData\Local\pypoetry\Cache\virtualenvs\non-package-mode-7fHQA28B-py3.12\Scripts\python.exe`

- `EXEPATH=C:\Program Files\Git\bin`
- `HOME=C:\Users\abcde`
- `MSYS=disable_pcon`
- `MSYSTEM=MINGW64`
- `PATH=c:\Users\abcde\.vscode\extensions\ms-python.python-2024.12.3-win32-x64\python_files\deactivate\bash;C:\Users\abcde\AppData\Local\pypoetry\Cache\virtualenvs\non-package-mode-7fHQA28B-py3.12\Scripts;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\Users\abcde\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0;C:\WINDOWS\System32\OpenSSH;C:\Program Files\Git\cmd;C:\Users\abcde\AppData\Local\Programs\Python\Python312\Scripts;C:\Users\abcde\AppData\Local\Programs\Python\Python312;C:\Users\abcde\AppData\Local\Microsoft\WindowsApps;C:\Users\abcde\AppData\Local\Programs\Microsoft VS Code\bin;C:\users\abcde\.local\bin;${env:PATH}`
- `PLINK_PROTOCOL=ssh`
- `PS1=(non-package-mode-py3.12) ${env:PS1}`
- `PWD=C:/Users/abcde/AppData/Local/Programs/Microsoft VS Code`
- `PYTHONIOENCODING=utf-8`
- `PYTHONUNBUFFERED=1`
- `TERM=xterm-256color`
- `VIRTUAL_ENV=C:/Users/abcde/AppData/Local/pypoetry/Cache/virtualenvs/non-package-mode-7fHQA28B-py3.12`
- `VIRTUAL_ENV_PROMPT=non-package-mode-py3.12`

This information and more is also available in the original post (some of it might be slightly outdated).

The invalid environment variables are: HOME, EXEPATH, and a part of PATH.
The only other environment contributor is the git extension, and their variables seem fine. The issue does not occur when the Python extension is turned off.

I realized from more testing, it is indeed fixable without more issues by opting out of the terminal var experiment with settings.json. Turning off the feature makes the Python extension not contribute to the environment using variables, and instead uses activation script as you said.

"python.experiments.optOutFrom": ["pythonTerminalEnvVarActivation"],

Previously I said that this setting caused even worse errors on some projects, but it seems that this only applied to one specific older project I have, which I fixed.
The issue will also only manifest if the selected python interpreter ("Python: Select Interpreter" command) is the poetry virtual environment one, not the global python interpreter, otherwise the venv doesn't get activated.

Conditions for bug, updated:

  • Windows (a specific user name is not required, but affects severity)
  • Using GitBash (from Git for Windows), as terminal inside VS Code
  • In a project, a poetry virtual environment is created and used
  • The poetry virtual environment interpreter is the selected python interpreter in VS Code

--> Causes certain Python extension environment variable contributions to be invalid because of backslashes in paths

Copy link

github-actions bot commented Oct 4, 2024

Hey @anthonykim1, this issue might need further attention.

@aatle, you can help us out by closing this issue if the problem no longer exists, or adding more information.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-terminal info-needed Issue requires more information from poster
Projects
None yet
Development

No branches or pull requests

2 participants