Pytest Discovery does nothing when python.testing.cwd is set #22504

dmakhno · 2023-11-20T17:54:34Z

Type: Bug

Behaviour

Expected vs. Actual

Expect pytest discovery can found tests. And if not - reports what is wrong.

Steps to reproduce:

create workspace-folder | create workspace with folder (I checked both, thinking of Overload methods testing rewrite #21082 python.testing.pytestEnabled isn't respected for discovery #22218)
inside any folder create pytest_issue
inside pytest_issue/test_foo.py with any test, e.g. def test_foo(): ....
ensure config for workspace or folder:

    "python.testing.cwd": "pytest_issue",
    "python.testing.pytestArgs": [],
    "python.testing.unittestEnabled": false,
    "python.testing.pytestEnabled": true,

Run Discovery couple times:

2023-11-20 19:27:24.225 [info] Discover tests for workspace name: projects - uri: /Users/user/projects
2023-11-20 19:27:24.225 [info] Running discovery for pytest using the new test adapter.
2023-11-20 19:27:32.465 [info] Discover tests for workspace name: projects - uri: /Users/user/projects
2023-11-20 19:27:32.465 [info] Running discovery for pytest using the new test adapter.
2023-11-20 19:37:25.089 [info] Discover tests for workspace name: projects - uri: /Users/user/projects
2023-11-20 19:37:25.089 [info] Running discovery for pytest using the new test adapter.

nothing. :(
No steps further of:

vscode-python/src/client/testing/testController/controller.ts

Line 271 in f6cfa6e

traceInfo(`Running discovery for pytest using the new test adapter.`);

As soon as I delete python.testing.cwd, it starts working. (But my real env needs python.testing.cwd, so no WA)
If I Reload Window, I got:

2023-11-20 19:38:32.144 [info] > ./venv/bin/python -m pytest -p vscode_pytest --collect-only
2023-11-20 19:38:32.144 [info] cwd: pytest_issue
2023-11-20 19:38:32.636 [info] Starting Pylance language server.
2023-11-20 19:38:39.050 [info] Discover tests for workspace name: projects - uri: /Users/user/projects
2023-11-20 19:38:39.050 [info] Running discovery for pytest using the new test adapter.

In other times to --collect-only no calls. Silence.

Diagnostic data

Python version (& distribution if applicable, e.g. Anaconda): Cpython 3.9 - 3.11
Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Venv or Local
Value of the python.languageServer setting: Default
I tried on Darwing, Ubuntu, Centos
Local env was used with Remote SSH. While ven fully on host - as repro.

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

2023-11-20 19:27:24.225 [info] Discover tests for workspace name: projects - uri: /Users/user/projects
2023-11-20 19:27:24.225 [info] Running discovery for pytest using the new test adapter.
2023-11-20 19:27:32.465 [info] Discover tests for workspace name: projects - uri: /Users/user/projects
2023-11-20 19:27:32.465 [info] Running discovery for pytest using the new test adapter.
2023-11-20 19:37:25.089 [info] Discover tests for workspace name: projects - uri: /Users/user/projects
2023-11-20 19:37:25.089 [info] Running discovery for pytest using the new test adapter.

User Settings

    "python.testing.cwd": "pytest_issue",
    "python.testing.pytestArgs": [],
    "python.testing.unittestEnabled": false,
    "python.testing.pytestEnabled": true,

Extension version: 2023.20.0
VS Code version: Code 1.84.2 (Universal) (1a5daa3a0231a0fbba4f14db7ec463cf99d7768e, 2023-11-09T10:52:33.687Z)
OS version: Darwin arm64 23.1.0
Modes:

System Info

Item	Value
CPUs	Apple M2 Max (12 x 24)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: enabled
Load (avg)	1, 2, 2
Memory (System)	96.00GB (29.92GB free)
Process Argv	--crash-reporter-id 0ddd1806-9bb8-4a88-8fc3-dfe586936e48
Screen Reader	no
VM	0%

A/B Experiments

vsliv368:30146709
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vswsl492:30256859
vslsvsres303:30308271
vserr242cf:30382550
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vshan820:30294714
vstes263cf:30335440
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593:30376534
pythonvs932:30410667
py29gd2263:30880072
vsclangdc:30486549
c4g48928:30535728
dsvsc012cf:30540253
pynewext54:30695312
azure-dev_surveyone:30548225
282f8724:30602487
f6dab269:30613381
2i9eh265:30646982
showlangstatbar:30737416
pythonfmttext:30731395
fixshowwlkth:30771522
showindicator:30805244
pythongtdpath:30769146
i26e3531:30792625
pythonnosmt12:30797651
pythonidxpt:30866567
pythonnoceb:30805159
synctok:30869157
dsvsc013:30795093
dsvsc014:30804076
dsvsc015:30845448
pythontestfixtcf:30871695
pythonregdiag2:30871582
pyreplss1:30897532
pythonmypyd1:30879173
pythoncet0:30885854
2e7ec940:30885897
pythontbext0:30879054
dsvsc016cf:30886111
dsvsc017:30886112
dsvsc018cf:30886115
aa_t_chat:30882232

The text was updated successfully, but these errors were encountered:

ivan-the-terrible · 2023-11-20T23:59:29Z

Happy to see this reported as I'm also suddenly experiencing this issue as well:

BUT
If I revert to version 1.83, it works as expected and doesn't get stuck.

eleanorjboyd · 2023-11-27T17:23:33Z

Hello! I have just returned from vacation and will take a look at this shortly, thank you for your patience!

eleanorjboyd · 2023-11-27T19:57:51Z

able to repro- will investigate!

macieyng · 2023-12-08T12:54:09Z

@eleanorjboyd any progress?
I'm also having this issue.

eleanorjboyd · 2023-12-08T18:41:28Z

@dmakhno, I took a look and it seems that if you specify your cwd as "python.testing.cwd": "./pytest_issue", it should work!

eleanorjboyd · 2023-12-08T18:42:13Z

closing as that was the fix for @dmakhno, but anyone please comment if a fix to your path in your cwd is not working. If you do, please provide logs and a project structure like @dmakhno did. Thanks

eleanorjboyd · 2023-12-08T18:43:23Z

@ivan-the-terrible, it seems like your issue might be different since you aren't specifying a cwd. If you could open a new issue I can help you there. Thanks

eleanorjboyd · 2023-12-08T19:11:11Z

actually I am going to reopen this, there should be some sort of error message to signal the incorrect cwd is the problem. Will investigate that, thanks

dmakhno · 2023-12-11T09:11:17Z

@eleanorjboyd I need some time to check.
I'm definitely sure it used to work as I mentioned. Not sure why need to start the path for ./
And I fully agree - silence is not right.

AndreasHaure · 2023-12-11T10:01:36Z

@eleanorjboyd encountering the same error after latest update of vscode, this is my settings.json:

{
  "python.defaultInterpreterPath": ".venv/bin/python",
  "python.testing.pytestArgs": [
    "tests"
  ],
  "python.testing.unittestEnabled": false,
  "python.testing.pytestEnabled": true,
  "python.testing.cwd": "${workspaceFolder}",
  "python.envFile": "${workspaceFolder}/config/test.env"
}

And this is the pytest discovery output from vscode:

2023-12-11 10:55:45.495 [info] Discover tests for workspace name: [REDACTED] - uri: /Users/andreashaure/source/[REDACTED]
2023-12-11 10:55:45.504 [info] > ~/source/[REDACTED]/.venv ~/.vscode/extensions/ms-python.python-2023.22.0/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ~/source/[REDACTED] -s --cache-clear  tests
2023-12-11 10:55:45.504 [info] cwd: ~/source/[REDACTED]
2023-12-11 10:55:45.506 [error] Error discovering pytest tests:
 Error: spawn /Users/andreashaure/source/[REDACTED]/.venv EACCES
    at Process.ChildProcess._handle.onexit (node:internal/child_process:283:19)
    at onErrorNT (node:internal/child_process:476:16)
    at processTicksAndRejections (node:internal/process/task_queues:82:21) {
  errno: -13,
  code: 'EACCES',
  syscall: 'spawn /Users/andreashaure/source/[REDACTED]/.venv',
  path: '/Users/andreashaure/source/[REDACTED]/.venv',
  spawnargs: [
    '/Users/andreashaure/.vscode/extensions/ms-python.python-2023.22.0/pythonFiles/testing_tools/run_adapter.py',
    'discover',
    'pytest',
    '--',
    '--rootdir',
    '/Users/andreashaure/source/[REDACTED]',
    '-s',
    '--cache-clear',
    'tests'
  ]
}

eleanorjboyd · 2023-12-11T16:37:32Z

Thanks @AndreasHaure!
Working with my team to determine the reason an error message isn't returning and some more logging when the failure does occur to help.
Ill keep this issue updated as I go.

dmakhno · 2023-12-14T17:05:49Z

@eleanorjboyd,
today I tried on

Version: 1.85.0 (Universal)
Commit: af28b32d7e553898b2a91af498b1fb666fdebe0c
Date: 2023-12-06T18:18:04.614Z
Electron: 25.9.7
ElectronBuildId: 25551756
Chromium: 114.0.5735.289
Node.js: 18.15.0
V8: 11.4.183.29-electron.0
OS: Darwin arm64 23.1.0

"python.testing.cwd": "./pytest_issue"

WA - Does not work!

Some tiny anomaly I see, if I just start/restart vscode - discovery pretend to start.
Under "pretend" I mean extra log

2023-12-14 19:00:47.304 [info] Discover tests for workspace name: projects - uri: /Users/user/projects
2023-12-14 19:00:47.304 [info] Running discovery for pytest using the new test adapter.
2023-12-14 19:00:47.305 [info] All environment variables set for pytest discovery for workspace <CUTTED>
2023-12-14 19:00:47.321 [info] > /usr/bin/python3 -m pytest -p vscode_pytest --collect-only
2023-12-14 19:00:47.321 [info] cwd: ./pytest_issue

And no more.

But when I run "Test: Refresh Tests" command. Absolutely nothing

2023-12-14 19:02:51.655 [info] Discover tests for workspace name: projects - uri: /Users/user/projects
2023-12-14 19:02:51.655 [info] Running discovery for pytest using the new test adapter.
2023-12-14 19:03:26.626 [info] Discover tests for workspace name: projects - uri: /Users/user/projects
2023-12-14 19:03:26.626 [info] Running discovery for pytest using the new test adapter.

It happens for both cwd with/wo dot.

eleanorjboyd · 2023-12-14T19:00:22Z

Hi @dmakhno, sorry it isn't working still. This is the project I made to test it, let me know how yours differs. You could also try my project on your machine and see if that works.
cwd_specified 2.zip

dmakhno · 2023-12-15T16:27:09Z

@eleanorjboyd, yep it is the same. Thanks!

I found what was wrong for me. Hope following make sense for you.
All times I restarted vscode with "python.testing.cwd": "pytest_issue", after this when I even fixed path, it didn't help. Something got stuck.
When I opened your project, path was correct from the start, and it keep working.
So, I set path with dot for me, restarted vscode and it worked!

I wonder why first discovery atempt makes such effect? (shrug)

Checked on full size project, WA works. After proper setting of cwd need vscode restart. "Refresh Tests" is useless.

Some minor observation.
Remote (centos like os):
"python.testing.cwd": "./unit_test"

2023-12-15 18:18:36.929 [info] > /bin/python -m pytest -p vscode_pytest --collect-only
2023-12-15 18:18:36.929 [info] cwd: /local/user/repos/project/unit_tests

cwd is resolved to full path

Local - your project (darwin):

2023-12-15 18:09:11.542 [info] > /usr/bin/python3 -m pytest -p vscode_pytest --collect-only
2023-12-15 18:09:11.542 [info] cwd: ./pytest_issue

cwd stays relative.

eleanorjboyd · 2023-12-15T20:53:04Z

ah yes this makes sense "After proper setting of cwd need vscode restart. "Refresh Tests" is useless.", I am looking into why the extension doesn't error out but it has a behavior where if discovery is already running it will not run again even with new args because discovery is expensive. As I work to figure out how to get the extension to error out on the wrong cwd I had added some logging to notify the user that discovery is not being re-run, hopefully this will help the next user. Sorry for the difficulty! a05dee8

ivan-the-terrible · 2023-12-25T22:10:56Z

@ivan-the-terrible, it seems like your issue might be different since you aren't specifying a cwd. If you could open a new issue I can help you there. Thanks

Yeah I think I might running up against the fact that discovery doesn't rerun and reopening VSCode helped as it was able to be rediscovered after some attempts. I'm not entirely sure...

But I downloaded the sample project you created and get the same error on Windows (running VSCode v1.85.1).
As I was ensuring the virtual environment was configured properly, I also noticed in the Testing panel it will append the testing structure and leave some vestigial structures.

Setting the "python.testing.cwd": "${workspaceFolder}/pytest_issue", (or to ./pytest_issue) and removing it leaves discovery in a bad state.

github-actions · 2024-01-25T12:32:09Z

Hey @eleanorjboyd, this issue might need further attention.

@dmakhno, you can help us out by closing this issue if the problem no longer exists, or adding more information.

dmakhno · 2024-01-25T19:09:05Z

@eleanorjboyd, github-actions is worried that something is needed. :-)

Lookiing forward for:

nicer logging
(optionally) for relative path, no need to use . (dot). As it used to work.

github-actions bot added the triage-needed Needs assignment to the proper sub-team label Nov 20, 2023

dmakhno changed the title ~~Pytest Discovery does nothing when python.testing.cmd is set~~ Pytest Discovery does nothing when python.testing.cwd is set Nov 20, 2023

karthiknadig assigned eleanorjboyd Nov 20, 2023

eleanorjboyd added the area-testing label Nov 27, 2023

macieyng mentioned this issue Dec 8, 2023

Test discovery command does not call the python interpreter #22612

Closed

eleanorjboyd closed this as completed Dec 8, 2023

github-actions bot added the info-needed Issue requires more information from poster label Dec 8, 2023

eleanorjboyd reopened this Dec 8, 2023

eleanorjboyd removed the info-needed Issue requires more information from poster label Dec 8, 2023

github-actions bot added the info-needed Issue requires more information from poster label Dec 11, 2023

github-actions bot removed the info-needed Issue requires more information from poster label Dec 14, 2023

github-actions bot added the info-needed Issue requires more information from poster label Dec 14, 2023

github-actions bot removed the info-needed Issue requires more information from poster label Dec 15, 2023

github-actions bot added the info-needed Issue requires more information from poster label Dec 15, 2023

github-actions bot removed the info-needed Issue requires more information from poster label Jan 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pytest Discovery does nothing when python.testing.cwd is set #22504

Pytest Discovery does nothing when python.testing.cwd is set #22504

dmakhno commented Nov 20, 2023

ivan-the-terrible commented Nov 20, 2023

eleanorjboyd commented Nov 27, 2023

eleanorjboyd commented Nov 27, 2023

macieyng commented Dec 8, 2023

eleanorjboyd commented Dec 8, 2023

eleanorjboyd commented Dec 8, 2023

eleanorjboyd commented Dec 8, 2023

eleanorjboyd commented Dec 8, 2023

dmakhno commented Dec 11, 2023

AndreasHaure commented Dec 11, 2023

eleanorjboyd commented Dec 11, 2023

dmakhno commented Dec 14, 2023

eleanorjboyd commented Dec 14, 2023 •

edited

Loading

dmakhno commented Dec 15, 2023 •

edited

Loading

eleanorjboyd commented Dec 15, 2023

ivan-the-terrible commented Dec 25, 2023

github-actions bot commented Jan 25, 2024

dmakhno commented Jan 25, 2024

Pytest Discovery does nothing when python.testing.cwd is set #22504

Pytest Discovery does nothing when python.testing.cwd is set #22504

Comments

dmakhno commented Nov 20, 2023

Behaviour

Expected vs. Actual

Steps to reproduce:

Diagnostic data

ivan-the-terrible commented Nov 20, 2023

eleanorjboyd commented Nov 27, 2023

eleanorjboyd commented Nov 27, 2023

macieyng commented Dec 8, 2023

eleanorjboyd commented Dec 8, 2023

eleanorjboyd commented Dec 8, 2023

eleanorjboyd commented Dec 8, 2023

eleanorjboyd commented Dec 8, 2023

dmakhno commented Dec 11, 2023

AndreasHaure commented Dec 11, 2023

eleanorjboyd commented Dec 11, 2023

dmakhno commented Dec 14, 2023

eleanorjboyd commented Dec 14, 2023 • edited Loading

dmakhno commented Dec 15, 2023 • edited Loading

eleanorjboyd commented Dec 15, 2023

ivan-the-terrible commented Dec 25, 2023

github-actions bot commented Jan 25, 2024

dmakhno commented Jan 25, 2024

eleanorjboyd commented Dec 14, 2023 •

edited

Loading

dmakhno commented Dec 15, 2023 •

edited

Loading