pytest test discovery failed with pytest-pep8 installed #7287

zztalker · 2019-09-09T05:53:56Z

Environment data

Version: 1.38.0
Commit: 3db7e09f3b61f915d03bbfa58e258d6eee843f35
Date: 2019-09-03T21:51:09.716Z
Electron: 4.2.10
Chrome: 69.0.3497.128
Node.js: 10.11.0
V8: 6.9.427.31-electron.0
OS: Linux x64 4.15.0-36-generic
Extension Name: Python
Id: ms-python.python
Version: 2019.9.34911
Publisher: Microsoft
Python version: 3.5/3.6/3.7

Expected behavior

Tests discovered

Actual behavior

Tests discovering failed.

The workaround is to remove --pep8 keyword from pytestArgs property.

Steps to reproduce:

clone https://github.com/zztalker/just_for_test ( git clone https://github.com/zztalker/just_for_test)
Install requrements (pip install -r requirements.txt), we need only pytest-pep8
Open VS Code from folder (code .)
Turn on test discovery and add in settings:

{
    "python.testing.pytestArgs": [
        "--pep8",
        "."
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.pytestEnabled": true
}

Try to discover tests.

Logs

Output for Python Test Log in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

python /home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/run_adapter.py discover pytest -- -s --cache-clear --pep8 .
Test Discovery failed: 
Error: /home/pz/.local/lib/python3.7/site-packages/pep8.py:110: FutureWarning: Possible nested set at position 1
  EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')
============================= test session starts ==============================
platform linux -- Python 3.7.4, pytest-4.6.3, py-1.8.0, pluggy-0.12.0
rootdir: /home/pz/snippets/cool-tests, inifile: pytest.ini
plugins: pep8-1.0.6
DBG parse item <Pep8Item test_pytest1.py> test_pytest1.py None
collected 9 items
<Pep8Item test_pytest1.py>
<Module test_pytest1.py>
  <Function test_pytest1_test1>
  <Function test_pytest1_test2>
  <Function test_even_failed>
  <Function test_odd_failed>
  <Function test_random>
<Pep8Item test_foldered/test_pytest2.py>
<Module test_foldered/test_pytest2.py>
  <Function test_pytest2_test3>
  <Function test_pytest2_test4>
DBG parse item <Pep8Item test_pytest1.py> test_pytest1.py None
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/_pytest/main.py", line 498, in perform_collect
INTERNALERROR>     session=self, config=self.config, items=items
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/manager.py", line 87, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/manager.py", line 81, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 80, in pytest_collection_modifyitems
INTERNALERROR>     test, parents = self.parse_item(item)
INTERNALERROR>   File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 65, in parse_item
INTERNALERROR>     return parse_item(item)
INTERNALERROR>   File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 163, in parse_item
INTERNALERROR>     testroot, relfile = _split_fspath(str(item.fspath), fileid, item)
INTERNALERROR>   File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 142, in <lambda>
INTERNALERROR>     _split_fspath=(lambda *a: _split_fspath(*a)),
INTERNALERROR>   File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 232, in _split_fspath
INTERNALERROR>     relsuffix = fileid[1:]  # Drop (only) the "." prefix.
INTERNALERROR> TypeError: 'NoneType' object is not subscriptable
INTERNALERROR> 
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR> 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/_pytest/main.py", line 206, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/_pytest/main.py", line 249, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/manager.py", line 87, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/manager.py", line 81, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/_pytest/main.py", line 259, in pytest_collection
INTERNALERROR>     return session.perform_collect()
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/_pytest/main.py", line 501, in perform_collect
INTERNALERROR>     hook.pytest_collection_finish(session=self)
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/manager.py", line 87, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/manager.py", line 81, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 94, in pytest_collection_finish
INTERNALERROR>     test, parents = self.parse_item(item)
INTERNALERROR>   File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 65, in parse_item
INTERNALERROR>     return parse_item(item)
INTERNALERROR>   File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 163, in parse_item
INTERNALERROR>     testroot, relfile = _split_fspath(str(item.fspath), fileid, item)
INTERNALERROR>   File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 142, in <lambda>
INTERNALERROR>     _split_fspath=(lambda *a: _split_fspath(*a)),
INTERNALERROR>   File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 232, in _split_fspath
INTERNALERROR>     relsuffix = fileid[1:]  # Drop (only) the "." prefix.
INTERNALERROR> TypeError: 'NoneType' object is not subscriptable

========================== 1 warnings in 0.02 seconds ==========================

Traceback (most recent call last):
  File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/run_adapter.py", line 18, in <module>
    main(tool, cmd, subargs, toolargs)
  File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/__main__.py", line 90, in main
    parents, result = run(toolargs, **subargs)
  File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 35, in discover
    raise Exception('pytest discovery failed (exit code {})'.format(ec))
Exception: pytest discovery failed (exit code 3)

The text was updated successfully, but these errors were encountered:

kimadeline · 2019-09-09T21:04:18Z

Hi @zztalker 👋 Thank you for reporting this issue and providing sample code 👍
There is no requirements.txt in the repo you linked, so I installed the following packages in order to reproduce your issue:

pytest==4.6.3
pytest-pep8==1.0.6

Notes

Issue can be reproduced with 2019.9.34911 and 2019.8.30787 (did not try earlier versions of the extension)
Directly calling pytest --pep8 . in a terminal window works

zztalker · 2019-09-11T02:28:20Z

@kimadeline My apologies, I've forgotten to push it (requirements.txt) to the repo. I suppose all worked well till:

Name: Python
Id: ms-python.python
Description: Linting, Debugging (multi-threaded, remote), Intellisense, code formatting, refactoring, unit tests, snippets, and more.
Version: 2019.5.18875
Publisher: Microsoft
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=ms-python.python

zztalker · 2019-09-11T06:28:24Z

I believe, that problem in section, and will try to look into code for more details:

<Pep8Item test_pytest1.py>

Hecatron · 2019-09-23T00:32:32Z

I also ran into this same problem / error output when using --codestyle in the pytest options
which runs pycodestyle

kergoth · 2019-09-26T17:00:19Z

pytest-cram fails in this way as well.

For #7287.

kimadeline · 2019-11-12T23:29:21Z

✅ Validated in 2019.12.48501-dev, test discovery doesn't fail with "python.testing.pytestEnabled": true and the following settings:

 "python.testing.pytestArgs": [
        "--pep8",
        "."
    ],

zztalker added triage-needed Needs assignment to the proper sub-team bug Issue identified by VS Code Team member as probable bug labels Sep 9, 2019

kimadeline added the triage label Sep 9, 2019

kimadeline self-assigned this Sep 9, 2019

ghost removed the triage-needed Needs assignment to the proper sub-team label Sep 9, 2019

kimadeline removed their assignment Sep 9, 2019

kimadeline added area-testing needs PR and removed triage labels Sep 9, 2019

zztalker mentioned this issue Oct 28, 2019

Skip tests generated by plugins (like pep8) for discovery command. #8242

Merged

10 tasks

kimadeline added external contributor labels Oct 28, 2019

kimadeline added this to the 2019 October Sprint 3 milestone Oct 28, 2019

gramster added in progress and removed in progress labels Oct 29, 2019

kimadeline closed this as completed Nov 4, 2019

ghost removed the needs PR label Nov 4, 2019

kimadeline reopened this Nov 4, 2019

ghost added the triage-needed Needs assignment to the proper sub-team label Nov 4, 2019

kimadeline added the needs PR label Nov 4, 2019

ghost removed the triage-needed Needs assignment to the proper sub-team label Nov 4, 2019

ericsnowcurrently pushed a commit that referenced this issue Nov 6, 2019

Skip tests generated by pytest plugins during discovery. (#8242)

355a325

For #7287.

karthiknadig removed this from the 2019 October Sprint 3 milestone Nov 6, 2019

karthiknadig added this to the 2019 November Sprint 1 milestone Nov 6, 2019

zztalker mentioned this issue Nov 7, 2019

Make sure tests generated by pytest plugins are not ignored. #8425

Closed

karthiknadig assigned kimadeline Nov 7, 2019

karthiknadig removed the in review label Nov 11, 2019

kimadeline closed this as completed Nov 12, 2019

ghost removed the needs PR label Nov 12, 2019

lock bot locked as resolved and limited conversation to collaborators Nov 20, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pytest test discovery failed with pytest-pep8 installed #7287

pytest test discovery failed with pytest-pep8 installed #7287

zztalker commented Sep 9, 2019

kimadeline commented Sep 9, 2019

zztalker commented Sep 11, 2019 •

edited

Loading

zztalker commented Sep 11, 2019

Hecatron commented Sep 23, 2019

kergoth commented Sep 26, 2019

kimadeline commented Nov 12, 2019

pytest test discovery failed with pytest-pep8 installed #7287

pytest test discovery failed with pytest-pep8 installed #7287

Comments

zztalker commented Sep 9, 2019

Environment data

Expected behavior

Actual behavior

Steps to reproduce:

Logs

kimadeline commented Sep 9, 2019

Notes

zztalker commented Sep 11, 2019 • edited Loading

zztalker commented Sep 11, 2019

Hecatron commented Sep 23, 2019

kergoth commented Sep 26, 2019

kimadeline commented Nov 12, 2019

zztalker commented Sep 11, 2019 •

edited

Loading