Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pytest plugin re-runs all tests if a crash occurs inside a pytest_runtestloop hook when using pytest-xdist #24846

Open
DetachHead opened this issue Feb 25, 2025 · 1 comment
Assignees
Labels
triage-needed Needs assignment to the proper sub-team

Comments

@DetachHead
Copy link

DetachHead commented Feb 25, 2025

Type: Bug

Behaviour

when a crash occurs inside a pytest_runtestloop hook while the xdist plugin is active, it will attempt to re-run all tests.

Steps to reproduce:

  1. create the following files:

    # tests/test_foo.py
    
    from pytest import mark
    
    
    @mark.parametrize('i', list(range(2)))
    def test_asdf(i: int):
        pass
    # tests/conftest.py
    
    from pytest import Session, hookimpl
    
    @hookimpl(tryfirst=True)
    def pytest_runtestloop(session: Session):
        raise Exception(f'failed running tests with args: {session.config.args}')
    // .vscode/settings.json
    
    {
        "python.testing.pytestArgs": [
            "tests", "-n", "auto"
        ],
        "python.testing.unittestEnabled": false,
        "python.testing.pytestEnabled": true
    }
  2. use the test explorer to run the parametrized test:
    Image

  3. check the test results tab. notice there are two separate crashes:

    INTERNALERROR> Exception: failed running tests with args: ['c:\\Users\\user\\asdf2\\tests\\test_foo.py::test_asdf[0]', 'c:\\Users\\user\\asdf2\\tests\\test_foo.py::test_asdf[1]']
    
    INTERNALERROR> Exception: failed running tests with args: ['c:\\Users\\user\\asdf2']
    

the first crash is expected, but the second one is not. it seems to be re-running pytest after the crash with the project root as the only argument, which tells pytest to run all tests in the project.

Extension version: 2025.1.2025022102
VS Code version: Code 1.97.2 (e54c774e0add60467559eb0d1e229c6452cf8447, 2025-02-12T23:20:35.343Z)
OS version: Windows_NT x64 10.0.22621
Modes:

@github-actions github-actions bot added the triage-needed Needs assignment to the proper sub-team label Feb 25, 2025
@DetachHead DetachHead changed the title pytest plugin re-runs all tests if a crash occurs inside a pytest_runtestloop hook pytest plugin re-runs all tests if a crash occurs inside a pytest_runtestloop hook when using pytest-xdist Feb 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
triage-needed Needs assignment to the proper sub-team
Projects
None yet
Development

No branches or pull requests

2 participants