[DO NOT MERGE] Testing pytest regression for #11895

[nicoddemus]

This PR is only to ensure pytest-dev/pytest#11936 fixes the original regression, as I could not reproduce this on my local machine.

[nicoddemus]

I'm having trouble running this PR for more recent Python versions (where the development branch can be used), help appreciated.

[Erotemic]

The requirements.txt is parsed by the setup.py and used to populate install_requires and extra_requires, so using requirements from github will likely not work.

I'm not sure if you can specify github links for install_requires or extras_require. If you can, and you want to fix parse_requirements in setup.py to handle that. I can add that to my repo templates, so then all of my repos could handle that. (But I might move to poetry in the future, so it may be moot).

But to do this quickly, you can probably hack .github/workflows/tests.yml to get what you want. After line 328, which should look like:

        pip install --prefer-binary "xdoctest[$INSTALL_EXTRAS]==$MOD_VERSION" -f wheelhouse

you can add the pip install line to target the specific dev version of pytest.

To get rid of lower python versions, just comment them out in the test.yml. (Lines 156-295)

[nicoddemus]

But to do this quickly, you can probably hack .github/workflows/tests.yml to get what you want.

Thanks for the explanation, that will do -- I do not all builds to pass, just would like to test to ensure your original problem was fixed.

I did change tests.yml, but the build is not triggering, I guess it needs an approval to run?

[Erotemic]

You forgot to remove the change from requirements.txt, I commited it and started the CI for you.

[nicoddemus]

Reduced the test matrix now -- CI is configured to stop on first failure, so it stopped immediately in 3.6 (which is no longer supported by the installed pytest version).

[Erotemic]

Looks like this failed, if you know how to give specific users to run CI without needing me to manually click "run" each time, I'm willing to give you that permission, but I'm distracted with other tasks / meetings, so I don't have the focus to search for the right way to do this, but if you send a pointer to instructions I'll follow them.

[nicoddemus]

I believe my PRs would automatically trigger the GH Actions workflow after my first contribution, or if you give me access right away as a contributor.

However no rush, I'm also taking small stabs at this when I have some free time.

[nicoddemus]

Just noticed you sent me an invitation, thanks -- I will just test this a bit and then remove myself. 👍

[nicoddemus]

From the logs, the failures seem unrelated to the reported originally in pytest-dev/pytest#11895 (comment):

=========================== short test summary info ===========================
FAILED ..\tests\test_plugin.py::TestXDoctest::test_collect_testtextfile - pluggy._manager.PluginValidationError: Plugin 'doctest' for hook 'pytest_collect_file'
hookimpl definition: pytest_collect_file(path, parent)
Argument(s) {'path'} are declared in the hookimpl but can not be found in the hookspec
FAILED ..\tests\test_plugin.py::TestXDoctest::test_collect_module_empty - pluggy._manager.PluginValidationError: Plugin 'doctest' for hook 'pytest_collect_file'
hookimpl definition: pytest_collect_file(path, parent)
Argument(s) {'path'} are declared in the hookimpl but can not be found in the hookspec
FAILED ..\tests\test_plugin.py::TestXDoctest::test_junit_report_for_doctest - pluggy._manager.PluginValidationError: Plugin 'doctest' for hook 'pytest_collect_file'
hookimpl definition: pytest_collect_file(path, parent)
Argument(s) {'path'} are declared in the hookimpl but can not be found in the hookspec
FAILED ..\tests\test_plugin.py::TestXDoctestModuleLevel::test_doctestmodule - pluggy._manager.PluginValidationError: Plugin 'doctest' for hook 'pytest_collect_file'
hookimpl definition: pytest_collect_file(path, parent)
Argument(s) {'path'} are declared in the hookimpl but can not be found in the hookspec
FAILED ..\tests\test_plugin.py::TestXDoctestAutoUseFixtures::test_doctest_module_session_fixture - Failed: remains unmatched: '*2 passed*'
FAILED ..\tests\test_plugin.py::TestXDoctestAutoUseFixtures::test_fixture_scopes[True-module] - Failed: remains unmatched: '* 3 passed*'
FAILED ..\tests\test_plugin.py::TestXDoctestAutoUseFixtures::test_fixture_scopes[True-session] - Failed: remains unmatched: '* 3 passed*'
FAILED ..\tests\test_plugin.py::TestXDoctestAutoUseFixtures::test_fixture_scopes[True-class] - Failed: remains unmatched: '* 3 passed*'
FAILED ..\tests\test_plugin.py::TestXDoctestAutoUseFixtures::test_fixture_scopes[True-function] - Failed: remains unmatched: '* 3 passed*'
FAILED ..\tests\test_plugin.py::TestXDoctestAutoUseFixtures::test_fixture_scopes[False-module] - Failed: remains unmatched: '* 2 passed*'
FAILED ..\tests\test_plugin.py::TestXDoctestAutoUseFixtures::test_fixture_scopes[False-session] - Failed: remains unmatched: '* 2 passed*'
FAILED ..\tests\test_plugin.py::TestXDoctestAutoUseFixtures::test_fixture_scopes[False-class] - Failed: remains unmatched: '* 2 passed*'
FAILED ..\tests\test_plugin.py::TestXDoctestAutoUseFixtures::test_fixture_scopes[False-function] - Failed: remains unmatched: '* 2 passed*'

So it seems the original issue is fixed?

[Erotemic]

It seems like pytest-dev/pytest#11936 fixed the path issue! 🎉

Not sure why the above failures are happening, but they do look unrelated. Those particular tests are the ones I ported from pytest's doctest test suite, so maybe something changed there. In any case, when the new version of pytest releases I'll remove the windows patch on my side and see if the error persists.

[nicoddemus]

Thanks for taking another look.

Let's keep this open until pytest-dev/pytest#11936 gets merged, as we are discussing different alternatives there. Don't worry I will close this myself.

[nicoddemus]

Thanks @Erotemic for everything, the fix has been merged and this can be closed now.

Can you please remove my access now? Thanks again.

[nicoddemus]

Can you please remove my access now?

No need, I found how to do it myself, thanks.