Please reinstate "testing" extra

[mbra]

We are using the tox.pytest module to test our pytest plugin. Since it is importing quite a few non-standard dependencies we use the "testing" extra to include its dependencies when running tests. In tox 4.22.0 this extra was removed in favor of dependency-groups (#3409). Since there seems to be no error when using non-existent extras, our test-suits have started to fail with missing dependencies.

Since the tox.pytest module is part of the public interface (and for good reason, it is very very helpful in testing plugins), it would be very much appreciated if the testing extra could be added back to the publicly exposed interface of the package, too.

[gaborbernat]

I would like to keep testing dev dependency. I am open to add a testing extras though, that would only include pytest though, no?

[mbra]

Well, if there is anything in the current test dependency-group that is only really needed for running the tox testsuite itself, also listing it in the testing extra would certainly force additional dependencies on users that don't really need them, from that point of view, simply reverting to using an extra might be a bit too much and a "step back".

On the other hand it is sometimes hard to tell what exactly is required to be listed in the extra in cases where code splitting inside the code-base is sufficiently sophisticated.

In case of tox.pytest, the pytest package itself is technically a problem, though in our case we usually use pytest anyway an list it in our own dependencies anyway. The problem in "our" case is the devpi_process import from (IIRC) the devpi-server package. Technically the virtualenv module for completeness' sake, but I think with the way tox works this dependency might be handled "automatically" further down the line anyway. Also, if TYPE_CHECKING is enabled, the pytest-mock dependency also seems to be one non-standard dependency that would be required. Everything else in tox.pytest seems to be imported from included batteries or from the standard tox code-base and should not require any extra dependencies, at least as far as I can tell.

I can prepare a PR with the above dependencies included, the activity on the imports of pytest.tox does not seem to be too heavy, so I guess it would be best to just see how far maintaining this manually would get us. :-)

[gaborbernat]

I strongly disagree that this should be able to run the tox test suite. This should only be able to provide the dependencies required for tox.pytest module.

[mbra]

That was not what I was trying to suggest at all.

I am sorry if I was too verbose and did not make this obvious, but after reading my first paragraph again I am still not sure how you ended up with that impression. It is not my opinion that it should be possible to run the whole test-suite from the dependencies contained in the extra.

I am still prepared to create a PR to reintroduce a trimmed down testing extra to make tox.pytest work again for third-party plugin authors.

With that out of the way, I would be very thankful if you could spare the time to point out to me where I went wrong. When reading what I wrote earlier, my first paragraph even states that moving back everything into an extra would force users to install too much and be a step back, but maybe my wording was too convoluted.

As my last paragraph suggests to create a PR for the dependencies I gathered from tox.pytest, which I mentioned in the previous paragraph and is only a portion of the test dependency-group, I am still not sure what led you to believe it was my intent to run the whole test-suite from the extra, though. :-(

[gaborbernat]

I am still prepared to create a PR to reintroduce a trimmed down testing extra to make tox.pytest work again for third-party plugin authors.

This would be welcomed. Sorry, for missunderstandings.

[gaborbernat]

To not confuse this extra with tox test suites old testing, can we name it test instead?

[mbra]

Sure, will do.