Parallelize the unit tests

[woodruffw]

Opening this o/b/o @DarkaMaul from @miketheman's original work.

On my local machine, this results in a ~2 speedup in testing: from roughly 2 minutes for the entire suite to 1. More speedups may be possible, e.g. in stand-up and coverage merging.

Closes #16178.

[woodruffw]

NB: This file could probably live somewhere other than the repo root, as long as it gets copied into the right place at container time.

[miketheman]

🤔 Since this file is being mounted in docker-compose.yml , we could place it in dev/ and mount it from there, but that would skip it from being run in CI.
I'm not opposed to leaving it where it is for now, but wanted to call that possibility out.

[woodruffw]

Looks like the speedup isn't as significant on CI, which is interesting.

Edit: For comparison, a random test run before this PR: https://github.com/pypi/warehouse/actions/runs/9762910094/job/26947742230

...and with this PR: https://github.com/pypi/warehouse/actions/runs/9762940705/job/26947794957?pr=16206

So 4m30s to 2m58s.

[miketheman]

Very cool, thanks for figuring out all the pieces to this puzzle.

On my 12-core machine, drops the tests runtime from ~83 to ~41 seconds - that's about 50% speed up 🐎

[miketheman]

🤔 Since this file is being mounted in docker-compose.yml , we could place it in dev/ and mount it from there, but that would skip it from being run in CI.
I'm not opposed to leaving it where it is for now, but wanted to call that possibility out.

[di]

Is there maybe a way to locally disable this? For whatever reason, starting up the xdist workers on my machine made the test suite run take longer than it did before this change. I'm also missing the output which shows which test files failed as the test suite runs, as they're no longer displayed.

[woodruffw]

Is there maybe a way to locally disable this?

This should disable parallelization (by forcing everything onto one worker): make tests TESTARGS='-n 1'. On my local machine that spawns only one worker, at least.

I'll look into improving the failure render as well.

Edit: For improving the render you could pass TESTARGS='-v', although that makes the report output much more verbose than the default non-xdist output. Another option here would be to add pytest-pretty for report generation, which apparently works well with pytest-xdist. I can send a PR demonstrating that.

[di]

FWIW, looks like -n 0 fully disables pytest-xdist. Using -n 1 for me was still just as slow.