DEV: Use pytest-xdist

[MartinThoma]

Before (example):

After (this PR):

and a second run:

[codecov]

Codecov Report

All modified lines are covered by tests ✅

Comparison is base (448c379) 94.44% compared to head (792ea4b) 94.44%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2254   +/-   ##
=======================================
  Coverage   94.44%   94.44%           
=======================================
  Files          43       43           
  Lines        7634     7634           
  Branches     1506     1506           
=======================================
  Hits         7210     7210           
  Misses        262      262           
  Partials      162      162           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[MartinThoma]

It still seems not to combine the coverage data correctly. I guess the reason is a race-condition between the different python versions.

[stefan6419846]

This appears to be a naming issue: Looking at https://github.com/py-pdf/pypdf/actions/runs/6516162435/job/17699481914, we have

Run actions/download-artifact@v3
Starting download for coverage-data
Directory structure has been setup for the artifact
Total number of files that will be downloaded: 7
Total file count: 7 ---- Processed file #2 (28.5%)
Total file count: 7 ---- Processed file #6 (85.7%)
Artifact coverage-data was downloaded to /home/runner/work/pypdf/pypdf
Artifact download has finished successfully

Run python -m coverage combine
Combined data file .coverage.fv-az460-300.3519.834511
Combined data file .coverage.fv-az670-727.3495.027472
Combined data file .coverage.fv-az296-65.3484.685834
Combined data file .coverage.fv-az457-547.3483.149504
Combined data file .coverage.fv-az615-884.4343.623930
Combined data file .coverage.fv-az137-215.3683.313624
Combined data file .coverage.fv-az956-684.3600.108483
Wrote XML report to coverage.xml

For https://github.com/py-pdf/pypdf/actions/runs/6523292132/job/17713841210, we get

Run actions/download-artifact@v3
Starting download for coverage-data
Directory structure has been setup for the artifact
Total number of files that will be downloaded: 1
Artifact coverage-data was downloaded to /home/runner/work/pypdf/pypdf
Artifact download has finished successfully

Run python -m coverage xml
Wrote XML report to coverage.xml

instead.

Possible solution: Add prefix or suffix to the coverage database before uploading the artifact to ensure unique names instead of one common name and thus the files overwriting theirselves.

[MartinThoma]

@stefan6419846 I think I fixed the issue :-)

What do you think about the overall PR?

It seems to reduce the CI wall-clock time from ~14min 23s to 9min 20s.

[stefan6419846]

While in general the randomness source should be sufficient enough, I am not sure whether we can use a more stable approach there (the current Python version as given by the matrix?) or at least ensure that when combining the coverage/downloading the artifacts we actually have the correct number of files.

As for the runtime: As it does not have any apparent side effects and it does not change anything relevant on the remaining test setup, this at least is some enhancement. Reducing the runtime by roughly 1/3 while using two parallel executors seems to be a little bit worse than expected, but it does not take into account the pure tests runtime, but the general setup, and thus this might be as good as we can get without adding more parallel executors.