Dependencies: update requirement pytest~=6.0 and use pyproject.toml

[sphuber]

Fixes #4340

Starting from v6.0, pytest supports using the pyproject.toml instead
of a pytest.ini to define its configuration. Given that this is
quickly becoming the Python packaging standard and allows us to reduce
the number of configuration files in the top level of the repository, we
increase the version requirement of pytest.

[csadorf]

Looks like there are some issues with this update since the CI tests are failing: https://github.com/aiidateam/aiida-core/pull/4410/checks?check_run_id=1172693876

Did you already have a look at this? Is this a pytest incompatibility?

[sphuber]

It seems to be a problem with pylint-rerunfailures: pytest-dev/pytest-rerunfailures#128 . This is only a problem for pytest==6.1. We can either limit pytest<6.1, wait for a fix of pytest-rerunfailures or hold off with this PR.

[csadorf]

It seems to be a problem with pylint-rerunfailures: pytest-dev/pytest-rerunfailures#128 . This is only a problem for pytest==6.1. We can either limit pytest<6.1, wait for a fix of pytest-rerunfailures or hold off with this PR.

I believe that rerunfailures was not actually that successful in resolving some of the issues it was supposed to address. I would recommend to revert the related commit. @ltalirz Could you comment?

[ltalirz]

I believe that rerunfailures was not actually that successful in resolving some of the issues it was supposed to address.

I believe it resolves some issues but not all - some tests will continue to fail even if you run them N times in a row since the failure is related to some hidden state in the environment.

I don't have time now, but it would be interesting to gather some statistics from the travis logs on how often flaky tests are being successfully rerun by rerunfailures (it's reported in the output).

I would suggest to keep it and limit the pytest version instead until the mentioned issue is fixed (unless we really need pytest v6.1)

[greschd]

FYI, pytest-rerunfailures has released 9.1.1 which supposedly fixes the problem: https://pypi.org/project/pytest-rerunfailures/#history

[sphuber]

FYI, pytest-rerunfailures has released 9.1.1 which supposedly fixes the problem: https://pypi.org/project/pytest-rerunfailures/#history

Good call, I actually just updated the PR to put an upper limit. I even checked the issue but somehow didn't notice it had been closed 🤦‍♂️ Will revert the change

[codecov]

Codecov Report

Merging #4410 into develop will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop    #4410   +/-   ##
========================================
  Coverage    79.22%   79.22%           
========================================
  Files          475      475           
  Lines        34826    34826           
========================================
  Hits         27589    27589           
  Misses        7237     7237           

Flag	Coverage Δ
#django	73.07% <ø> (ø)
#sqlalchemy	72.30% <ø> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9144924...456115a. Read the comment docs.

[sphuber]

@csadorf this is ready for review. pytest-rerunfailures released 9.1.1 with a fix, which is now added as a minimum requirement