Test failures with Matplotlib 3.5.0b1

[QuLogic]

Building with Matplotlib 3.5.0b1 fails on two tests:

_______________________________________________________________________________________________________________________________________________________ test_gaussian ________________________________________________________________________________________________________________________________________________________
Error: Image files did not match.
  RMS Value: 21.09554109679737
  Expected:  
    /tmp/tmpmwx2uric/test_gaussian.test_gaussian/baseline.png
  Actual:    
    /tmp/tmpmwx2uric/test_gaussian.test_gaussian/result.png
  Difference:
    /tmp/tmpmwx2uric/test_gaussian.test_gaussian/result-failed-diff.png
  Tolerance: 
    20
______________________________________________________________________________________________________________________________________________________ test_reglatlon1 _______________________________________________________________________________________________________________________________________________________
Error: Image files did not match.
  RMS Value: 30.061142323234787
  Expected:  
    /tmp/tmpmwx2uric/test_reglatlon.test_reglatlon1/baseline.png
  Actual:    
    /tmp/tmpmwx2uric/test_reglatlon.test_reglatlon1/result.png
  Difference:
    /tmp/tmpmwx2uric/test_reglatlon.test_reglatlon1/result-failed-diff.png
  Tolerance: 
    20

[QuLogic]

So the problem is that the coast lines 'disappeared'. That's because they have zorder=1 in Cartopy, and contours also have zorder=1 in Matplotlib. With 3.5.0b1, Matplotlib respects plotting order and no longer arbitrarily re-orders artists (matplotlib/matplotlib#18216), so the coast lines are drawn under the contour. The quick fix would be to move the ax.coastlines call later.

I do wonder if we should do something about this in Cartopy and/or Matplotlib.

[jdkloe]

I just encountered the same issue. Simply adding zorder=2 to the ax.coastlines() calls solves it.
I modified 1 line in pygrib-2.1.3/test/test_gaussian.py and 3 lines in pygrib-2.1.3/test/test_reglatlon.py. Then all runs again fine with matplotlib 3.5.0.

[QuLogic]

Yes, that will work; we are also fixing this in Cartopy: SciTools/cartopy#1840

[QuLogic]

Latest Cartopy fixes this for you, but it also changes the results slightly. The limits are a little better and higher, and the new gridline labelling changes the +/- to N/S or E/W.

[jdkloe]

New test results from the latest fedora rawhide rebuild look different, but I still see failing tests related to cartopy plots created by the pygrib test suite that do not match expectations. See the attached file:
pygrib_test.tar.gz
I see 3 failing tests now:
test_reduced_gg.test_reduced_gg
test_reduced_ll.test_reduced_ll
test_reglatlon.test_reglatlon2
All 3 seem to have a changed bounding box, especially on the upper side.
For test_reduced_gg the tick label texts changed, they now have added N/S/E/W and an added circle/degree symbol.

Any advice how to change/fix the tests to prevent these failures are welcome.

Software versions used for this test are:
Python 3.10.2, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
Matplotlib: 3.5.1
Freetype: 2.11.1
Cartopy 0.20.1

[QuLogic]

If you really don't want to change them, I think you can just set the limits of the plot.