Use mamba + pytest-xdist to speed up CI testing

[andersy005]

Description

This PR attempts to speed the CI setup by using mamba for conda environment creation.

To-Do List

Feel free to add a checklist of steps to be performed while you are working through creating this PR.

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update
• Performance (if you touched existing code run asv to detect performance changes)
• Refactoring

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. This could point to a cell in the updated notebooks. Or a snippet of code with accompanying figures here.

Checklist (while developing)

• I have added docstrings to all new functions.
• I have commented my code, particularly in hard-to-understand areas
• Tests added for pytest, if necessary.
• I have updated the sphinx documentation, if necessary.
• Any new functions are added to the API. (See contribution guide)
• CHANGELOG is updated with reference to this PR.

Pre-Merge Checklist (final steps)

• I have rebased onto master or develop (wherever I am merging) and dealt with any conflicts.
• I have squashed commits to a reasonable amount, and force-pushed the squashed commits.

References

Please add any references to manuscripts, textbooks, etc.

[andersy005]

Python 3.8

• Without mamba
  

• With mamba
  

Python 3.7

• Without mamba
  

• With mamba
  

[bradyrx]

Wow, that's a great speedup! Makes for quicker development. Always rough when you're waiting on tests to run. Just re-triggered testing since one failed due to pulling data from our climpred-data repo, which just happens sometimes.

[aaronspring]

Locally I remembered that it wasn't too easy to see which test failed and the corresponding error message when using xdist. Do I remember correctly?

[bradyrx]

Locally I remembered that it wasn't too easy to see which test failed and the corresponding error message when using xdist. Do I remember correctly?

I don't recall trying it honestly. Good thing to look for. Maybe we should force a few tests from different modules to fail, once this is set up and passing.

[bradyrx]

It looks like potentially a pytest-xdist issue with trying to download data? We have a few tests that pull from our climpred-data server and then cache it locally.

[andersy005]

it wasn't too easy to see which test failed and the corresponding error message when using xdist.

I just added the -rf flag when running pytest: pytest -rf. Let me know if the produced output is useful.

It looks like potentially a pytest-xdist issue with trying to download data? We have a few tests that pull from our climpred-data server and then cache it locally.

Sounds like we have a race condition in

climpred/climpred/tutorial.py

Line 102 in 4ce5e3d

def load_dataset(

I will look into this, and hopefully I will find a fix for this issue.

[bradyrx]

@andersy005, maybe there is an issue with multiple instances of the parallel pytest trying to access the FTP/data server at the same time? Or with both trying to cache it? Maybe try turning off cache. I think we have a switch for that in load_dataset.

[bradyrx]

Getting closer. Only 3.6 failed this time 😂 . I just re-ran the tests once more..

[bradyrx]

Might be worth just using mamba for the speedup if there's issues with pytest-xdist accessing cached copies of the tutorial data at the same time. But still open to this if you find a solution!

[andersy005]

Getting closer. Only 3.6 failed this time 😂 . I just re-ran the tests once more..

Hehehe.... I disabled pytest-xdist for the time being... Hopefully, we'll figure out a solution in the near future...

[bradyrx]

@andersy005 just one question about xesmf/esmpy. Agreed to stick with just mamba now. Thanks for this PR!

[bradyrx]

Thanks @andersy005! Lots of solid additions to the CI.