Tests are failing on Circle because the cached test code uses an ancient fiona

[valeriupredoi]

https://app.circleci.com/pipelines/github/ESMValGroup/ESMValTool/4012/workflows/d9967e41-789d-4a98-9493-e1d113a129ea/jobs/41067/steps

Keep calm and update fiona! Don't worry about the the Circle test fails, the fiona that gets installed on Circle is 1.8.13 which is ancient, the testing suite works perfectly fine with 1.8.18 - in fact in ESMValGroup/ESMValCore#885 I have added a note about it saying we may have to switch to install it from conda in the very near future anyway

[valeriupredoi]

@bouweandela @jvegasbsc do you mind updating the cached image so we use 21st century software, gents 😁 I am working on a pure-development install test on Github Actions, so we catch testing issues that appear and are not seen by testing with the cached image 👍

[bouweandela]

The image is built from the master branch every time there is a change in it. The last build was just 4 days ago: https://hub.docker.com/r/esmvalgroup/esmvaltool/tags?page=1&ordering=last_updated

[bouweandela]

The esmvalgroup/esmvaltool:development image used for testing is built from this file: docker/Dockerfile.dev. I see it updates the conda base environment, maybe that's why it picks up an older version of fiona?

[valeriupredoi]

yes you are right @bouweandela - cheers, I have seen the issue with the very old fiona myself here - where I have implemented a full development installation and test of ESMValTool - I need to investigate more the environment. Note, however, that that install and its environment work, unlike the Circle environment that doesn't build correctly and we still have the fails bc the missing library that's asked by the old fiona, see the latest PR test https://app.circleci.com/pipelines/github/ESMValGroup/ESMValTool/4061/workflows/a502613f-d704-40a8-bf44-f932f29b4b1d/jobs/41318 - looking at the Circle tests I see the esmvalcore is off Conda /opt/conda/lib/python3.8/site-packages/esmvalcore/ so are you sure that the core is picked up and installed correctly?

[bouweandela]

I see the esmvalcore is off Conda /opt/conda/lib/python3.8/site-packages/esmvalcore/ so are you sure that the core is picked up and installed correctly?

That looks fine to me, why would that be a problem?

[valeriupredoi]

well, if the package is used off the roughly latest master then it should be pip installed and not conda installed, conda install means it's off the Conda channel and is the latest stable release that may be getting us in the old dependencies situation

[bouweandela]

We intentionally run all ESMValTool tests with the latest released version of ESMValCore. This is to provide a stable development environment for diagnostic developers.

[valeriupredoi]

OK well in that case we shouldn't use the image but rather let conda settle its own environment and download esmvalcore from the channel as a dependency; in addition I recommend using the latest master of esmvalcore in a pure dev test like I implemented in that Github Action to see the bleeding edge work fine (maybe no need for that on Circle now that we have it on GA - bump to review that PR) 👍