get_bc_ppe_data not accessible when installing from pip/conda

[alextsfox]

It looks like a number of functions used in the example notebooks (Emulating using CNNs, Emulating using GPs, etc) such as get_bc_ppe_data() and get_crm_data() are kept in docs/examples/utils.py rather than esem/utils.py, which is a little confusing since this isn't installed when using pip/conda.

Also, I'm personally having trouble opening the files that these functions try to grab from https://zenodo.org/record/3856645/. I get the error

"OSError: [Errno -101] NetCDF: HDF error",

followed by

"ProductPluginException: An error occurred retrieving data using the product NetCDF_Gridded. Check that this is the correct product plugin for your chosen data. Exception was OSError: -101."

I thought this might have something to do with HDF5 file locking, but setting HDF5_USE_FILE_LOCKING=FALSE and
HDF5_DISABLE_VERSION_CHECK=1 doesn't seem to change anything.

Does anyone have any input on these two things?

Thanks

[duncanwp]

Hi Alex, thanks for the feedback!

The docs/examples utils.py was really just intended for running the example notebooks rather than a core functionality. I take your point though and would be keen to hear how other users feel about this.

Either way you should be able to read the data from zenodo. Could you share the code you use to read it and the full stack-trace you get?

Cheers

[alextsfox]

Okay, well this seems to be one of those errors that just goes away when someone asks for proof. Having changed nothing in my code, get_bc_ppe_data() seems to work absolutely fine now, so unfortunately the only record I have is the names of the exceptions above. I swear I was trying to solve this for over an hour last week, and couldn't get anything to work. Maybe I just had to turn it off and on again. Sorry to disappoint...

[duncanwp]

No worries - glad it's working now anyway!