Obs data for recipe_anav13jclim.yml

[pabretonniere]

Hello,

I'm trying to run the recipe "recipe_anav13jclim.yml" and I have some issues trying to collect all the observational data needed.

Without redownloading everything from each individual data source, I imagine somebody gathered the data somewhere to test the recipe. Would it be possible to point me to a place where I could obtain the data (cmorized or not)? Else, if you have the list of access points from where to download each dataset, that would also be appreciated.

@mattiarighi @schlunma : I see you are the authors of https://github.com/ESMValGroup/ESMValTool/blob/90eaba9156b5f13d98ba2bc206b1e80277e1b025/doc/sphinx/source/recipes/recipe_anav13jclim.rst . Would you be able to help me with this?

Thank you very much in advance,
Best regards,

[bouweandela]

Are you running the recipe on your own computer, or on a scientific compute cluster?

Note that we also have some work in progress to make it easier to download observational datasets: #1657

[pabretonniere]

Thanks for your prompt reply! I'm working on a scientific cluster.

Actually I'm working together with @jvegasbsc who just pointed me to https://github.com/ESMValGroup/ESMValTool/tree/master/esmvaltool/cmorizers/obs in which there seems to be indication on how to download most of the observations used in this recipe. So now that I have more detailed instructions on each sources in the comments of the different python scripts. I'll do the download by hand from there. I was aware of #1657 but this doesn't seem to be exactly what I needed yet.

Thank you.

[bouweandela]

Which cluster are you on? We have a common pool of downloaded data available on a few systems @valeriupredoi, @mattiarighi or @bjoernbroetz might be able to tell you more about that.

[bouweandela]

Documentation on how to run the cmorizer scripts can be found here: https://esmvaltool.readthedocs.io/en/latest/input.html#using-a-cmorizer-script, but indeed information on how to download is still missing from the documentation on readthedocs #1219.

[pabretonniere]

Apart from the BSC facilities, I have access to Jasmin for example.

[bouweandela]

Great, in that case @valeriupredoi should be able to help you out!

[pabretonniere]

Thanks! I got access the esm(e)val workspace on Jasmin (/group_workspaces/jasmin4/esmeval/obsdata-v2) on Friday but could only find the data from CRU and HADISST, so I started contacting the different providers, based on the information from https://github.com/ESMValGroup/ESMValTool/tree/master/esmvaltool/cmorizers/obs but if the whole data is available somewhere on Jasmin, I'll be very happy to access them from there.

[bouweandela]

Ping @valeriupredoi Could you please have a look at this?

[remi-kazeroni]

Hi @pabretonniere, is the OBS data access still an issue? I see that the recipe_anav13jclim.yml requires quite a few different OBS datasets. While the Tier2 should be available on Jasmin, the Tier3 ones are not. And several of the Tier3 datasets unfortunately do not have an auto-downloader script so the new CMORizer interface is not solving your issue fully. Please let us know how we can help or if this issue can be closed. Thanks 👍

[pabretonniere]

Hi @remi-kazeroni

From what I remember, I found a way to download the datasets "manually" contacting the different providers. Nothing very fancy but I managed to get the data, pseudo cmorize them and run the recipe.

We can close the issue.

Thanks