contribution guide

[aaronspring]

edit: was initial RPSS example. Then only contrib guide

after installing pangeo-data/climpred#566 and xarray-contrib/xskillscore#277

Todo:

• North America spatial mean
• use correct distributions for bins with alignment='same_verifs'
• same after remove_bias
• North America map
• connect to cluster/client
• get weekly verifs
• plot rpss here

[judithberner]

So we are working on some issues around weekly averages, which is in our court.
As we had discussed the skill is so low, that we don't take the weekly average over the daily metrics
(e.g. rmse, rps, ...), but compute the metric over weekly anomalies (e.g. can the model predict a warm week
from a cold week). So some of our input zarr-files (on NCAR machine) might not be correct.

Also - just to be clear - we want to keep lat/lon dependence in the metrics as long as possible, so that we can show maps (as you do). The lat/lon averaging is done just before making line plots of the metrics.

Once we have solved this, here are some sanity checks.
The RPSS for T2m for ECMWF week 1-2 is around 0.01 (reference to follow), so it's positive but very small. For RPS I would expect a larger number > 1.

“The approximate relationship between RPSS and correlation being such that a RPSS value of 0.1 corresponds to a correlation of about 0.44 (Tippett et al. 2010).”

[aaronspring]

So we are working on some issues around weekly averages, which is in our court.
As we had discussed the skill is so low, that we don't take the weekly average over the daily metrics
(e.g. rmse, rps, ...), but compute the metric over weekly anomalies (e.g. can the model predict a warm week
from a cold week). So some of our input zarr-files (on NCAR machine) might not be correct.

might be related to #11 says @abjaye

[aaronspring]

on capser I have a weekly verif example. skill looks different.

[aaronspring]

wasnt clever of me to overwrite parts of the rpss notebook. maybe I just merge the contributing guide then.

[judithberner]

Just to be clear: our latest notebook (with issue 11) is here:
https://github.com/judithberner/climpred_CESM1_S2S/blob/main/all_scores_rpss.ipynb

rpss.ipynb is an older version.

[aaronspring]

thanks. then I would cleanup my notebook here and overwrite the previous rpss notebook once the machine is back online