Understanding how rolling_window_statistics works

[mo-abodas]

Dear ESMValTool community, I've tried to use the rolling_window_statistics preprocessor, but I'm not confident that I understand what's doing. I've compared the outputs of using cube.rolling_window('time', iris.analysis.MEAN, 11) with this preprocessor

preprocessors:
  running_global_mean:
    annual_statistics:
      operator: mean
    area_statistics:
      operator: mean

and the outputs from this preprocessor:

preprocessors:
  running_global_mean:
    annual_statistics:
      operator: mean
    area_statistics:
      operator: mean
    rolling_window_statistics:
      coordinate: time
      operator: mean
      window_length: 11

These figures show the annual means and the outputs when a rolling window is used as explained above:

The bottom plot shows a reduction in the high frequency variance as I'd expect, whereas the impact in the middle figure (using the rolling_window preprocessor) is not as noticeable.
Any help understanding this greatly appreciated, Thanks!

[malininae]

Hi @mo-abodas, I was the person who wrote this preprocessor, and I think I know where your issue is coming from. The preprocessor uses the time stamps of the input cube. I'm assuming, you are using monthly data to generate the cube. Now, in the default order of the preprocessor functions (which you are using) rolling_window_statistics comes before annual_statistics and area_statistics, so you basically end up doing the rolling_window_statitics on the monthly data, which is "ish" equivalent to the annual statistics and that's why you don't see much difference. At the lower plot, you are using rolling_window_statistics on the annual data, so you are getting the desired smoothing. You can obtain the low plot output by simply adding custom_oder: True in you preprocessor, this way you are telling ESMValTool to use the preprocessing functions in the order you want it and not how it is set up by default. Your final preprocessor would look like that:

preprocessors:
  running_global_mean:
    custom_order: True
    annual_statistics:
      operator: mean
    area_statistics:
      operator: mean
    rolling_window_statistics:
      coordinate: time
      operator: mean
      window_length: 11

Hope it solves your issue, if not, please let me know!

[mo-abodas]

@malininae thank you very much for your quick and brilliant answer! Now it works as expected.