Incrementally tracking long climate dataset

[robertsmalcolm]

I'd like to be able to incrementally track my dataset (I may eventually have 100's years of simulation). For this the obvious way seems to be to identify eddies over a time period (e.g. 1 year), track them to produce a tracked dataset, then identify eddies over the next year, and add them to the tracked dataset. Looking at the documentation I suspect py_eddy_tracker.tracking.Correspondances may be what I need, but I cannot figure out how to use it in this way, so any guidance would be great. We plan to do this for multiple eddy-rich climate models and compare to the AVISO observational dataset. Thanks.

[AntSimi]

I just add a short introduction here:
https://py-eddy-tracker.readthedocs.io/en/latest/python_module/08_tracking_manipulation/pet_how_to_use_correspondances.html#sphx-glr-python-module-08-tracking-manipulation-pet-how-to-use-correspondances-py

[AntSimi]

To reduce storage and tracking cost you could also play with sampling option in eddy_identification, default value is 50 and could be set between 20 to 30.
This example show how visvalingam work:
https://py-eddy-tracker.readthedocs.io/en/latest/python_module/14_generic_tools/pet_visvalingam.html#sphx-glr-python-module-14-generic-tools-pet-visvalingam-py

[robertsmalcolm]

Thanks, I may do that. A further question - if I do the tracking in python, and use the Correspondances function (which is working well and saves a lot of time in the tracking), how do I create the four output files as produced when using the yaml method, e.g. EddyTracking conf.yaml -v DEBUG. I'd like to use TrackEddiesObservations.load_file to read in the tracked eddies (as in your examples), but this does not work with the correspondances file as input. Thanks again.

[AntSimi]

First Tracking file and Correspondances are two differents object. TrackEddiesObservations.load_file must use only for atlas (tracked eddies)

If you want create three atlas (ie untracked/short/long) you could take a look at this code. It's EddyTracking code. If it's too hard to understand, i will add comment in code.

[robertsmalcolm]

This works fantastically, thank you very much :)). So now the tracking runs fast with Correspondances, and I can use the code for analysis so I am very happy. One last question - I'd like to exactly mirror the identification/tracking used on AVISO (that you use to generate your obs datasets) for our models. I think I understand the pre-processing and the identification steps from https://www.aviso.altimetry.fr/en/data/products/value-added-products/global-mesoscale-eddy-trajectory-product.html. For the tracking, do I just use the default method (that is. no specification of class_method in the Correspondances call), with virtual=4? Thanks again.

[AntSimi]

Several product are described in this page, Meta3.1 is keep only for hitorical reason, but don't use it.
META 3.2 are described in this paper.
To create something close to META3.2
Identification parameters:

• filter adt with 700km order 1 bessel_high_filter method
• step every 2 mm
• maximum shape error 70%, Exemple of different shape error
• pixel_limit must be set with care of your own resolution product
• nb_step_to_be_mle = 0 => don't allow micro relief in an eddy

Tracking parameters:

• use AreaTracker from from py_eddy_tracker.featured_tracking.area_tracker import AreaTracker work like in this figure
  • with cmin = 0.05 (5% of overlap) but it could be interessant to use hybrid feature available in match method
• virtual set to 4