xtclim

ML-based extreme events detection and characterization (CERFACS)

The code is adapted from CERFACS' repository. The implementation of a pipeline with itwinai framework is shown below.

Method

Convolutional Variational AutoEncoder.

Input

"3D daily images", daily screenshots of Europe for three climate variables (maximum temperature, precipitation, wind).

Output

Error between original and reconstructed image: postprocessed for analysis in the scenario_season_comparison.ipynb file.

Idea

The more unusual an image (anomaly), the higher error.

Information on files

In the preprocessing folder, the preprocess_functions_2d_ssp.py class loads NetCDF files from a data folder, which has to be specified in dataset_root in the config file config.yaml (please change the location). The data can be found here. The given class normalizes and adjusts the data for the network. The function preprocess_2d_seasons.py splits the data into seasonal files. Preprocessed data is stored in the input folder.

The file train.py trains the network. Caution: It will overwrite the weights of the network already saved in outputs (unless you change the path name outputs/cvae_model_3d.pth in the script).

The anomaly.py file evaluates the network on the available datasets - train, test, and projection.

How to launch training workflow

The config file config.yaml contains all the steps to execute the workflow. You can launch it from the root of the repository with:

itwinai exec-pipeline --config config.yaml

To dynamically override some (nested) fields from terminal you can do:

itwinai exec-pipeline --config config.yaml \
    -o GENERAL.dataset_root=/path/to/data \
    -o GENERAL.input_path=input \
    -o GENERAL.output_path=output

To run only some steps, e.g., only training step after the training dataset has been generated, use:

itwinai exec-pipeline --config config.yaml --steps training-step

TODOs

Integration of post-processing step + distributed strategies