DAPPER is a set of templates for benchmarking the performance of data assimilation (DA) methods. The tests provide experimental support and guidance for new developments in DA. Example diagnostics:
The typical set-up is a twin experiment, where you
- specify a
- dynamic model*
- observational model*
- use these to generate a synthetic
- "truth"
- and observations thereof*
- assess how different DA methods perform in estimating the truth, given the above starred (*) items.
DAPPER enables the numerical investigation of DA methods through a variety of typical test cases and statistics. It (a) reproduces numerical benchmarks results reported in the literature, and (b) facilitates comparative studies, thus promoting the (a) reliability and (b) relevance of the results. DAPPER is (c) open source, written in Python, and (d) focuses on readability; this promotes the (c) reproduction and (d) dissemination of the underlying science, and makes it easy to adapt and extend. In summary, it is well suited for teaching and fundamental DA research. Also see its drawbacks.
Works on Linux/Windows/Mac.
-
Prerequisite: python>=3.6.
If you're not {admin | expert}:
1a. Install it with Anaconda.
1b. Use the Anaconda terminal to run the commands below. -
Install:
Download and extract (orgit clone) DAPPER,
cdinto the resulting folder (ensure you're at the level with asetup.pyfile):
pip install -e .(don't forget the.). -
Test by running:
python example_1.py
Step 2 can be replaced by running
pip install da-dapper
but this is not recommended since this hides away DAPPER as a library in your python path.
If the installation fails, you probably need to create a new Python environment.
Read, run, and understand the scripts example_{1,2,3}.py.
Then, get familiar with the code.
The docs provide processed docstrings, but are far from complete.
Alternatively, see DA-tutorials for an intro to DA.
| Method | Literature reproduced |
|---|---|
| EnKF 1 | Sak08, Hot15 |
| EnKF-N | Boc12, Boc15 |
| EnKS, EnRTS | Raa16b |
| iEnKS / iEnKF / EnRML / ES-MDA 2 | Sak12, Boc12, Boc14 |
| LETKF, local & serial EAKF | Boc11 |
| Sqrt. model noise methods | Raa15 |
| Particle filter (bootstrap) 3 | Boc10 |
| Optimal/implicit Particle filter 3 | Boc10 |
| NETF | Töd15, Wil16 |
| Rank histogram filter (RHF) | And10 |
| 4D-Var | |
| 3D-Var | |
| Extended KF | |
| Optimal interpolation | |
| Climatology |
1: Stochastic, DEnKF (i.e. half-update), ETKF (i.e. sym. sqrt.).
Serial forms are also available.
Tuned with inflation and "random, orthogonal rotations".
2: Also supports the bundle version, and "EnKF-N"-type inflation.
3: Resampling: multinomial (including systematic/universal and residual).
The particle filter is tuned with "effective-N monitoring", "regularization/jittering" strength, and more.
| Model | Lin? | TLM? | PDE? | Phys.dim. | State len | Lyap≥0 | Implementer |
|---|---|---|---|---|---|---|---|
| Linear Advect. (LA) | Yes | Yes | Yes | 1d | 1000 * | 51 | Evensen/Raanes |
| DoublePendulum | No | Yes | No | 0d | 4 | 2 | Matplotlib/Raanes |
| LotkaVolterra | No | Yes | No | 0d | 5 * | 1 | Wikipedia/Raanes |
| Lorenz63 | No | Yes | "Yes" | 0d | 3 | 2 | Sakov |
| Lorenz84 | No | Yes | No | 0d | 3 | 2 | Raanes |
| Lorenz95 | No | Yes | No | 1d | 40 * | 13 | Raanes |
| LorenzUV | No | Yes | No | 2x 1d | 256 + 8 * | ≈60 | Raanes |
| Kuramoto-Sivashinsky | No | Yes | Yes | 1d | 128 * | 11 | Kassam/Raanes |
| Quasi-Geost (QG) | No | No | Yes | 2d | 129²≈17k | ≈140 | Sakov |
*: flexible; set as necessary
As mentioned above, DAPPER reproduces literature results. There are also plenty of results in the literature that DAPPER does not reproduce. Typically, this means that the published results are incorrect.
A list of experimental settings that can be compared with literature papers
can be obtained using gnu's find:
$ find . -iname "[a-z]*[0-9].py" | grep mods
Some of these files contain settings that have been used in several papers.
DAPPER is aimed at research and teaching (see discussion on top). Example of limitations:
- It is not suited for very big models (>60k unknowns).
- Time-dependent error covariances and changes in lengths of state/obs (although the Dyn and Obs models may otherwise be time-dependent).
- Non-uniform time sequences not fully supported.
Also, DAPPER comes with no guarantees/support. Therefore, if you have an operational (real-world) application, such as WRF, you should look into one of the alternatives, sorted by approximate project size.
| Name | Developers | Purpose (approximately) |
|---|---|---|
| DART | NCAR | Operational, general |
| PDAF | AWI | Operational, general |
| JEDI | JCSDA (NOAA, NASA, ++) | Operational, general (in develpmt?) |
| ERT | Statoil | Operational, history matching (Petroleum) |
| OpenDA | TU Delft | Operational, general |
| Verdandi | INRIA | Biophysical DA |
| PyOSSE | Edinburgh, Reading | Earth-observation DA |
| SANGOMA | Conglomerate* | Unify DA research |
| EMPIRE | Reading (Met) | Research (high-dim) |
| MIKE | DHI | Oceanographic. Commercial? |
| OAK | Liège | Oceaonagraphic |
| Siroco | OMP | Oceaonagraphic |
| FilterPy | R. Labbe | Engineering, general intro to Kalman filter |
| DASoftware | Yue Li, Stanford | Matlab, large-scale |
| Pomp | U of Michigan | R, general state-estimation |
| PyIT | CIPR | Real-world petroleum DA (?) |
| Datum | Raanes | Matlab, personal publications |
| EnKF-Matlab | Sakov | Matlab, personal publications and intro |
| EnKF-C | Sakov | C, light-weight EnKF, off-line |
| IEnKS code | Bocquet | Python, personal publications |
| pyda | Hickman | Python, personal publications |
The EnKF-Matlab and IEnKS codes have been inspirational in the development of DAPPER.
*: AWI/Liege/CNRS/NERSC/Reading/Delft
Patrick N. Raanes, Colin Grudzien, Maxime Tondeur, Remy Dubois
If you use this software in a publication, please cite as follows.
@misc{raanes2018dapper,
author = {Patrick N. Raanes and others},
title = {nansencenter/DAPPER: Version 0.8},
month = December,
year = 2018,
doi = {10.5281/zenodo.2029296},
url = {https://doi.org/10.5281/zenodo.2029296}
}- https://www.geosci-model-dev-discuss.net/gmd-2019-136/
- https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.3386
- https://www.nonlin-processes-geophys-discuss.net/npg-2019-10
