Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v1 release #107

Merged
merged 103 commits into from
Jul 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
103 commits
Select commit Hold shift + click to select a range
fad80c4
Scripts for table 1
jasonjewik May 21, 2023
97525ac
Fixed typos in table1.py
prakhar6sharma May 22, 2023
c52b84c
Temp fix
prakhar6sharma May 22, 2023
e52e18a
iter module working with new refactor
prakhar6sharma May 23, 2023
454a6e9
Checkpoint
jasonjewik May 23, 2023
c4e6ff0
Merge branch 'iterhotfix' into jason-experiments
jasonjewik May 23, 2023
9097a0d
Update table 1 code
jasonjewik May 23, 2023
5ab3733
Adding table experiments for other architectures
jasonjewik May 23, 2023
42323c6
Update ViT architecture
jasonjewik May 23, 2023
f4d599d
Add notebook
jasonjewik May 24, 2023
e2431bd
Checkpoint
jasonjewik May 24, 2023
7ee312a
Checkpointing
jasonjewik May 24, 2023
26b6f15
Updated log dir for checkpoints
prakhar6sharma May 25, 2023
8dac45c
Checkpoint
jasonjewik May 25, 2023
830a1b9
Fix mse, rmse, and acc metrics
jasonjewik May 26, 2023
3be7777
Update acc code
jasonjewik May 26, 2023
8c0af7d
Reduce precision, change early stopping metric
jasonjewik May 26, 2023
bd8c4ac
Fixes
jasonjewik May 26, 2023
ce7f742
add lr monitor, modify checkpointing, use ddp instead of ddp_spawn
tung-nd May 26, 2023
d680ffc
modify logging keys to separate train/val/test
tung-nd May 26, 2023
0c92978
use normalized lat_mse as a validation metric
tung-nd May 26, 2023
635e342
change training setting
tung-nd May 26, 2023
d7460ce
Checkpoint
jasonjewik May 26, 2023
1d6d349
Downscaling
jasonjewik May 28, 2023
9ea1596
DL models for downscaling
jasonjewik May 28, 2023
f438229
Pearson now does not rely on scipy
jasonjewik May 28, 2023
1a76a30
ViT experiment and PRISM support
jasonjewik May 28, 2023
ed7f82b
Multiple transformation bug fix
jasonjewik May 29, 2023
116f36a
fix duplicate denormalization bug
tung-nd May 29, 2023
f520ead
add ReduceLROnPlateau scheduler
tung-nd May 29, 2023
cb80d74
update gitignore
tung-nd May 29, 2023
08a129e
fix resnet to match with weatherbench implementation
tung-nd May 30, 2023
7966523
current best setting
tung-nd May 30, 2023
7ec0ad0
update projection and prediction layers for unet
tung-nd May 30, 2023
b52122a
update run scripts for resnet and unet
tung-nd May 30, 2023
c1ce550
avoid spiking loss
tung-nd May 31, 2023
8b5e37c
update vit models and scripts
tung-nd May 31, 2023
0de7344
Downscaling exp changes
jasonjewik May 31, 2023
6432325
Downscaling changes
jasonjewik May 31, 2023
7fbe021
rollback resnet and unet architectures
tung-nd May 31, 2023
e0d1b0c
Merge Tung's changes
jasonjewik May 31, 2023
7335b5e
Downscaling updates
jasonjewik Jun 1, 2023
73d9382
use 16 precision
tung-nd Jun 1, 2023
543eb8c
CMIP6 running with iterdatamodule
prakhar6sharma Jun 1, 2023
402afec
add continuous data modules
tung-nd Jun 2, 2023
d9027d4
update nc2npz to accumulate precipitation data
tung-nd Jun 2, 2023
5047224
merge tung exps (#105)
prakhar6sharma Jun 2, 2023
b31ff20
ERA5 downscaling updates
jasonjewik Jun 2, 2023
1cf32b1
use 1 worker, somehone increasing # workers affects performance
tung-nd Jun 2, 2023
66cbd77
update scripts to be consistent
tung-nd Jun 3, 2023
2d5cf8e
prepare iterative exps
tung-nd Jun 3, 2023
b8c3887
Downscaling checkpoint: PRISM implemented
jasonjewik Jun 4, 2023
2d5a1f4
evaluate iterative models
tung-nd Jun 5, 2023
23f040c
summarize results
tung-nd Jun 5, 2023
b50c8a9
update json results and add plot scripts
tung-nd Jun 5, 2023
73e3e3b
Add probabilistic metrics
jasonjewik Jun 11, 2023
aca9b8c
Add class metrics
jasonjewik Jun 11, 2023
8cecb86
Add rank histogram visualization
jasonjewik Jun 11, 2023
b78f111
update plot script
tung-nd Jun 12, 2023
9e9b0ff
add climatebench experiments
tung-nd Jun 13, 2023
bdeabb2
Resolve merge conflicts
jasonjewik Jun 18, 2023
efe7f88
Merge branch 'tung_exps' into v1-release
jasonjewik Jun 18, 2023
13b5fed
Disable all data tests
jasonjewik Jun 18, 2023
3284421
Formatting
jasonjewik Jun 18, 2023
17f8cbb
Fix trainer
jasonjewik Jun 18, 2023
53f0a9f
Cleanup files
jasonjewik Jun 18, 2023
045c59f
Moved processing scripts, added PRISM processing
jasonjewik Jun 18, 2023
f9ee0c5
Cleanup data downloading
jasonjewik Jun 20, 2023
2978346
Add CLI utility for downloads, processing scripts
jasonjewik Jun 20, 2023
c12caf6
Combine iter modules
jasonjewik Jun 21, 2023
31da4e7
Add data modules
jasonjewik Jun 21, 2023
ee95af9
Reorganize files
jasonjewik Jun 21, 2023
2aa61c3
Remove unused notebooks
jasonjewik Jun 21, 2023
065e6a7
Cleaning utilities
jasonjewik Jun 21, 2023
b9af64f
Add visualizations for bias and mean bias
jasonjewik Jun 21, 2023
97cde8a
Add rank histogram visualization
jasonjewik Jun 22, 2023
1d04977
Organize experiments, add ERA5 crop script
jasonjewik Jun 23, 2023
b473aea
Clean up downscaling and climate projection
jasonjewik Jun 23, 2023
c817b49
Add forecasting baselines
jasonjewik Jun 23, 2023
486db13
Cleanup forecasting experiments
jasonjewik Jun 23, 2023
755ae4a
Removed unneeded tables
jasonjewik Jun 23, 2023
4c163ab
Add climate projection experiments
jasonjewik Jun 26, 2023
694b2ca
Iterative forecasting experiments
jasonjewik Jun 26, 2023
5893a26
Iterative forecasting experiments
jasonjewik Jun 26, 2023
647bc49
Add extreme ERA5
jasonjewik Jun 28, 2023
02470a6
Changes for quickstart notebook
jasonjewik Jun 28, 2023
0ae1ce6
Update notebooks
jasonjewik Jun 28, 2023
813d0cb
Formatting
jasonjewik Jun 28, 2023
c0c32d8
Update era5_constants.py
jasonjewik Jun 29, 2023
8b090ee
Update cmip6_constants.py
jasonjewik Jun 29, 2023
d3a9896
Add units for visualizations
jasonjewik Jun 29, 2023
e73a031
Updates for notebook
jasonjewik Jul 3, 2023
64bcf1b
Update docs and notebook
jasonjewik Jul 5, 2023
f271d2e
Update docs
jasonjewik Jul 5, 2023
fc2e456
Finish tasks and datasets
jasonjewik Jul 5, 2023
a720f1d
Finish docs
jasonjewik Jul 5, 2023
2ff90b6
Update configs
jasonjewik Jul 5, 2023
eb18b5a
Fix pyproject.toml
jasonjewik Jul 5, 2023
1f24ead
Remove rasterio version for rtd
jasonjewik Jul 6, 2023
f721886
Change rtd python version
jasonjewik Jul 6, 2023
8f17d59
Add OS
jasonjewik Jul 6, 2023
70f3025
Add images
jasonjewik Jul 6, 2023
ae0353d
v1 release
jasonjewik Jul 6, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -161,3 +161,4 @@ cython_debug/

# Ignore checkpoint files
*.ckpt
checkpoints/
6 changes: 6 additions & 0 deletions .readthedocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@
# Required
version: 2

# Set the version of Python
build:
os: ubuntu-22.04
tools:
python: "3.8"

# Build documentation in the docs/ directory with Sphinx
sphinx:
configuration: docs/source/conf.py
Expand Down
34 changes: 11 additions & 23 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -2,39 +2,27 @@
# Visit https://bit.ly/cffinit to generate yours today!

cff-version: 1.2.0
title: "ClimateLearn: Benchmarking Machine Learning for Data-driven Climate Science"
title: "ClimateLearn: Benchmarking Machine Learning for Weather and Climate Modeling"
message: >-
If you use this software, please cite it using the
metadata from this file.
type: software
authors:
- given-names: Hritik
family-names: Bansal
email: hbansal@g.ucla.edu
affiliation: 'University of California, Los Angeles'
- given-names: Shashank
family-names: Goel
email: shashankgoel@g.ucla.edu
- given-names: Tung
family-names: Nguyen
email: tungnd@cs.ucla.edu
affiliation: 'University of California, Los Angeles'
- given-names: Jason
family-names: Jewik
email: jason.jewik@cs.ucla.edu
affiliation: 'University of California, Los Angeles'
- given-names: Siddharth
family-names: Nandy
email: sidd.nandy@gmail.com
email: jason.jewik@ucla.edu
affiliation: 'University of California, Los Angeles'
- given-names: Tung
family-names: Nguyen
email: tungnd@g.ucla.edu
affiliation: 'University of California, Los Angeles'
- given-names: Seongbin
family-names: Park
email: shannonsbpark@gmail.com
- given-names: Hritik
family-names: Bansal
email: hbansal@ucla.edu
affiliation: 'University of California, Los Angeles'
- given-names: Jingchen
family-names: Tang
email: tangtang1228@ucla.edu
- given-names: Prakhar
family-names: Sharma
email: prakhar6sharma@gmail.com
affiliation: 'University of California, Los Angeles'
- given-names: Aditya
family-names: Grover
Expand Down
42 changes: 21 additions & 21 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
# MIT License
#
#@title Copyright (c) 2021 CCAI Community Authors { display-mode: "form" }
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.
MIT License

Copyright (c) 2021-present Machine Intelligence Group at UCLA

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
15 changes: 4 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,33 +3,26 @@
[![Documentation Status](https://readthedocs.org/projects/climatelearn/badge/?version=latest)](https://climatelearn.readthedocs.io/en/latest/?badge=latest)
[![CI Build Status](https://github.com/aditya-grover/climate-learn/actions/workflows/ci.yaml/badge.svg)](https://github.com/aditya-grover/climate-learn/actions/workflows/ci.yaml)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
[![Google Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1WiNEK1BHsiGzo_bT9Fcm8lea2H_ghNfa)
[![Google Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1LcecQLgLtwaHOwbvJAxw9UjCxfM0RMrX?usp=sharing)

**ClimateLearn** is a Python library for accessing state-of-the-art climate data and machine learning models in a standardized, straightforward way. This library provides access to multiple datasets, a zoo of baseline approaches, and a suite of metrics and visualizations for large-scale benchmarking of statistical downscaling and temporal forecasting methods. For further context on our past motivation and future plans, check out our announcement [blog post](https://aditya-grover.github.io/blog/2023/climate-learn/).

## Usage

[**Python3**](https://www.python.org/) is required.
[**Python 3.8+**](https://www.python.org/) is required. The xESMF package has to be installed separately since one of its dependencies, ESMpy, is available only through Conda.
```
conda install -c conda-forge xesmf
pip install climate-learn
```

### Quickstart
We have a series of tutorial Jupyter notebooks in the `notebooks` folder. We recommend reading them in the following order to see a typical ClimateLearn workflow.
1. Data Processing
2. Model Training & Evaluation
3. Visualization

To run the notebooks, please upload them to [Google Colab](https://colab.research.google.com/).
We have a quickstart notebook in the `notebooks` folder titled `Quickstart.ipynb`. It is intended for use in Google Colab and can be launched by clicking the Google Colab badge above or this link: https://colab.research.google.com/drive/1LcecQLgLtwaHOwbvJAxw9UjCxfM0RMrX?usp=sharing.

We also previewed some key features of ClimateLearn at a spotlight tutorial in the "Tackling Climate Change with Machine Learning" Workshop at the Neural Information Processing Systems 2022 Conference. The slides and recorded talk can be found on [Climate Change AI's website](https://www.climatechange.ai/papers/neurips2022/114).

### Documentation
Find us on [ReadTheDocs](https://climatelearn.readthedocs.io/).

### Integrations
- [Weights & Biases](https://wandb.ai/site)

## About Us
ClimateLearn is managed by the Machine Intelligence Group at UCLA, headed by [Professor Aditya Grover](https://aditya-grover.github.io).

Expand Down
17 changes: 7 additions & 10 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,20 +10,17 @@
# -- Project information

project = "ClimateLearn"
copyright = "2022; Bansal, Goel, Jewik, Nandy, Nguyen, Park, Tang, Grover"
author = """
Hritik Bansal,
Shashank Goel,
Jason Jewik,
Siddharth Nandy,
copyright = "2023; Nguyen, Jewik, Bansal, Sharma, Grover"
author = """
Tung Nguyen,
Seongbin Park,
Jingchen Tang,
Jason Jewik,
Hritik Bansal,
Prakhar Sharma,
Aditya Grover
"""

release = "0.1"
version = "0.1.0"
release = "1.0.0"
version = "1.0.0"

# -- General configuration

Expand Down
52 changes: 36 additions & 16 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,36 +12,56 @@ statistical downscaling and temporal forecasting methods.

.. note::

This project is under active development.

About Us
--------
ClimateLearn is managed by the Machine Intelligence Group at UCLA, headed by
`Professor Aditya Grover <https://aditya-grover.github.io/>`_.
This project is under active development. The API might undergo extensive
changes in the near future.

Getting Started
---------------
Please see the `quickstart section <https://github.com/aditya-grover/climate-learn#quickstart>`_ of our GitHub repository.
`Python 3.8+ <https://www.python.org/>`_ is required. The xESMF package has
to be installed separately since one of its dependencies, ESMpy, is available
only through Conda.

.. code-block:: shell

conda install -c conda-forge xesmf
pip install climate-learn

We have a quickstart notebook in the ``notebooks`` folder titled
``Quickstart.ipynb`` that walks through an example usage of ClimateLearn for
weather forecasting from downloading the data through visualizing the
predictions of a trained model. It is intended for use in Google Colab and can
be launched by clicking
`this link <https://colab.research.google.com/drive/1LcecQLgLtwaHOwbvJAxw9UjCxfM0RMrX?usp=sharing>`_.

.. toctree::
:caption: User Guide
:maxdepth: 2

user-guide/datasets
user-guide/tasks_and_datasets
user-guide/models
user-guide/metrics
user-guide/visualizations

.. toctree::
:caption: API Reference
:maxdepth: 1
:glob:

reference/*

.. toctree::
:caption: Development Guide
:maxdepth: 1

development-guide/for-developers
development-guide/for-maintainers
development-guide/for-maintainers

Why did we build ClimateLearn?
------------------------------

In recent years, there has been a growing interest in the application of
ML-based methods for weather and climate modeling. While there are some
leaderboard benchmarks, such as WeatherBench, ClimateBench, and FloodNet, that
propose datasets and baselines for specific tasks in climate science, a
holistic software ecosystem that encompasses the entire data, modeling, and
evaluation pipeline across several tasks is lacking. Hence, we built
ClimateLearn to standardize datasets, model implementations, and evaluation
protocols for rigorous and reproducible data-driven climate science.

About Us
--------
ClimateLearn is built and maintained by the Machine Intelligence Group at UCLA,
headed by `Professor Aditya Grover <https://aditya-grover.github.io/>`_.
23 changes: 0 additions & 23 deletions docs/source/reference/data.rst

This file was deleted.

7 changes: 0 additions & 7 deletions docs/source/reference/models.rst

This file was deleted.

4 changes: 0 additions & 4 deletions docs/source/reference/training.rst

This file was deleted.

24 changes: 0 additions & 24 deletions docs/source/reference/utils.rst

This file was deleted.

54 changes: 0 additions & 54 deletions docs/source/user-guide/datasets.rst

This file was deleted.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[ZoneTransfer]
ZoneId=3
HostUrl=about:internet
Binary file added docs/source/user-guide/images/bias_at_index.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[ZoneTransfer]
ZoneId=3
HostUrl=about:internet
Binary file added docs/source/user-guide/images/gt_at_index.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[ZoneTransfer]
ZoneId=3
HostUrl=about:internet
Binary file added docs/source/user-guide/images/mean_bias.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions docs/source/user-guide/images/mean_bias.png:Zone.Identifier
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[ZoneTransfer]
ZoneId=3
HostUrl=about:internet
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed docs/source/user-guide/images/visualize.png
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[ZoneTransfer]
ZoneId=3
HostUrl=about:internet
Binary file not shown.
Loading
Loading