Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplified and added setup section #315

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .github/link-checker-config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"projectBaseUrl":"${https://github.com/ESMValGroup/ESMValTool_Tutorial}",
"ignorePatterns": [
{
"pattern": "^http://example.net"
}
],
"timeout": "20s",
"retryOn429": true,
"retryCount": 5,
"fallbackRetryDelay": "30s",
"aliveStatusCodes": [200, 206]
}
17 changes: 17 additions & 0 deletions .github/workflows/link-checker.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
name: Check Markdown links

on: [pull_request]

jobs:
markdown-link-check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@master
- uses: gaurav-nelson/github-action-markdown-link-check@v1
with:
use-quiet-mode: 'yes'
use-verbose-mode: 'yes'
config-file: 'link-checker-config.json'
file-path: './README.md, ./LICENSE.md, ./CONTRIBUTING.md, ./CODE_OF_CONDUCT.md, ./aio.md, ./index.md, ./reference.md, ./setup.md'


12 changes: 12 additions & 0 deletions _episodes/09-cmorization.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,12 @@ run the CMORizer scripts:
esmvaltool data format --config_file <path to config-user.yml> <dataset-name>
```

The options `--start` and `--end` can be added to command above to restrict the
formatting of raw data to a time range. They will be ignored if a specific
dataset does not support (i.e. because it is provided as a single file).
Valid formats are `YYYY`, `YYYYMM`, `YYYYMMDD`. The same way is also applicable for
the option `esmvaltool data download`.

The ``config-user.yml`` is the file in which we define the different data
paths, see the episode on [Configuration]({{ page.root }}{% link _episodes/03-configuration.md %}).
In the ``rootpath`` of your ``config-user.yml``, make sure to add the right
Expand Down Expand Up @@ -193,6 +199,12 @@ You can also see the path where ESMValTool stores the reformatting script:
have a look at this file if you want. The script also uses a configuration file:
`~/ESMValTool/esmvaltool/cmorizers/data/cmor_config/FLUXCOM.yml`.

To get help on CMORizer commands, run the tool with:

```bash
esmvaltool data --help
```

## Make a test recipe

To verify that the data is correctly CMORized, we will make a simple test
Expand Down
Binary file added fig/flowchart_setup.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
54 changes: 27 additions & 27 deletions setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,33 @@ ESMValTool code.
- git:
[https://swcarpentry.github.io/git-novice/](https://swcarpentry.github.io/git-novice/)

![Data flow with ESMValTool](../fig/flowchart_setup.png)

### Using your own machine

Please skip this section if you are not going to use ESMValTool on your
local machine and go [here](#github-account-advanced).

If you are planning on running ESMValTool on your own machine, please make sure
that you are able to download CMIP data and that you have a few GB of space
available to install conda and ESMValTool, but also enough to make a copy of some
data (~125MB) needed for this tutorial.

You can use ESMValTool to automatically download data needed for test recipes.
Please see the [Configuration][lesson-configuration] episode
or the [configuration file documentation][config-file] for more information.
This the recommended option as it has the advantage that data is
stored in subdirectories, and features such as wildcards and recording
the version of the data will work automatically.

Alternatively, you can run the following command using
[wget](https://en.wikipedia.org/wiki/Wget):

~~~shell
wget --no-clobber --input-file \
https://github.com/ESMValGroup/ESMValTool_Tutorial/raw/main/data/dataset.urls \
--directory-prefix $HOME/esmvaltool_tutorial/data/
~~~

## Access to CMIP and Observational data and a suitable compute cluster

Expand Down Expand Up @@ -227,33 +254,6 @@ be found [here](https://docs.dkrz.de/doc/levante/running-jobs/index.html).

Congratulations! Please go here [here](#github-account-advanced) next.

### Using your own machine

Please skip this section if you are not going to use ESMValTool on your
local machine and go [here](#github-account-advanced).

If you are planning on running ESMValTool on your own machine, please make sure
that you are able to download CMIP data and that you have a few GB of space
available to install conda and ESMValTool, but also enough to make a copy of some
data (~125MB) needed for this tutorial.

You can use ESMValTool to automatically download data needed for test recipes.
Please see the [Configuration][lesson-configuration] episode
or the [configuration file documentation][config-file] for more information.
This the recommended option as it has the advantage that data is
stored in subdirectories, and features such as wildcards and recording
the version of the data will work automatically.

Alternatively, you can run the following command using
[wget](https://en.wikipedia.org/wiki/Wget):

~~~shell
wget --no-clobber --input-file \
https://github.com/ESMValGroup/ESMValTool_Tutorial/raw/main/data/dataset.urls \
--directory-prefix $HOME/esmvaltool_tutorial/data/
~~~


## GitHub account (Advanced)

You don’t need a github account to participate in the tutorial. However, if you
Expand Down
Loading