Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add coupled support and update UFS (#500)
Adds support for full coupled model except aerosols (FV3-WW3-MOM6-CICE) following the prototype settings. Support for aerosols will be added soon.
This also updates the UFS version to develop as of Oct 7. There are associated additions or changes to settings, but we've tried to have defaults maintain the same behavior as previously.
Issues related to memory in prep and ocnpost on Hera and Orion due to changes in the slurm configuration have been addressed.
Due to a change in the dycore that uses the checksum in the NetCDF files to check for data integrity, warm start ICs may need an additional offline step before use to update the checksum:
```
ncatted -a checksum,,d,, ${RESTARTinp}/${PDY}.${cyc}0000.fv_core.res.tile1.nc ./fv_core.res.tile1.nc
ncatted -a checksum,,d,, ${RESTARTinp}/${PDY}.${cyc}0000.fv_tracer.res.tile1.nc ./fv_tracer.res.tile1.nc
```
A new setting APP controls what components (and their respective jobs) are turned on. The setting can be found in config.base, but can be set at experiment setup time using the new --app option. Recognized values are ATM (atm-only), ATMW (atm and waves), S2S (atm-ocn-ice), S2SW (atm-ocn-ice-wav). If no value is given, the default is ATM, so there is no change in behavior if you omit the option. For now, these values follow their UFS equivalents, but they may diverge in the future based on the needs of global workflow. ATMW is known to not quite work; S2S is so far untested.
When run using the S2SW app, the workflow will automatically substitute the coupled IC copying job in place of the normal gfsprep jobs and use roughly the settings from prototype 7.2 (except aerosols). The alternate settings for the coupled configuration (compared to ATM) are taken from the config.defaults.s2sw file.
Coupled prototype ICs are currently being maintained on Hera, Orion, and WCOSS-Dell. The locations are set in the config.coupled_ic. There is a base location (BASE_CPLIC) and a setting for each component specifying a subdirectory within the base location.
There are new options available for the checkout.sh, build_all, and link_fv3 scripts:
- checkout.sh has a new -m option that allows you to override the UFS hash that is checked out.
- build_all.sh has two new options: -c to build UFS as the coupled model, and -a to build UFS as ATM-GOCART (but aerosols are not fully supported yet). These options are then passed to build_ufs.sh. The options are temporary, as all configurations will be available from a single executable soon. When building for coupled, the subcomponents built is controlled by cpl_build.cfg.
- link_fv3.sh is renamed to link_workflow.sh and adds an optional third argument, "coupled". When provided, the files necessary for coupled are linked. This option is also temporary, as the link script will be updated to always link these files.
### To run in coupled mode with prototype settings/inputs
1. Clone as usual
2. Run checkout.sh
3. Run build_all.sh with the -c option `./build_all.sh -c` (the -c option is temporary)
4. Run link using the coupled argument: `./link_workflow.sh emc <machine> coupled`
5. Run setup_expt_fcstonly.py using the --app option S2SW
6. Modify config files as necessary. config.defaults.s2sw will automatically override some settings when app is S2SW
7. Run setup_workflow_fcstonly.py as usual
8. Setup rocoto to run your experiment
### Technical udpates
Converts global forecast into a set of modular scripts. The scripts define a bunch of functions to be called, with separate functions for each component in each classification. For instance, there is an FV3_GFS_predet, MOM6_predet, CICE_predet, etc. that are conditionally run if the associated component is on.
The diag table for AOD 550 (diag_table_aod) is reduced to just the portion needed in addition to the normal diag table so it can be appended. Ideally, in the future, we extend this treatment so diag tables can be built based on settings instead of having separate ones for each combination.
Wave input file templates that were formerly kept in the fix directory are now more appropriately moved inside the global-workflow in parm.
The group labels for the post jobs now indicates the forecast hours included instead of just an index. This required minor updates to check for 'anl' instead of 0 for analysis jobs.
The checkout script now writes its logs to the logs/ directory the same way as build does, instead of leaving them in sorc/
The UFS model source has been moved to a ufs_model.fd directory, and the build script has been renamed build_ufs.sh.
Error checking was added to the build_ww3prepost.sh script.
The archive script has been streamlined and the section that saves gaussian grid files was fixed, but turned off by default (see PR #517).
rocoto_viewer has been updated to python3 and the dependency on prod_utils was removed, which makes the script more portable. Be sure to load a sufficient version of python to use.
For those coming from coupled-crow: ice and ocean output files are now linked to COM from the forecast run directory instead of being copied at the end of the forecast. This allows post to run as files are produced instead of after the forecast is complete. Ocean post jobs are handled the same as atmosphere post: forecast hours are grouped into a number of tasks determined by NPOSTGRP in config.ocnpost. The ocean and ice output are now also located in their own component directories in COM (the joint ocean-ice files are located in the ocean directory for now). The atmosphere latitudes are reversed from coupled-crow (see c59260b).
Co-authored-by: jikuang <jikuang@umd.edu>
Co-authored-by: Kate.Friedman <Kate.Friedman@noaa.gov>
Co-authored-by: Rahul Mahajan <aerorahul@users.noreply.github.com>
Co-authored-by: JessicaMeixner-NOAA <Jessica.Meixner@noaa.gov>
Co-authored-by: JianKuang-UMD <51758200+JianKuang-UMD@users.noreply.github.com>
Co-authored-by: lgannoaa <37596169+lgannoaa@users.noreply.github.com>- Loading branch information
7 people
authored
Dec 10, 2021
1 parent
8abe1df
commit dd03ed0