Skip to content

rrfs‐workflow code norm

Jump to bottom Edit New page
Guoqing Ge edited this page Aug 15, 2024 · 5 revisions

-This is only a preliminary draft. A final version will be made available after developers' inputs and a code management meeting
-The "develop user" on this page refers to GSL/EMC users who don't run the workflow in the NCO production space

  1. Adopt the NCO standards.The following rules are mostly based on the NCO standards.

  2. Think big. The workflow will include complicated DA logics, different domains/resolutions (such as CONUS vs NA, 3km vs 12km, etc), spinup and prod cycles, ensemble components, air quality components, RTMA applications, etc.

  3. The core of the workflow will only consider the NCO naming convention for all existing operational products (such as gfs grib2 files, etc). However, the workflow will provide link scripts to use hard or soft links to convert users' specific naming conventions to match the NCO standard.

  4. Reduce the Python library dependencies as much as possible for the workflow.

  5. It is recommended to use only BASH or Python for scripting.
    Every script file (bash or python) should be set as executable and can run from the command line directly as ./myscript.sh or ./myscript.py
    so add the following shebang to the first line of the script files:
    #!/usr/bin/env bash
    the above is preferred, but #!/bin/bash is also acceptable.
    or
    #!/usr/bin/env python

  6. One can source other bash files to set relevant environment variables at the beginning of a J-job

  7. Put the following three lines in a J-job script at the beginning:

declare -rx PS4='+ $(basename ${BASH_SOURCE[0]:-${FUNCNAME[0]:-"Unknown"}})[${LINENO}]${id}: '
set -x
date

This will show information about which line of which script file generates the output, as follows:

+ JRRFS_IC[31]: export pgmout=/lfs5/BMC/wrfruc/gge/nco/stmp/conus12km/1.0.1/rrfs.20240527/00/ic/OUTPUT.ic
+ JRRFS_IC[31]: pgmout=/lfs5/BMC/wrfruc/gge/nco/stmp/conus12km/1.0.1/rrfs.20240527/00/ic/OUTPUT.ic
+ JRRFS_IC[32]: /mnt/lfs5/BMC/wrfruc/gge/rrfsx/scripts/exrrfs_ic.sh
+ exrrfs_ic.sh[4]: cpreq='ln -snf'
+ exrrfs_ic.sh[5]: prefix=GFS
+ exrrfs_ic.sh[6]: cd /lfs5/BMC/wrfruc/gge/nco/stmp/conus12km/1.0.1/rrfs.20240527/00/ic

  1. Use source instead of a dot for better readability. (There is no difference between source and dot in BASH)

  2. The ending of a J-job should be like this:

#
#----------------------------------------
# Execute the script.
#----------------------------------------
export pgmout="${DATA}/OUTPUT.${task_id}"
$SCRIPTSrrfs/exrrfs_${task_id}.sh
export err=$?; err_chk

if [ -e "$pgmout" ]; then
  cat $pgmout
fi
#
#----------------------------------------
# Remove the Temporary working directory
#----------------------------------------
cd ${DATAROOT}
KEEPDATA_da=${KEEPDATA_da:-${KEEPDATA}}
[[ "${KEEPDATA_da}" == "NO" ]] && rm -rf ${DATA}
#
date
echo "JOB ${jobid:-} HAS COMPLETED NORMALLY!"
exit 0

  1. It is preferred to use [[ instead of [. [[ is bash's extension to the [ command. It has several enhancements that make it a better choice if you write scripts that target bash.

  2. Use == instead of = for string comparison. Double-quote strings to be compared. For example: if [[ "${begin}" == "YES" ]]; then

  3. Enfore base-10 arithmetic operations to avoid unexpected errors when dealing with such as numbers "03" or "003":
    if (( 10#${ENS_SIZE:-0} > 0 )); then

  4. use ${cpreq} to copy files/directories that are required for a job to function. In most situations, soft links work better for develop users, so the following line is added in workflow/rocoto/launch.sh to tweak the cpreq command for develop users:
    export cpreq="ln -snf" #use a soft link instead of copy for non-NCO experiments

  5. All files under workflow/rocoto will NOT go into the NCO operation. It is used to do some tweaks, mimic ecflow job cards, and then provide flexibility for develop users.

  6. The following environmental variables should always be available for any tasks per the NCO standard:
    HOMErrfs, EXPDIR, CDATE, PDY, cyc, COMROOT, DATAROOT, VERSION, MACHINE, NET, RUN, TAG
    Examples for CDATE, PDY, cyc: (NOTE: cyc is an exception and all in lower cases)

CDATE=2024052703
PDY=20240527
cyc=03

  1. All tasks have an input and output. Input data should be under a directory defined by a COMIN variable. For example, COMINgfs provides IC/LBC grib2 files
    Output data should be under a directory defined by a COMOUT variable.

  2. The working directory should be defined by the ${DATA} variable. In NCO, a working directory will be removed immediately after a job is completed successfully. Develop users can set KEEPDATA=YES to keep working directories.
    In the rrfs-workflow, users can further choose to keep data for individual tasks by setting such as KEEPDATA_da=YES.

  3. In scripts or config files, except for a few exceptions due to NCO practices (such as cyc), a variable whose name starts with upper cases is assumed to be exported to a subshell (such as LBC_OFFSET_HRS, COMINgfs, etc) while all lower cases mean a temporary variable that is only visible to the script defining it and will not go into sub-shells

  4. Use -s instead of -f to check if a file exists and is not size zero.

  5. Catch and handle return code on all cases (Run executable, wgrib2, python, ush, script, utility...).

  6. Correctly label output information as INFO, WARNING or FATAL ERROR.

  7. Use ${NDATE} to find previous or future cycles/dates, the date command is only used to output a format string

  8. A workflow calls a J-job, a J-job calls an ex-script, and an ex-script calls scripts/executables under ush/exec respectively.

24. Use nouns for task names, and avoid verbs as much as possible.

  1. rrfs-workflow adopts a config cascade and a resource environmental variable cascade so that one can optionally fine-tune settings for individual tasks.
    For example: To get the walltime setting for a spinup forecast job, the workflow checks the following variables in this order:
    WALLTIME_FCST_SPINUP, WALLTIME_FCST, WALLTIME
    until a variable is defined. Check this link for more information.

  2. rrfs-workflow uses the powerful but at the same time intuitive Python language to generate the rocoto workflow (and potentially the ecflow, cylc workflow in the future) directly.

  3. Use 2 spaces for indentation in Python and BASH scripts. Avoid using a TAB or 4 spaces.

  4. In bash, double quotes and single quotes function differently while in Python there are no differences.

  5. In config files, don't forget to add "export" for any variables that will be exported to sub-shells.

  6. To be safe, put a space before )) and after (( in BASH:

FHRin=$((10#${FHR}+10#${offset}) # This is wrong, but may not be easy to debug
FHRin=$(( 10#${FHR} + 10#${offset} )) #If we add spaces, it will help reduce bugs
  1. Need spaces before and after == 
if [[ "${TYPE}"=="IC" ]] || [[ "${TYPE}"=="ic" ]]; then  # This is wrong
if [[ "${TYPE}" == "IC" ]] || [[ "${TYPE}" == "ic" ]]; then  # this is correct

  1. export WALLTIME_UPP=${WALLTIME_UPP:"00:50:00"} # this is wrong. Be sure to have :- instead of : only.

  2. Use ${var} to reference a variable instead of $var. This is more robust and avoids situations where $var may cause trouble.

  3. Python's f-string uses {var} to reference a variable while BASH uses ${var} to reference a variable.

  4. In Bash, use the double parentheses (( )) instead of [[ ]] for arithmetic operations and comparisons. The former is more intuitive and less error-prone. For example, use if (( 10#$num1 > 10#$num2 )); then, avoid if [[ "${num1}" -gt "${num2}" ]]; then.
    More examples about (( )):

if (( 10#$num1 == 10#$num3 && 10$num1 < 10$num2 )); then
if (( 10#$num1 != 10#$num2 )); then
if (( 10#$num1 > 10#num2 )); then
if (( 10#$num1 < 10#num2 )); then
if (( 10#$num1 >= 10#num2 )); then

  1. Test whether a variable is empty:
    if [[ -z ${cycles} ]]; then # this is not right.
    if [[ -z "${cycles}" ]]; then # this is correct

  2. Any file/directory preparation work or postprocessing work should be done in an ex-script or ush-script, not in a Python script. Python scripts should NOT be used to replace anything that can be easily done in BASH, such as linking files, copying files, creating directories, etc. Python scripts are only used to fulfill a function which is almost impossible in BASH. Python scripts accept an input (a variable or an input namelist) from BASH scripts and generate outputs (return texts or an output file) for BASH scripts to continue.

Add a custom footer
Add a custom sidebar
Clone this wiki locally