[develop] Add process lightning task

[EdwardSnyder-NOAA]

DESCRIPTION OF CHANGES:

This PR adds the process lightning task to the SRW App from the NOAA-GSL RRFS_dev1 branch. The process lightning task is part of the RRFS configuration of the SRW App.

A wrapper script was added since this task can be ran outside of the workflow. The following are the instructions on how to run this process on Jet:

• Clone repo and build App
• salloc compute node by doing: salloc -N 1 -n 1 -A hfv3gfs -t 00:30:00 -q batch
• Load envs
  • Module use /path/to/ufs_srweather_app/modulefiles
  • Module load build_jet_intel
    • Note: only need the hpc, hpc-intel, hpc-impi modules so:
    • module use /mnt/lfs4/HFIP/hfv3gfs/role.epic/hpc-stack/libs/intel-2022.1.2/modulefiles/stack
    • Module load hpc hpc-intel hpc-impi
  • Module load wflow_jet
  • Conda activate regional_workflow
• Create EXPT
  • cd into ush and copy config.community.yaml to config.yaml
  • Update the following vars:
    • MACHINE: jet
    • ACCOUNT: hfv3gfs
    • PREDEF_GRID_NAME: RRFS_CONUS_3km
    • DATE_FIRST_CYCL: '2022072003'
    • DATE_LAST_CYCL: '2022072003'
    • FCST_LEN_HRS: 6
  • Add a rrfs section at the end of the config.yaml file with DO_NLDN_LGHT variable. See example below:
    • rrfs:
      • DO_NLDN_LGHT: true
  • Run generate_FV3LAM_wflow.py script to create expt
• Run EXPT
  • cp run_process_lightning.sh wrapper from ush/wrappers to expt_dir
  • Export var: export EXPTDIR=$PWD
  • Run script: ./run_procs_lightning.sh 2>&1 | tee proc_lght.log

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

TESTS CONDUCTED:

• hera.intel
• orion.intel
• cheyenne.intel
• cheyenne.gnu
• gaea.intel
• jet.intel
• wcoss2.intel
• NOAA Cloud (indicate which platform)
• Jenkins
• fundamental test suite
• comprehensive tests (specify which if a subset was used)

DEPENDENCIES:

None.

DOCUMENTATION:

ISSUE:

CHECKLIST

• My code follows the style guidelines in the Contributor's Guide
• I have performed a self-review of my own code using the Code Reviewer's Guide
• I have commented my code, particularly in hard-to-understand areas
• My changes need updates to the documentation. I have made corresponding changes to the documentation
• My changes do not require updates to the documentation (explain).
• My changes generate no new warnings
• New and existing tests pass with my changes
• Any dependent changes have been merged and published

LABELS (optional):

A Code Manager needs to add the following labels to this PR:

• Work In Progress
• bug
• enhancement
• documentation
• release
• high priority
• run_ci
• run_we2e_fundamental_tests
• run_we2e_comprehensive_tests
• Needs Cheyenne test
• Needs Jet test
• Needs Hera test
• Needs Orion test
• help wanted

CONTRIBUTORS (optional):

[christinaholtNOAA]

@EdwardSnyder-NOAA This is shaping up nicely. I have left a few comments/suggestions below.

I'm curious if you made changes to the default_configs.yaml file. Perhaps it's missing here? Also, if you tested the script standalone, would you mind including the necessary environment variables you passed the j-job in the j-job header?

[christinaholtNOAA]

I don't think we have CDATE anymore. The date is now passed with NCO-compliant PDY and cyc variables. For the timestamp variables below, I think that translates to:

YYYYMMDDHH=${PDY}${cyc}
YYYYMMDD=${PDY}
HH=${cyc}

[EdwardSnyder-NOAA]

I'm unsure what the NCO-complaint variables are. But in the develop branch, CDATE is used in a number of ufs-srweather-app/scripts/* and CDATE is defined in the job_preamble.sh script as export CDATE=${PDY}${cyc}.

[MatthewPyle-NOAA]

I would consider CDATE to be non-standard, but tolerated in operations (is pretty widely used in the GFS).

[EdwardSnyder-NOAA]

Ah okay. That's good to know. I switched the code to be: START_DATE=$(echo "${PDY} ${cyc}")

[EdwardSnyder-NOAA]

@EdwardSnyder-NOAA This is shaping up nicely. I have left a few comments/suggestions below.

I'm curious if you made changes to the default_configs.yaml file. Perhaps it's missing here? Also, if you tested the script standalone, would you mind including the necessary environment variables you passed the j-job in the j-job header?

@christinaholtNOAA I added the default_config.yaml file and the wrapper script I used to run this process. I also included instructions on how to run this task with it. In addition, I addressed your comments from earlier. Let me know what you think. I just noticed I still have a number of date variables commented out in the exregional script, which can be removed.

[christinaholtNOAA]

I think it would be good to stage some lightning data for testing with the other test data, but we'd likely want to preserve the lightning data root variable for any supported near-real time data streams.

[EdwardSnyder-NOAA]

I think it would be good to stage some lightning data for testing with the other test data, but we'd likely want to preserve the lightning data root variable for any supported near-real time data streams.

I added some lightning data to Hera and Jet to test lightning functionality and it worked. This PR is ready for review.

[christinaholtNOAA]

LGTM.

[christinaholtNOAA]

@MichaelLueken We'd also like to focus on getting this one merged for RRFS, without waiting on any others. Thanks!

[MichaelLueken]

@EdwardSnyder-NOAA I'm working on reviewing this PR now and see that there is a conflict in ush/config_defaults.yaml. Please merge the latest develop into your feature/rrfs-lightning branch to address the conflict at your earliest convenience. Thanks!

[EdwardSnyder-NOAA]

@christinaholtNOAA in #647 you mentioned that these do switches at this level and quantity shouldn't be how the workflow is configured, and that we should define the tasks based on its definition. If that's the case, do I need to include the DO_NLDN_LGHT variable?

[christinaholtNOAA]

I'm fine with it staying. I think we can assess whether it's needed for the workflow when it's integrated. One of these here or there can be handled. It's hard to balance/manage a bunch of them, though.

[EdwardSnyder-NOAA]

@MichaelLueken thanks for catching that! Conflict should be resolved now. While resolving the conflict, I noticed some logic I may not need, so I tagged Christina for assistance.

[MichaelLueken]

@EdwardSnyder-NOAA These changes look good to me and I was able to successfully run the wrapper script on Jet. I will now approve these changes, but will hold off on running the Jenkins tests on this work until @christinaholtNOAA has confirmed whether to include DO_NLDN_LGHT in ush/config_defaults.yaml.

[MichaelLueken]

@EdwardSnyder-NOAA Just to check, when this work is merged to develop, then issue #487 will be complete, correct? Thanks!

[christinaholtNOAA]

@MichaelLueken 👍 from me.

[MichaelLueken]

The Jenkins tests completed. The tests on Hera Intel failed and grid_RRFS_CONUS_25km_ics_NAM_lbcs_NAM_suite_GFS_v16 failed on Cheyenne Intel. The failed test was manually ran on Cheyenne and successfully passed. The Hera Intel tests were ran manually and passed (with the exception of grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_RAP_suite_HRRR due to issue #688). Proceeding with merging this work to develop.