MYNN fix for numerical stability issues with mixing snow

[joeolson42]

Description

This is simply a removal of the snow mixing, reverting back to the RRFS Base E configuration. For some unknown reason, mixing snow was contributing to numerical instabilities. The ultimate cause of this instability will be investigated in the future but for now, snow mixing is deactivated.

There is also a small change to table values of Leaf Area Index to match them better with the LAI monthly climatology that is used in HRRR.

Also, 2-line change was made for a non-standard Fortran bug discovered by the Cray compiler (from John Michalakes). No known impact when using other compilers.

Input data additions/changes

• No changes are expected to input data.
• There will be new input data.
• Input data will be updated.

Anticipated changes to regression tests:

• No changes are expected to any regression test.
• Changes are expected to the following tests:

Subcomponents involved:

• AQM
• CDEPS
• CICE
• CMEPS
• CMakeModules
• FV3
• GOCART
• HYCOM
• MOM6
• NOAHMP
• WW3
• stochastic_physics
• none

Combined with PR's (If Applicable):

N/A

Commit Queue Checklist:

• Link PR's from all sub-components involved
• Confirm reviews completed in sub-component PR's
• Add all appropriate labels to this PR.
• Run full RT suite on either Hera/Cheyenne with both Intel/GNU compilers
• Add list of any failed regression tests to "Anticipated changes to regression tests" section.

Linked PR's and Issues:

• Depends on MYNN fix for numerical stability issues with mixing snow NOAA-EMC/fv3atm#656
• Depends on MYNN fix for numerical stability issues with mixing snow ccpp-physics#71
• fixes binary/unary operator precedence error on Cray compiler ccpp-physics#72

Testing Day Checklist:

• This PR is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR.
• Move new/updated input data on RDHPCS Hera and propagate input data changes to all supported systems.

Testing Log (for CM's):

• RDHPCS
  • Intel
    • [ x] Hera
    • Orion
    • Jet
    • Gaea
    • Cheyenne
  • GNU
    • Hera
    • Cheyenne
• WCOSS2
  • Dogwood/Cactus
  • Acorn
• CI
  • Completed
• opnReqTest
  • N/A
  • Log attached to comment

[joeolson42]

Regression test results. Only fails are due to changes in the physics for all RAP, HRRR, and regional tests.
RegressionTests_hera.gnu.log
RegressionTests_hera.intel.log

[joeolson42]

Updated regression tests. All compile & run. Only fails are due to changes from the baseline in suites that use MYNN or RUC (i.e., HRRR, RAP, regional).
RegressionTests_hera.gnu.log
RegressionTests_hera.intel.log

[zach1221]

@joeolson42 It looks like this PR may be ready, you've provided hera logs and ccpp sub-pr has approval. We can start testing against it, if you can resolve the conflicts and ensure your branch is synced up?

[joeolson42]

I'll sync it again now. Thanks! -joe

…

On Fri, Jun 2, 2023 at 12:45 PM zach1221 ***@***.***> wrote: @joeolson42 <https://github.com/joeolson42> It looks like this PR may be ready, you've provided hera logs and ccpp sub-pr has approval. We can start testing against it, if you can resolve the conflicts and ensure your branch is synced up? — Reply to this email directly, view it on GitHub <#1754 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADLRR3S5IJHNW2QLSXNWAWLXJIYDJANCNFSM6AAAAAAYAF4KX4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Joseph Olson Physics Branch Chief Environmental Prediction Advancement Division NOAA-Global Systems Laboratory Boulder, Colorado

[zach1221]

@BrianCurtis-NOAA I think this pr is ready. Going to run jenkins-ci first.

[zach1221]

Please see jenkins-ci logs attached. ORTs passed. I'll move forward with the remaining autoRT tests now. @BrianCurtis-NOAA fyi.
ufs-weather-model » ort-docker-pipeline » PR-1754 #1 Console [Jenkins].pdf

[BrianCurtis-NOAA]

Automated RT Failure Notification
Machine: jet
Job: BL
Repo location: /lfs4/HFIP/hfv3gfs/role.epic/autort/pr/1349366313/20230604185457/ufs-weather-model
Please manually delete: /lfs4/HFIP/hfv3gfs/role.epic/RT_RUNDIRS/role.epic/FV3_RT/rt_231427
Please make changes and add the following label back: jet-BL

[BrianCurtis-NOAA]

@zach1221 I think we're working on #1743 currently

[zach1221]

@joeolson42 The majority of the Regression Test cases passed, when ran against your PR, on Gaea. However, there are some issues with cpld_control_p8_faster & cpld_debug_p8 causing them to fail and it appears system related. I recommended we turn these two tests off for Gaea in rt.conf, for the time being, and create a separate issue to have this resolved. This will allow us to continue moving with this PR. Please let me know your thoughts. @jkbk2004

[joeolson42]

I agree with you that it's not likely a problem with my modifications. None of the files I changed should be used in the cpld_control_p8_faster & cpld_debug_p8 experiments.

…

On Mon, Jun 5, 2023 at 6:37 PM zach1221 ***@***.***> wrote: @joeolson42 <https://github.com/joeolson42> The majority of the Regression Test cases passed, when ran against your PR, on Gaea. However, there are some issues with cpld_control_p8_faster & cpld_debug_p8 causing them to fail and it appears system related. I recommended we turn these two tests off for Gaea in rt.conf, for the time being, and create a separate issue to have this resolved. This will allow us to continue moving with this PR. Please let me know your thoughts. @jkbk2004 <https://github.com/jkbk2004> [image: image] <https://user-images.githubusercontent.com/99902696/243514336-59e7c583-9821-4db4-afb2-d90a78f4dabc.png> — Reply to this email directly, view it on GitHub <#1754 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADLRR3WPPJKJXFTQSDTRP6LXJZ3TDANCNFSM6AAAAAAYAF4KX4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Joseph Olson Physics Branch Chief Environmental Prediction Advancement Division NOAA-Global Systems Laboratory Boulder, Colorado

[zach1221]

Hello, update on issue #1790 , updating the tasks per node to TPN=24 in default_vars.sh passes for the majority of cases out of rt.conf, except for about 10 consisting of various hafs, regional and datm cases. For these failed cases, if we keep them at TPN=36 at the tests/tests level they too will pass. We can make the necessary updates in this PR, as a temporary solution, while we wait for the Gaea admin to respond to our inquiry and dig further into the cause of the issue. @jkbk2004

[jkbk2004]

Gaea tests pass with default TPN=24 but some cases need to keep TPN=36: datm_cdeps_lnd_gswp3, hafs_regional_specified_moving_1nest_atm, regional_2dwrtdecomp. I agree with the idea to finalize through the issue #1790 after receiving some information from gaea admin.

[zach1221]

Good morning, @BrianCurtis-NOAA . How's WCOSS2 and Acorn coming along?

[BrianCurtis-NOAA]

Good morning, @BrianCurtis-NOAA . How's WCOSS2 and Acorn coming along?

I was out Friday, so i've been working on baselines so far this AM. Almost ready to start comparisons.

[zach1221]

@BrianCurtis-NOAA understood. No problem!

[zach1221]

Testing is now complete, so we can proceed with merging the sub-PRs. I'll follow up on CCPP #71

[zach1221]

@joeolson42 fv3atm pr #656 has been merged. Please go ahead to update submodule pointer and revert the change in .gitmodules. @jkbk2004