Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using relatively new Thompson cloud fraction scheme, tune a few things to cause fewer overall clouds, particularly high clouds #809

Merged
merged 9 commits into from
Jan 18, 2022

Conversation

gthompsnWRF
Copy link
Collaborator

@gthompsnWRF gthompsnWRF commented Dec 13, 2021

The changes in the code are rather minor. Each change will cause some reduction of all cloud fraction amounts, but high clouds should be affected (reduced) the most. These changes are in response to a 7-day simulation from Anning (@AnningCheng-NOAA) that continues to show overall about 50% global average high clouds whereas the target is closer to 40%.

[Edit: adding in @yangfanglin and @ericaligo-NOAA and @RuiyuSun to see the code changes and Dom or someone else can add them as reviewers if desired.]

Copy link
Collaborator

@climbfuji climbfuji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't know a lot about the science behind these changes, but from a technical point of view these look ok.

@AnningCheng-NOAA
Copy link
Collaborator

I am running a test, seems promising on decreasing the clouds. Some analysis will be performed when the 7-day forecast is done tomorrow .

@AnningCheng-NOAA
Copy link
Collaborator

approved. The cloud fractions have been pretty reasonable now. However, the OLR is still too low (224 W/m^2, targeted 240 W/m^2) and the time series of evaporation and precipitation is not unbalanced. I am thinking about put conservation constrain on qi and qc to resolve the issue. However, It is a great progress to make the cloud fraction right.
grgrm1_prcp_20190611
TS_C768L127_grgrm1_20190611
grgrm1_prcp_20190611

@@ -236,7 +237,8 @@ subroutine GFS_rrtmg_pre_run (im, levs, lm, lmk, lmp, n_var_lndp, &
LP1 = LM + 1 ! num of in/out levels


gridkm = sqrt(2.0)*sqrt(dx(1)*0.001*dx(1)*0.001)
ii_half = nint(0.5*LM)
gridkm = sqrt(dx(1)*0.001*dx(ii_half)*0.001)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gthompsnWRF I'm confused by this change. The dx variable is an array over the horizontal index but you're referencing with ii_half which is a function of the vertical domain size. Also, the original formulation had sqrt(2) presumably to calculate the diagonal length of the grid, which is now missing. Would you mind explaining this change? I realize that gridkm is only used in the progcld_thompson, so any effects are confined there, but in the spirit of good code reviewing, I'm trying to understand this.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doggone! That should have been variable IM not LM. Thanks for catching it! Yes, I dropped out the diagonal (sqrt(2)). I really do not know the DX spacing of a cube-sphere grid, but does DX vary a whole lot across 1..IM? Perhaps I don't even need to bother with the half-point and first-point average, I just thought it might better represent avg. grid spacing.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @gthompsnWRF. Regarding your question, I'm definitely not the authority to answer it, but my guess would be that it depends quite a bit on how big the "chunks" (IM) are and whether the columns are near a cubed sphere edge or not. Also, I don't think that we can assume that there is a specific horizontal relationship amongst the columns. I.e., using the midpoint index of the array (IM/2) doesn't necessarily mean that it will be in the "center" of the current chunk of columns in any physically-consistent sense. My guess would be that if you want one "average" gridkm value, you would actually need to calculate an average, which may be overkill for your use, I don't know.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am curious, will this impact the decomposition reproducibility? At the single point, it seems the gridkm will change if we have different decompositions (different mpi tasks) or different chunksize (IM).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it likely will. UGWP had exactly the same problem. I think the best solution would be to make gridkm a gridkm(i) with i=1...IM. That will also help with non-uniform grids such as stretched nests and regional grids that are not using the ESG (Extended Schmidt Gnomonic) grid. It's a small overhead but necessary.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As others have already mentioned, it is best to use the exact grid spacing - especially as CCPP has application beyond the UFS/GFS with the FV3 dycore.

As I haven't delved very far into the inner workings of CCPP metadata and variable definitions, is it always the case that dx will be provided/stored in meters?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not necessarily, it could be in km as well. Meters of course is the standard unit and makes a lot of sense, and it is also the unit in which the host model (fv3atm) stores the variable. If host model and scheme use different units, then CCPP can implement automatic unit conversions, as long as these unit conversions are implemented (for my example above, CCPP knows how to convert m to km and back). These conversions obviously introduce an overhead in the auto-generated caps, therefore it is best to use the same unit as the host model. And, if all host models used the same standard units, then there would be no unit conversions at all.

@grantfirl
Copy link
Collaborator

All changes look fine to me, although I had one question. I'll approve after @gthompsnWRF responds.

Copy link
Collaborator

@ericaligo-NOAA ericaligo-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks okay after the change from LM to IM.

@yangfanglin
Copy link
Collaborator

I think dx can be quite different for stretched grid; however, all current UFS applications are on uniform grids. @bensonr I added Rusty here to see if he can help answer the question.

@climbfuji climbfuji self-requested a review December 17, 2021 13:56
@gthompsnWRF
Copy link
Collaborator Author

I think dx can be quite different for stretched grid; however, all current UFS applications are on uniform grids. @bensonr I added Rusty here to see if he can help answer the question.

I appreciate the added info regarding the DX variable in FV3, but I really do not think the exactness of this variable is remotely a concern because of the impact it would have within the cloud fraction scheme would be hardly noticeable. So I made an ultra simplification and prefer to get this PR merged. It can easily be made more complex later on.

@@ -237,8 +236,7 @@ subroutine GFS_rrtmg_pre_run (im, levs, lm, lmk, lmp, n_var_lndp, &
LP1 = LM + 1 ! num of in/out levels


ii_half = nint(0.5*IM)
gridkm = sqrt(dx(1)*0.001*dx(ii_half)*0.001)
gridkm = dx(nint(0.5*IM))*0.001
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need to use gridkm(i) = dx(i)*0.001 or simply pass dx with units km in the CCPP metadata (unless it is used elsewhere in units m in the same routine) and use dx(i) directly.

@climbfuji
Copy link
Collaborator

I think dx can be quite different for stretched grid; however, all current UFS applications are on uniform grids. @bensonr I added Rusty here to see if he can help answer the question.

I appreciate the added info regarding the DX variable in FV3, but I really do not think the exactness of this variable is remotely a concern because of the impact it would have within the cloud fraction scheme would be hardly noticeable. So I made an ultra simplification and prefer to get this PR merged. It can easily be made more complex later on.

The problem is that the results can (will for non-uniform grids) differ if you change the MPI decomposition in the model setup. This is not acceptable for operations.

@gthompsnWRF
Copy link
Collaborator Author

I just changed gridkm. Hopefully this is accepted now.

Copy link
Collaborator

@climbfuji climbfuji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for agreeing to the gridkm change request.

@grantfirl
Copy link
Collaborator

Thanks, @gthompsnWRF, looks good to me. This is one of those instances where coding for generality (allowing for different horizontal grids/decompositions) within the CCPP requires a compromise. At this point, schemes within the CCPP really have to assume that every column within a horizontal chunk of columns is independent and no spatial relationship among them can be assumed.

@grantfirl
Copy link
Collaborator

@gthomsnWRF It looks like this is in the UFS merge queue for next week (tentatively 12/28). Will you be working during this time to update this to the latest main when it is its turn in the queue? Otherwise, I can create a wrapper for this PR in my fork that I can update as necessary.

@gthompsnWRF
Copy link
Collaborator Author

@grantfirl You can hold off a couple more days please. I think with some iterative work together with Anning, we might still want to tune a couple numbers in the code just a bit more. It might also depend on his scheduled PTO for the holiday since I don't know his plans. I'm taking off very little during the holidays so I'll be actively involved if Anning is still giving me some sensitivity experiment results. Thanks.

@grantfirl
Copy link
Collaborator

@gthompsnWRF @AnningCheng-NOAA Are we ready to return to this PR? There haven't been any commits since before the holidays, but there will be an opportunity within the next 10 days or so to get this merged into main if you would like.

@AnningCheng-NOAA
Copy link
Collaborator

AnningCheng-NOAA commented Jan 5, 2022 via email

@grantfirl
Copy link
Collaborator

Grant, I am testing one of the final modifications that Greg made.

On Tue, Jan 4, 2022 at 5:25 PM Grant Firl @.> wrote: @gthompsnWRF https://github.com/gthompsnWRF @AnningCheng-NOAA https://github.com/AnningCheng-NOAA Are we ready to return to this PR? There haven't been any commits since before the holidays, but there will be an opportunity within the next 10 days or so to get this merged into main if you would like. — Reply to this email directly, view it on GitHub <#809 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALQPMIKQIYZMGCQ3GM7YZYTUUNXVJANCNFSM5J7IPNLQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you were mentioned.Message ID: @.>

Thanks @AnningCheng-NOAA . Please, either you or @gthompsnWRF let me know via email or a comment on this pull request when you'd like to get this in the UFS commit queue so that I can try to schedule it.

@gthompsnWRF
Copy link
Collaborator Author

@grantfirl I just pushed my latest copy - the one tested by Anning, so I believe this PR is ready to schedule into UFS. Thanks!

@grantfirl
Copy link
Collaborator

@grantfirl I just pushed my latest copy - the one tested by Anning, so I believe this PR is ready to schedule into UFS. Thanks!

OK, @gthompsnWRF and @AnningCheng-NOAA . Thanks for the work and update. I'll go ahead and create upstream PRs in fv3atm and ufs-weather-model (assuming that those don't require any changes other than updating submodule pointers in those repos) in order to get this scheduled for merge. It's looking like this will get worked on by the UFS code managers early next week.

I'll run a set of regression tests on Hera/Intel. Based on the code changes, I would expect all tests that use Thompson MP to need changed baselines.

@grantfirl
Copy link
Collaborator

@gthompsnWRF @AnningCheng-NOAA It looks like the only failed RTs are those that use Thompson MP and all failures are in check_result. This is the expected result given the code changes.

Next week, when it is this PR's turn to get merged, we'll need @gthompsnWRF to update his PR branch to the latest ccpp-physics main. I'll try to remind you when this needs to be done.

@grantfirl
Copy link
Collaborator

@gthompsnWRF This is the next ccpp-physics PR to be merged. If you would please merge the latest ccpp-physics main into your PR branch and push to GitHub, I'll update the upstream repos to prepare for regression testing and merging.

@grantfirl grantfirl merged commit a37ea20 into NCAR:main Jan 18, 2022
junwang-noaa pushed a commit to NOAA-EMC/fv3atm that referenced this pull request Jan 18, 2022
*  Improve cloud fraction when using Thompson MP. See NCAR/ccpp-physics#809 for more details.
junwang-noaa pushed a commit to ufs-community/ufs-weather-model that referenced this pull request Jan 18, 2022
* improves cloud fraction when using Thompson MP. See NCAR/ccpp-physics#809 for more details.
davegill pushed a commit to wrf-model/WRF that referenced this pull request Jan 20, 2022
…-MP and icloud3 (cloud fraction) scheme (#1626)

TYPE: enhancement

KEYWORDS: microphysics, clouds, cloud fraction, Thompson microphysics

SOURCE:  Greg Thompson (UCAR/Joint Center for Satellite Data Assimilation)

DESCRIPTION OF CHANGES:
The purpose of these changes is to produce better cloud coverage and radiation amounts per tuning done with 
GFS global model and comparisons to observations.

Problem:
The Thompson-MP code (from ccpp-physics) has been used within UFS/GFS/FV-3 global model for multiple 
day simulations.  Anning Cheng and Ruiyu Sun (NCEP/EMC) have performed most of the simulations and 
analysis and iterated with me to make adjustments to the Thompson-MP.  At first, EMC found that high 
clouds were not present enough, particularly in the tropics.  Once switching to a larger upper limit of cloud 
ice before it becomes snow, the cloud amounts improved but some clouds and radiation balances were still 
not matching results from FV3's GFDL microphysics (which had already been tuned to observations).  
Therefore an effort to account for subgrid-scale clouds using the cloud fraction scheme (icloud=3) was 
adopted to GFS and tested with various parameter settings.

Solution:
The biggest change is in how cloud ice converts to snow at a threshold size of 300 microns (up from 200 
microns) and for rimed snow to convert to graupel (changed from 250 to 350 microns).  A change to allowed 
max size of ice means that the fall velocity constant for ice was changed to keep it aligned with snow at the 
same cut-over size (of 300 microns).  In addition, the cloud fraction scheme (icloud=3) can further improve 
the clouds and radiation together with Thompson-MP due to the overall under-prediction of clouds.  The 
icloud3 option was also making far too many clouds when compared to observations so its tuning knobs 
were adjusted until attaining the following overall improvements compared to cloud and radiation global 
climatologies that EMC uses:  cloud amounts of low, middle, high, and total cloud coverage, longwave 
radiation outgoing at top-of-atmosphere, and shortwave radiation reaching the ground.

ISSUE:  
There are corresponding issues or pull requests in the ccpp-physics repo, 
[778](NCAR/ccpp-physics#778)
[781](NCAR/ccpp-physics#781)
[809](NCAR/ccpp-physics#809)

LIST OF MODIFIED FILES: 
M  module_mp_thompson.F
M  module_radiation_driver.F
M  module_ra_rrtmg_lw.F
M  module_ra_rrtmg_lwf.F
M  module_ra_rrtmg_sw.F
M  module_ra_rrtmg_swf.F

TESTS CONDUCTED: 
1. A series of tests in GFS including 5, 7, 16, and 30-day long simulations compared to known cloud and 
radiation climatologies. The plots attached here in this comment were created by Anning Cheng. The numbers 
at the top of each panel represent global 3-day (days 3, 4, 5) average.

The first plot is the high/mid/low/total cloud amount.  The tunings in this PR reduced the high cloud amount 
from over 55% down to the low 40s, which matches observations of global cloud coverage pretty well.  And, 
another comparison is without the cloud fraction scheme, the mid-level clouds (and low/high) are simply less 
than observations, which is something well published in IPCC reports of global model clouds, especially the 
Southern Ocean and east sides of ocean basins (low stratus).
![gfnl_cf_20190611](https://user-images.githubusercontent.com/35609171/149175845-585914ce-84d4-4d72-a7e2-e96a5cb9d228.png)

The next plot's panel (a) shows the outgoing longwave radiation.  The target value global average is about 
240 W/m2.  Without the cloud fraction scheme and changes to ```D0s``` and ```D0g```, the result would be 
too much outgoing longwave since it will not contain enough clouds at lower temperatures.  Also, the older 
version of the cloud fraction scheme with its excessive amount of high clouds would make the outgoing 
radiation closer to 225 W/m2.  So the tuning of the scheme brings the results closer to observations.

![gfnl_prcp_20190611](https://user-images.githubusercontent.com/35609171/149176400-41f8b655-3d58-4cc4-adb0-6f6037beed52.png)

Lastly, the final plot shows the downward longwave reaching the ground (panel c) and downward shortwave 
reaching the ground (panel d). 

![gfnl_sfc_SWLW_20190611](https://user-images.githubusercontent.com/35609171/149177060-964ce8bd-02dc-4359-ad5d-37502b2d0f70.png)

2. Jenkins tests are passing.

RELEASE NOTE: Update of the Thompson microphysics scheme and cloud fraction scheme (icloud=3) to match the observations better. The modifications include updates to RRTMG LW and SW, and RRTMG fast LW and SW.
SamuelTrahanNOAA added a commit to NOAA-GSL/fv3atm that referenced this pull request Mar 8, 2022
* Feature/ccpp codeowners (NOAA-EMC#442)

* Per-file CODEOWNERS in ccpp/physics to set up automatic review requests

* Code cleanup. Remove used code/variables. Fix minor inconsistencies. (NOAA-EMC#440)

* Remove ESMF Alarm and TimeInterval variables from module_fv3_config.
* Variables nfhmax, nfhmax_hf are unused outside InitializeAdvertise. Declared them as local variables.
* There is no need to keep duplicates of all time related variables in atm_int_state when we can easily access them from atm_int_state%Atm.
* Remove redundant call to fms_init
* Add few missing ESMF_LogFoundError checks in module_fcst_grid_comp.F90
* Delete time_utils.F90. Unused.
* print only actual errors to stderr, everything else to stdout
* Move realizeConnectedCplFields to module_cplfields from module_cap_cpl
* Declare Atmos as module variable, and remove atmos_internalstate_wrapper
* Move code from clock_cplIntval to InitializeAdvertise
* Removed INTERNAL_FILE_NML from atmos_model.F90

* CCPP cloud cover change for Thompson MP associated with Xu Randall (NOAA-EMC#443)

* Remove old comments from GFS_typedefs.F90.
* Update logic that sets Model%cnvcld = .false.

* Wrapper for ccpp-physics NOAA-EMC#806, NOAA-EMC#807, NOAA-EMC#813 (NOAA-EMC#447)

* CCPP physics updates from PR NOAA-EMC#806(only diagnostic changes for RRTMGP - small diag-only RT impact for tests involving RRTMGP),NOAA-EMC#807 (only aborting model if sfc emis file not present when needed) ,NOAA-EMC#813(only SCM-specific physics changes)

* Wrapper for ccpp-physics NOAA-EMC#808 and 816 (roughness length over ice and NoahMP tsurf bugfix) (NOAA-EMC#452)

This PR contains the ccpp physics PR NOAA-EMC#808 and NOAA-EMC#816. 808 addresses an error in the momentum roughness length over tiles with ice. 816 fixes an occasional segfault bug related to the tsurf variable in NoahMP and updates to "improve snow simulation in NoahMP for P8".

* Use 'model set run clock' routine in FV3 NUOPC cap. (NOAA-EMC#450)

* Add 'SetRunClock' specialization routine to FV3 NUOPC cap, which sets fv3 model clock. FV3 cap does not keep local copy of clock (clock_fv3) as saved module variable anymore.
* Minor code cleanup.

* Clean up iovr=4 (exponential cloud overlapping method) in RRTMG (NOAA-EMC#445)

Clean up the exponential cloud overlapping method in RRTMG, which reflects the practice of the pre-2018 operational HWRF model.

* Bug fix for dimensions of eta level variables and WAM variables in Fortran code and CCPP metadata (NOAA-EMC#431)

* Add additional diagnostic arrays for radiation-microphysics development
* Bug fixes for WAM model runs with levr < levs

* Update inline post with latest UPP release upp_v10.0.11 (NOAA-EMC#449)

* Update UPP revision
* Add foundation temperaure in GFS read interface for inline post.

* Wrapper for ccpp-physics NOAA-EMC#812 (NOAA-EMC#453)

* update submodule pointer for regression testing of ccpp-physics#812

* MYNN sfclay (RAP suite) restart reproducibility, P8 suite definition files (NOAA-EMC#455)

* Fix uninitialized variable zmtnblck in ccpp/data/GFS_typedefs.F90
* Fix typo in CCPP standard name for ncnvwind in ccpp/data/GFS_typedefs.meta
* Add ten 2d variables required for MYNNSFC restart reproducibility to ccpp/driver/GFS_restart.F90
* create initial p8 suites, P8 initial SDFs as copies of FV3_GFS_v16_coupled_nsstNoahmpUGWPv1
and FV3_GFS_v16_nsstNoahmpUGWPv1

Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>

* Fixes on initializing snow depth over ice and changes z0ice (NOAA-EMC#461)

* modify FV3GFS_io.F90 by fixing errors associated with initializing snow depth over ice in the case where both land and water coexist (i.e. fractional grid case)
* z0ice is changed to 1.0 cm from 1.1cm in atmos_model.F90

* Radar-derived microphysics temperature tendencies similar to operational HRRR (NOAA-EMC#457)

- implements a feature of the operational HRRR, radar-derived microphysics temperature tendencies applied in the first N minutes of the forecast to improve clouds in the first few hours.
* HRRR-like radar-derived temperature tendencies
* Give a warning when convection is enabled with radar tten
* Fix uninitialized variable zmtnblck in ccpp/data/GFS_typedefs.F90
* Add ten 2d variables required for MYNNSFC restart reproducibility to ccpp/driver/GFS_restart.F90

* fixing snod bug in atmos_model.F90 (NOAA-EMC#465)

* Thompson MP cloud tuning (NOAA-EMC#463)

*  Improve cloud fraction when using Thompson MP. See NCAR/ccpp-physics#809 for more details.

* Feature/hwrf legacy (NOAA-EMC#459)

* Added a new suite definition file: suite_FV3_HAFS_v0_thompson_tedmf_gfdlsf.xml. This SDF is a legacy hwrf SDF but only the GFDL surface layer scheme and the Thompson scheme are kept.

* CCPP: Update consistency checks and effective radii calculation for Thompson MP (NOAA-EMC#462)

* This PR updates the submodule pointer for ccpp-physics for the changes described in NCAR/ccpp-physics#786 (Update consistency checks and effective radii calculation for Thompson MP)

* Unified write_netcdf, add single file 'cubed_sphere_grid' output (NOAA-EMC#466)

* Unify write_netcdf and write_netcdf_parallel modules.
* Add support for writing 'cubed_sphere_grid' history files in a single netcdf file
* Code refactoring and cleanup.

* Change standard name and unit of CCPP error flag variable in CCPP framework and physics (NOAA-EMC#467)

*Change standard name and unit of CCPP error flag variable in CCPP framework and physics.

* Add code to enable ESMF managed threading (NOAA-EMC#469)

* Implement ESMF-managed-threading for fcstComp and wrtComp's.

* Revisions to repair iovr=5 cloud overlap option (NOAA-EMC#471)

*This PR addresses part 2 of CCPP issue NOAA-EMC#748 to activate the exponential-random cloud overlap method (iovr=5) in RRTMG.

* Add 2d decomposition on the write grid comp (NOAA-EMC#470)

* update write grid comp to have 2D decomposition on output grid.
* combine PR#468: Bug fix in FV3GFS_io.F90 for allocation of temp2d

Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>

* 4DIAU bug when iau_filter_increments=T  (NOAA-EMC#458)

* Add lsm_cold_start variable for RUC LSM SCM support and gwdps bugfix (combined) (NOAA-EMC#475)

* add lsm_cold_start variable in GFS_typedefs.F90/meta and update ccpp/physics for testing
* includes @SMoorthi-emc 's bugfix for gwdps.f.

* add lon info in the write group (NOAA-EMC#476)

* Multiple output grids (NOAA-EMC#480)

Update fv3 cap and write grid component to enable outputting multiple domains.
This is done be creating an array of fcstGrids, and array of rout handles where each element of these arrays correspond to one atm domain.
In the write grid component updates were made to allow grid spec parameters for each output grid to be specified separately.

Co-authored-by: Gerhard Theurich <theurich@sourcespring.net>

* GPU-enabled version of Grell-Freitas convection in ccpp-physics (NOAA-EMC#479)

* Enable Thompson MP when coupling with UFS-Aerosols (NOAA-EMC#484)

* Add support for Stochastically Perturbed Parameterizations (SPP) in FV3 and add the FV3_RRFS_v1alpha SDF. (NOAA-EMC#454)

Adds the necessary code in fv3atm to allow for Stochastically Perturbed Parameterizations (SPP) in a set of RAP/HRRR-based physics parameterizations. Specific to the fv3atm repository, code in this PR defines the necessary variables associated with various SPP-related fields (e.g., logical to activate SPP, parameterization-specific SPP variables, etc.) that are then passed to ccpp-physics.

* lateral boundary fix for regional runs (NOAA-EMC#482)

* Updating pointer to ccpp/physics.

* Point to Christina's branches.

* Point to NOAA-GSL gsl/develop for ccpp/physics

Co-authored-by: Samuel Trahan (NOAA contractor) <39415369+SamuelTrahanNOAA@users.noreply.github.com>
Co-authored-by: Dusan Jovic <48258889+DusanJovic-NOAA@users.noreply.github.com>
Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>
Co-authored-by: Grant Firl <grant.firl@noaa.gov>
Co-authored-by: mzhangw <mzhangw@gmail.com>
Co-authored-by: WenMeng-NOAA <48260754+WenMeng-NOAA@users.noreply.github.com>
Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>
Co-authored-by: SMoorthi-emc <47667426+SMoorthi-emc@users.noreply.github.com>
Co-authored-by: ChunxiZhang-NOAA <49283036+ChunxiZhang-NOAA@users.noreply.github.com>
Co-authored-by: Jun Wang <37633869+junwang-noaa@users.noreply.github.com>
Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Jeff Whitaker <jswhit@fastmail.fm>
Co-authored-by: Gerhard Theurich <theurich@sourcespring.net>
Co-authored-by: DomHeinzeller <58610420+DomHeinzeller@users.noreply.github.com>
Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>
Co-authored-by: JeffBeck-NOAA <55201531+JeffBeck-NOAA@users.noreply.github.com>
Co-authored-by: MatthewPyle-NOAA <48285220+MatthewPyle-NOAA@users.noreply.github.com>
Co-authored-by: samuel.trahan <Samuel.Trahan@noaa.gov>
SamuelTrahanNOAA added a commit to NOAA-GSL/ufs-weather-model that referenced this pull request Mar 8, 2022
* Add ufs driver. Remove NEMS submodule. Update CMEPS. Update CDEPS. Feature/ort addition.  (ufs-community#931)

* Add UFS driver, a copy of two fortran files from the NEMS repository. Remove NEMS submodule.

* Update CMEPS for current escomp/master. Implement the ability to write mediator history files for ATM on tiles rather than mesh

* Update CDEPS component. The updated CDEPS includes code updates for GFS v16 data mode.

* Operation requirement test (ORT) for:
  - cpld_control_p7 thread and decomposition tests
  - cpld_bmark_p7 thread and decomposition tests
  - Due to the wave component in these tests, restart and debug do not work

Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com>
Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>
Co-authored-by: BinLi-NOAA <bin.li@noaa.gov>

* fv3atm cap cleanup (ufs-community#954)

FV3 nuopc cap code cleanup. Remove used code/variables. Fix minor inconsistencies.

* CCPP cloud cover change for Thompson MP associated with Xu Randall, several bug fixes for regression testing scripts (ufs-community#962)

This PR contains code changes 1) to remove old comments in GFS_typedefs.F90, increase cloud cover for Thompson MP by passing convective cloud water mixing ratio to progcld6. 2) Updates for auto fixes

Co-authored-by: Dusan Jovic <dusan.jovic@noaa.gov>

* ccpp-physics ufs-community#806, #807, ufs-community#813, and ort modifications (ufs-community#970)

- CCPP Physics changes: see Wrapper for ccpp-physics ufs-community#806, #807, ufs-community#813, NOAA-EMC/fv3atm#447.
- UFS weather model changes for ORT updates following changes in Fix Auto-RT on Gaea, Jet and Orion ufs-community#965

* Wrapper for ccpp-physics ufs-community#808 and 816 (roughness length over ice and NoahMP tsurf bugfix) (ufs-community#971)

* update submodule pointer for regression testing for ccpp-physics ufs-community#808,816: roughness length over ice and NoahMP tsurf bugfix
* Reduce time step from 600s to 450s for tests control_csawmg and control_csawmg_debug

Co-authored-by: Dom Heinzeller <climbfuji@ymail.com>
Co-authored-by: climbfuji <dom.heinzeller@icloud.com>

* Use 'model set run clock' routine in FV3 NUOPC cap. (ufs-community#967)

* Update FV3 cap by adding 'model set run clock' specialization routine. Because FV3 cap does not modify 'Earth clock' anymore in case of a restarted run, that adjustment is now done in the UFS driver.
* Fix model_configure file for two tests to define `fhrot` parameter.

* Revert iovr=4 (exponential cloud overlap method) in RRTMG (ufs-community#963)

Revert/cleanup iovr=4 in RRTMG per developer's request. The current exponential cloud overlapping method used in RRTMG represents the method used in the pre-2018 operational HWRF model.

* Add fhzero ort test (ufs-community#977)

* Fhzero ORT applies to the two tests `control` and `control_p7`
* Test can be run by invoking, for example, `./opnReqTest -n control -c fhz`

* Add a new CDEPS test to use GSFv16 data mode in CDEPS ,update rt_weekly.conf and update MOM6 to main branch (20211220) (ufs-community#976)

* add a new cdeps test using GFS data mode
* update rt_weekly.conf to include cpld_bmark_p7_35d tests
* update MOM6

Co-authored-by: jiandewang <jiande.wang@noaa.gov>

* Bug fixes for WAM physics and regression tests, remove duplicate modules from ccpp-physics CMakeLists.txt (ufs-community#947)

*adds a new regression test control_wam_debug and fixes problems with the existing control_wam test (e.g. IAER was not set as intended).

* Update global inline post for outputting foundation temperature (ufs-community#968)

Update upp submodule in fv3atm with latest UPP version for outputting foundation temperature and fixing land-sea mask issue in global inline post.

* Regression test bug fixes for running on Jet or with Rocoto (ufs-community#981)

* Bug fixes for running on jet or with rocoto
1. Correctly detect Jet when deciding build job runtimes
2. Do not delete out and err files in compile & test jobs
3. Correct dependencies for WW3 in Rocoto
4. Do not hard-code ACCNR to h-nems
This PR also contains fv3/ccpp physics (CCPP PR#812) updates that implements a vertically-integrated formula to properly compute tracer column mass and ensure mass conservation for all tracers except TKE. See issue NCAR/ccpp-physics#811 for details

* MYNN sfclay (RAP suite) restart reproducibility (ufs-community#984)

*This PR turns on the rap_restart tests. Bit-for-bit reproducibility is achieved with the bug fixes for MYNN sfclay in the associated fv3atm and ccpp-physics PRs.

* fixes on initializing snow over fractional ice and GFS_surface_composite and z0ice value change (ufs-community#996)

* Radar-derived microphysics temperature tendencies similar to operational HRRR (ufs-community#986)

This PR implements a feature of the operational HRRR, radar-derived microphysics temperature tendencies applied in the first N minutes of the forecast to improve clouds in the first few hours. Two regression tests are added.

* Turn on CICE albedo use by ATM for coupled model; update Prototype test suite to initial P8 configuration;  (ufs-community#944)

* add updates for suites, tests for initial P8 configuration
* add IOPT_SFC=3 as default; set IOPT_SFC=1 in p7 tests
* change min_seaice to 1e-6; set to 1e-11 in p7 tests
* set dddmp=0.1 in both P7 and P8. This matches the final
P7 configuration. It will change P7 baselines.
* set cdmbwd in both P7 and P8 to the final P7 configuration.
(1.0,2.2,1.0,1.0) for all resolutions. It will change P7 baselines.
* add dusan's ecflow fix for wcoss2
* update FV3 with Moorthis' snod fix
* update rt_weekly to use p8 tests
* change integration length to 5 days

* Thompson MP cloud tuning (ufs-community#1005)

* improves cloud fraction when using Thompson MP. See NCAR/ccpp-physics#809 for more details.

* Add a new HAFS regression test with Thompson MP and GFDL surface layer schemes and Corrected S4 template regression testing job card (ufs-community#995)

* Added a new regression test for HAFS. 
* Fixed the S4 RT template job card.

Co-authored-by: David Huber <david.huber@noaa.gov>
Co-authored-by: Bin Liu <Bin.Liu@noaa.gov>

* CCPP: Update consistency checks and effective radii calculation for Thompson MP (ufs-community#1000)

This PR contains changes in ccpp-physics (Update consistency checks and effective radii calculation for Thompson MP) and adds two new regression tests for both Intel and GNU that use the new prognostic cloud scheme progcld_thompson with GFS v16 and with RAP. The new cloud scheme is activated by setting icloud == 3 in input.nml.

* Rremove nems_datm related code, update CDEPS, and remove run directories that are no longer needed by other tests (ufs-community#1008)

* Add fhzero ort to coupled tests (ufs-community#1009)

* Add fhz ort to cpld tests
* Set ACCNR in rt.sh, not detect_machine.sh. Add some comments explaining where the variable should not be set or used.

Co-authored-by: Samuel Trahan <Samuel.Trahan@noaa.gov>

* Unified write_netcdf, add single file 'cubed_sphere_grid' output (ufs-community#1011)

* Unify write_netcdf and write_netcdf_parallel modules.
* Add support for writing 'cubed_sphere_grid' history files in a single netcdf file.

* Change standard name and unit of CCPP error flag variable in CCPP framework and physics (ufs-community#1013)

* updates the submodule pointers for fv3atm, ccpp-framework and ccpp-physics for:  Change standard name and unit of CCPP error flag variable in CCPP framework and physics.

* ESMF managed threading (ufs-community#1018)

Add code updates to enable the ESMF managed threading in UFS.


Co-authored-by: Gerhard Theurich <gerhard.j.theurich@noaa.gov >

* sync-ww3 (ufs-community#1026)

* sync with ww3 develop branch

* Revisions to repair iovr=5 cloud overlap option (ufs-community#1025)

* This PR addresses part 2 of issue ufs-community#748 to activate the exponential-random cloud overlap method (iovr=5) in RRTMG. RT tests were added for  iovr=4 and iovr=5 options
* change iovr to use the environment variable in control.nml.IN; modify all tests that use control.nml.IN to set env. var. iovr=3 (except for new tests); remove restart files from new tests

* 2D decomposition on write grid, bug fix in FV3GFS_IO.F90, and quad precision in stochastic physics (ufs-community#1028)

- The 2D decomposition is enabled on write grid component.
- FV3atm PR NOAA-EMC/fv3atm#468 from @MicroTed 
- stochastic physics PR NOAA-PSL/stochastic_physics#53 from Dom.

Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Dom Heinzeller <heinzell@ucar.edu>

* Update CMEPS for wave coupling (ufs-community#1034)


* add run phases to allow waves to run in the slow loop
* re-order hafs wave test
* update aliases in fd_nems.yaml
* update CMEPS to prevent advertise conflicts when s2sw is
still using connectors
* add correct masking variables to hafs nems.configure files

* 4DIAU bug fix when iau_filter_increments=T (ufs-community#990)

* Add lsm_cold_start variable for RUC LSM SCM support and gwdps bugfix (combined) (ufs-community#1037)

Co-authored-by: @SMoorthi-emc

* Update esmf 821b04 (ufs-community#1041)

* Update the version of ESMF library to 8.2.1b04
* Add the write group lon info for upp 2d decomposition test.

* Multiple output (nested) grids from fv3atm (ufs-community#1044)

Update fv3atm to support writing multiple history files for parent and nested domains.
Nested HAFS regression tests were updated to write out history files from all nested domains.

Co-authored-by: Bin Liu <bin.liu@noaa.gov>

* GPU-enabled version of Grell-Freitas convection in ccpp-physics (ufs-community#1043)

* Enable Thompson microphysics with UFS-Aerosols and update GOCART (ufs-community#1049)

This update allows the Thompson microphysics scheme to be used with prognostic aerosols (UFS-Aerosols) and updates GOCART to version 2.0.2.

* Add support for Stochastically Perturbed Parameterizations (SPP) in FV3 (ufs-community#982)

* Update CICE; add support for restart test when running single RT (was PR 1058); update MOM6 (was PR 1060) (ufs-community#1045)


* update cice
* (1) pointing MOM6 to testing branch (MOM6 main repo PR: mom-ocean/MOM6#1556 (comment))
    (2) add three new files in mom_cmake file
            core/MOM_porous_barriers.F90
            external/drifters/MOM_particles.F90
            external/drifters/MOM_particles_types.F90
    (3) explicitly set in MOM_input for all resolutions to reflect the default value changes in code to retain answer
        EPBL_MLD_BISECTION = True (default changed from T to F in this MOM6 PR)
        INTERNAL_WAVE_SPEED_BETTER_EST = False (default changed from F to T in this MOM6 PR)
        note BBL_USE_EOS now set to use the value of USE_EOS in this MOM6 PR,
        but we set BBL_USE_EOS explicit so it has no impact

* remove duplicated CHL input setting in MOM_input_template_050
* add support for restart tests while running single RT

* lateral boundary fix for regional runs (ufs-community#1050)

This PR fixes a transient increase in noise caused by incorrect processing of winds along the lateral boundary on the final acoustic time step before fresh boundary information is ingested into the model.

* Update pointer to FV3

* Point to Christina's branches.

* Update RT log for Hera Intel.

* Adding RT log for Hera GNU tests.

* Updating RT log for Jet Intel tests.

* Latest regression tests on Jet.

* Adding regression logs.

* Updating regression test date stamp.

* Point to GSL repositories for FV3 and FV3/ccpp/physics

Co-authored-by: Dusan Jovic <48258889+DusanJovic-NOAA@users.noreply.github.com>
Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com>
Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>
Co-authored-by: BinLi-NOAA <bin.li@noaa.gov>
Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>
Co-authored-by: Dusan Jovic <dusan.jovic@noaa.gov>
Co-authored-by: Grant Firl <grant.firl@noaa.gov>
Co-authored-by: Dom Heinzeller <climbfuji@ymail.com>
Co-authored-by: mzhangw <mzhangw@gmail.com>
Co-authored-by: jiandewang <jiande.wang@noaa.gov>
Co-authored-by: WenMeng-NOAA <48260754+WenMeng-NOAA@users.noreply.github.com>
Co-authored-by: Samuel Trahan (NOAA contractor) <39415369+SamuelTrahanNOAA@users.noreply.github.com>
Co-authored-by: SMoorthi-emc <47667426+SMoorthi-emc@users.noreply.github.com>
Co-authored-by: ChunxiZhang-NOAA <49283036+ChunxiZhang-NOAA@users.noreply.github.com>
Co-authored-by: David Huber <david.huber@noaa.gov>
Co-authored-by: Bin Liu <Bin.Liu@noaa.gov>
Co-authored-by: Samuel Trahan <Samuel.Trahan@noaa.gov>
Co-authored-by: Jun Wang <37633869+junwang-noaa@users.noreply.github.com>
Co-authored-by: Gerhard Theurich <gerhard.j.theurich@noaa.gov >
Co-authored-by: Ali.Abdolali <37336972+aliabdolali@users.noreply.github.com>
Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Dom Heinzeller <heinzell@ucar.edu>
Co-authored-by: Jeff Whitaker <jswhit@fastmail.fm>
Co-authored-by: DomHeinzeller <58610420+DomHeinzeller@users.noreply.github.com>
Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>
Co-authored-by: JeffBeck-NOAA <55201531+JeffBeck-NOAA@users.noreply.github.com>
Co-authored-by: MatthewPyle-NOAA <48285220+MatthewPyle-NOAA@users.noreply.github.com>
SamuelTrahanNOAA added a commit to NOAA-GSL/fv3atm that referenced this pull request Mar 15, 2022
* Merge NOAA-EMC develop into gsl/develop (#126)

* Feature/ccpp codeowners (NOAA-EMC#442)

* Per-file CODEOWNERS in ccpp/physics to set up automatic review requests

* Code cleanup. Remove used code/variables. Fix minor inconsistencies. (NOAA-EMC#440)

* Remove ESMF Alarm and TimeInterval variables from module_fv3_config.
* Variables nfhmax, nfhmax_hf are unused outside InitializeAdvertise. Declared them as local variables.
* There is no need to keep duplicates of all time related variables in atm_int_state when we can easily access them from atm_int_state%Atm.
* Remove redundant call to fms_init
* Add few missing ESMF_LogFoundError checks in module_fcst_grid_comp.F90
* Delete time_utils.F90. Unused.
* print only actual errors to stderr, everything else to stdout
* Move realizeConnectedCplFields to module_cplfields from module_cap_cpl
* Declare Atmos as module variable, and remove atmos_internalstate_wrapper
* Move code from clock_cplIntval to InitializeAdvertise
* Removed INTERNAL_FILE_NML from atmos_model.F90

* CCPP cloud cover change for Thompson MP associated with Xu Randall (NOAA-EMC#443)

* Remove old comments from GFS_typedefs.F90.
* Update logic that sets Model%cnvcld = .false.

* Wrapper for ccpp-physics NOAA-EMC#806, NOAA-EMC#807, NOAA-EMC#813 (NOAA-EMC#447)

* CCPP physics updates from PR NOAA-EMC#806(only diagnostic changes for RRTMGP - small diag-only RT impact for tests involving RRTMGP),NOAA-EMC#807 (only aborting model if sfc emis file not present when needed) ,NOAA-EMC#813(only SCM-specific physics changes)

* Wrapper for ccpp-physics NOAA-EMC#808 and 816 (roughness length over ice and NoahMP tsurf bugfix) (NOAA-EMC#452)

This PR contains the ccpp physics PR NOAA-EMC#808 and NOAA-EMC#816. 808 addresses an error in the momentum roughness length over tiles with ice. 816 fixes an occasional segfault bug related to the tsurf variable in NoahMP and updates to "improve snow simulation in NoahMP for P8".

* Use 'model set run clock' routine in FV3 NUOPC cap. (NOAA-EMC#450)

* Add 'SetRunClock' specialization routine to FV3 NUOPC cap, which sets fv3 model clock. FV3 cap does not keep local copy of clock (clock_fv3) as saved module variable anymore.
* Minor code cleanup.

* Clean up iovr=4 (exponential cloud overlapping method) in RRTMG (NOAA-EMC#445)

Clean up the exponential cloud overlapping method in RRTMG, which reflects the practice of the pre-2018 operational HWRF model.

* Bug fix for dimensions of eta level variables and WAM variables in Fortran code and CCPP metadata (NOAA-EMC#431)

* Add additional diagnostic arrays for radiation-microphysics development
* Bug fixes for WAM model runs with levr < levs

* Update inline post with latest UPP release upp_v10.0.11 (NOAA-EMC#449)

* Update UPP revision
* Add foundation temperaure in GFS read interface for inline post.

* Wrapper for ccpp-physics NOAA-EMC#812 (NOAA-EMC#453)

* update submodule pointer for regression testing of ccpp-physics#812

* MYNN sfclay (RAP suite) restart reproducibility, P8 suite definition files (NOAA-EMC#455)

* Fix uninitialized variable zmtnblck in ccpp/data/GFS_typedefs.F90
* Fix typo in CCPP standard name for ncnvwind in ccpp/data/GFS_typedefs.meta
* Add ten 2d variables required for MYNNSFC restart reproducibility to ccpp/driver/GFS_restart.F90
* create initial p8 suites, P8 initial SDFs as copies of FV3_GFS_v16_coupled_nsstNoahmpUGWPv1
and FV3_GFS_v16_nsstNoahmpUGWPv1

Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>

* Fixes on initializing snow depth over ice and changes z0ice (NOAA-EMC#461)

* modify FV3GFS_io.F90 by fixing errors associated with initializing snow depth over ice in the case where both land and water coexist (i.e. fractional grid case)
* z0ice is changed to 1.0 cm from 1.1cm in atmos_model.F90

* Radar-derived microphysics temperature tendencies similar to operational HRRR (NOAA-EMC#457)

- implements a feature of the operational HRRR, radar-derived microphysics temperature tendencies applied in the first N minutes of the forecast to improve clouds in the first few hours.
* HRRR-like radar-derived temperature tendencies
* Give a warning when convection is enabled with radar tten
* Fix uninitialized variable zmtnblck in ccpp/data/GFS_typedefs.F90
* Add ten 2d variables required for MYNNSFC restart reproducibility to ccpp/driver/GFS_restart.F90

* fixing snod bug in atmos_model.F90 (NOAA-EMC#465)

* Thompson MP cloud tuning (NOAA-EMC#463)

*  Improve cloud fraction when using Thompson MP. See NCAR/ccpp-physics#809 for more details.

* Feature/hwrf legacy (NOAA-EMC#459)

* Added a new suite definition file: suite_FV3_HAFS_v0_thompson_tedmf_gfdlsf.xml. This SDF is a legacy hwrf SDF but only the GFDL surface layer scheme and the Thompson scheme are kept.

* CCPP: Update consistency checks and effective radii calculation for Thompson MP (NOAA-EMC#462)

* This PR updates the submodule pointer for ccpp-physics for the changes described in NCAR/ccpp-physics#786 (Update consistency checks and effective radii calculation for Thompson MP)

* Unified write_netcdf, add single file 'cubed_sphere_grid' output (NOAA-EMC#466)

* Unify write_netcdf and write_netcdf_parallel modules.
* Add support for writing 'cubed_sphere_grid' history files in a single netcdf file
* Code refactoring and cleanup.

* Change standard name and unit of CCPP error flag variable in CCPP framework and physics (NOAA-EMC#467)

*Change standard name and unit of CCPP error flag variable in CCPP framework and physics.

* Add code to enable ESMF managed threading (NOAA-EMC#469)

* Implement ESMF-managed-threading for fcstComp and wrtComp's.

* Revisions to repair iovr=5 cloud overlap option (NOAA-EMC#471)

*This PR addresses part 2 of CCPP issue NOAA-EMC#748 to activate the exponential-random cloud overlap method (iovr=5) in RRTMG.

* Add 2d decomposition on the write grid comp (NOAA-EMC#470)

* update write grid comp to have 2D decomposition on output grid.
* combine PR#468: Bug fix in FV3GFS_io.F90 for allocation of temp2d

Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>

* 4DIAU bug when iau_filter_increments=T  (NOAA-EMC#458)

* Add lsm_cold_start variable for RUC LSM SCM support and gwdps bugfix (combined) (NOAA-EMC#475)

* add lsm_cold_start variable in GFS_typedefs.F90/meta and update ccpp/physics for testing
* includes @SMoorthi-emc 's bugfix for gwdps.f.

* add lon info in the write group (NOAA-EMC#476)

* Multiple output grids (NOAA-EMC#480)

Update fv3 cap and write grid component to enable outputting multiple domains.
This is done be creating an array of fcstGrids, and array of rout handles where each element of these arrays correspond to one atm domain.
In the write grid component updates were made to allow grid spec parameters for each output grid to be specified separately.

Co-authored-by: Gerhard Theurich <theurich@sourcespring.net>

* GPU-enabled version of Grell-Freitas convection in ccpp-physics (NOAA-EMC#479)

* Enable Thompson MP when coupling with UFS-Aerosols (NOAA-EMC#484)

* Add support for Stochastically Perturbed Parameterizations (SPP) in FV3 and add the FV3_RRFS_v1alpha SDF. (NOAA-EMC#454)

Adds the necessary code in fv3atm to allow for Stochastically Perturbed Parameterizations (SPP) in a set of RAP/HRRR-based physics parameterizations. Specific to the fv3atm repository, code in this PR defines the necessary variables associated with various SPP-related fields (e.g., logical to activate SPP, parameterization-specific SPP variables, etc.) that are then passed to ccpp-physics.

* lateral boundary fix for regional runs (NOAA-EMC#482)

* Updating pointer to ccpp/physics.

* Point to Christina's branches.

* Point to NOAA-GSL gsl/develop for ccpp/physics

Co-authored-by: Samuel Trahan (NOAA contractor) <39415369+SamuelTrahanNOAA@users.noreply.github.com>
Co-authored-by: Dusan Jovic <48258889+DusanJovic-NOAA@users.noreply.github.com>
Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>
Co-authored-by: Grant Firl <grant.firl@noaa.gov>
Co-authored-by: mzhangw <mzhangw@gmail.com>
Co-authored-by: WenMeng-NOAA <48260754+WenMeng-NOAA@users.noreply.github.com>
Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>
Co-authored-by: SMoorthi-emc <47667426+SMoorthi-emc@users.noreply.github.com>
Co-authored-by: ChunxiZhang-NOAA <49283036+ChunxiZhang-NOAA@users.noreply.github.com>
Co-authored-by: Jun Wang <37633869+junwang-noaa@users.noreply.github.com>
Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Jeff Whitaker <jswhit@fastmail.fm>
Co-authored-by: Gerhard Theurich <theurich@sourcespring.net>
Co-authored-by: DomHeinzeller <58610420+DomHeinzeller@users.noreply.github.com>
Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>
Co-authored-by: JeffBeck-NOAA <55201531+JeffBeck-NOAA@users.noreply.github.com>
Co-authored-by: MatthewPyle-NOAA <48285220+MatthewPyle-NOAA@users.noreply.github.com>
Co-authored-by: samuel.trahan <Samuel.Trahan@noaa.gov>

* Remove GF Consistency Check & Add SDFs - V2 (#127)

* - Removes consistency check in GF, allows GF to run with different or no shallow scheme
- Add additional RAP based SDF

* Point to gsl/develop for ccpp/physics

Co-authored-by: Samuel Trahan <Samuel.Trahan@noaa.gov>

* Update to top of gsl/physics to get codeowners (#130)

* Put a comment back in the ccpp/physics.

* point to head of ccpp/physics RRFS_dev branch

Co-authored-by: Christina Holt <56881914+christinaholtNOAA@users.noreply.github.com>
Co-authored-by: Dusan Jovic <48258889+DusanJovic-NOAA@users.noreply.github.com>
Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>
Co-authored-by: Grant Firl <grant.firl@noaa.gov>
Co-authored-by: mzhangw <mzhangw@gmail.com>
Co-authored-by: WenMeng-NOAA <48260754+WenMeng-NOAA@users.noreply.github.com>
Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>
Co-authored-by: SMoorthi-emc <47667426+SMoorthi-emc@users.noreply.github.com>
Co-authored-by: ChunxiZhang-NOAA <49283036+ChunxiZhang-NOAA@users.noreply.github.com>
Co-authored-by: Jun Wang <37633869+junwang-noaa@users.noreply.github.com>
Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Jeff Whitaker <jswhit@fastmail.fm>
Co-authored-by: Gerhard Theurich <theurich@sourcespring.net>
Co-authored-by: DomHeinzeller <58610420+DomHeinzeller@users.noreply.github.com>
Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>
Co-authored-by: JeffBeck-NOAA <55201531+JeffBeck-NOAA@users.noreply.github.com>
Co-authored-by: MatthewPyle-NOAA <48285220+MatthewPyle-NOAA@users.noreply.github.com>
Co-authored-by: Hannah C Barnes <38660891+hannahcbarnes@users.noreply.github.com>
SamuelTrahanNOAA added a commit to NOAA-GSL/ufs-weather-model that referenced this pull request Mar 15, 2022
* Merge ufs-community develop into gsl/develop (#119)

* Add ufs driver. Remove NEMS submodule. Update CMEPS. Update CDEPS. Feature/ort addition.  (ufs-community#931)

* Add UFS driver, a copy of two fortran files from the NEMS repository. Remove NEMS submodule.

* Update CMEPS for current escomp/master. Implement the ability to write mediator history files for ATM on tiles rather than mesh

* Update CDEPS component. The updated CDEPS includes code updates for GFS v16 data mode.

* Operation requirement test (ORT) for:
  - cpld_control_p7 thread and decomposition tests
  - cpld_bmark_p7 thread and decomposition tests
  - Due to the wave component in these tests, restart and debug do not work

Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com>
Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>
Co-authored-by: BinLi-NOAA <bin.li@noaa.gov>

* fv3atm cap cleanup (ufs-community#954)

FV3 nuopc cap code cleanup. Remove used code/variables. Fix minor inconsistencies.

* CCPP cloud cover change for Thompson MP associated with Xu Randall, several bug fixes for regression testing scripts (ufs-community#962)

This PR contains code changes 1) to remove old comments in GFS_typedefs.F90, increase cloud cover for Thompson MP by passing convective cloud water mixing ratio to progcld6. 2) Updates for auto fixes

Co-authored-by: Dusan Jovic <dusan.jovic@noaa.gov>

* ccpp-physics ufs-community#806, #807, ufs-community#813, and ort modifications (ufs-community#970)

- CCPP Physics changes: see Wrapper for ccpp-physics ufs-community#806, #807, ufs-community#813, NOAA-EMC/fv3atm#447.
- UFS weather model changes for ORT updates following changes in Fix Auto-RT on Gaea, Jet and Orion ufs-community#965

* Wrapper for ccpp-physics ufs-community#808 and 816 (roughness length over ice and NoahMP tsurf bugfix) (ufs-community#971)

* update submodule pointer for regression testing for ccpp-physics ufs-community#808,816: roughness length over ice and NoahMP tsurf bugfix
* Reduce time step from 600s to 450s for tests control_csawmg and control_csawmg_debug

Co-authored-by: Dom Heinzeller <climbfuji@ymail.com>
Co-authored-by: climbfuji <dom.heinzeller@icloud.com>

* Use 'model set run clock' routine in FV3 NUOPC cap. (ufs-community#967)

* Update FV3 cap by adding 'model set run clock' specialization routine. Because FV3 cap does not modify 'Earth clock' anymore in case of a restarted run, that adjustment is now done in the UFS driver.
* Fix model_configure file for two tests to define `fhrot` parameter.

* Revert iovr=4 (exponential cloud overlap method) in RRTMG (ufs-community#963)

Revert/cleanup iovr=4 in RRTMG per developer's request. The current exponential cloud overlapping method used in RRTMG represents the method used in the pre-2018 operational HWRF model.

* Add fhzero ort test (ufs-community#977)

* Fhzero ORT applies to the two tests `control` and `control_p7`
* Test can be run by invoking, for example, `./opnReqTest -n control -c fhz`

* Add a new CDEPS test to use GSFv16 data mode in CDEPS ,update rt_weekly.conf and update MOM6 to main branch (20211220) (ufs-community#976)

* add a new cdeps test using GFS data mode
* update rt_weekly.conf to include cpld_bmark_p7_35d tests
* update MOM6

Co-authored-by: jiandewang <jiande.wang@noaa.gov>

* Bug fixes for WAM physics and regression tests, remove duplicate modules from ccpp-physics CMakeLists.txt (ufs-community#947)

*adds a new regression test control_wam_debug and fixes problems with the existing control_wam test (e.g. IAER was not set as intended).

* Update global inline post for outputting foundation temperature (ufs-community#968)

Update upp submodule in fv3atm with latest UPP version for outputting foundation temperature and fixing land-sea mask issue in global inline post.

* Regression test bug fixes for running on Jet or with Rocoto (ufs-community#981)

* Bug fixes for running on jet or with rocoto
1. Correctly detect Jet when deciding build job runtimes
2. Do not delete out and err files in compile & test jobs
3. Correct dependencies for WW3 in Rocoto
4. Do not hard-code ACCNR to h-nems
This PR also contains fv3/ccpp physics (CCPP PR#812) updates that implements a vertically-integrated formula to properly compute tracer column mass and ensure mass conservation for all tracers except TKE. See issue NCAR/ccpp-physics#811 for details

* MYNN sfclay (RAP suite) restart reproducibility (ufs-community#984)

*This PR turns on the rap_restart tests. Bit-for-bit reproducibility is achieved with the bug fixes for MYNN sfclay in the associated fv3atm and ccpp-physics PRs.

* fixes on initializing snow over fractional ice and GFS_surface_composite and z0ice value change (ufs-community#996)

* Radar-derived microphysics temperature tendencies similar to operational HRRR (ufs-community#986)

This PR implements a feature of the operational HRRR, radar-derived microphysics temperature tendencies applied in the first N minutes of the forecast to improve clouds in the first few hours. Two regression tests are added.

* Turn on CICE albedo use by ATM for coupled model; update Prototype test suite to initial P8 configuration;  (ufs-community#944)

* add updates for suites, tests for initial P8 configuration
* add IOPT_SFC=3 as default; set IOPT_SFC=1 in p7 tests
* change min_seaice to 1e-6; set to 1e-11 in p7 tests
* set dddmp=0.1 in both P7 and P8. This matches the final
P7 configuration. It will change P7 baselines.
* set cdmbwd in both P7 and P8 to the final P7 configuration.
(1.0,2.2,1.0,1.0) for all resolutions. It will change P7 baselines.
* add dusan's ecflow fix for wcoss2
* update FV3 with Moorthis' snod fix
* update rt_weekly to use p8 tests
* change integration length to 5 days

* Thompson MP cloud tuning (ufs-community#1005)

* improves cloud fraction when using Thompson MP. See NCAR/ccpp-physics#809 for more details.

* Add a new HAFS regression test with Thompson MP and GFDL surface layer schemes and Corrected S4 template regression testing job card (ufs-community#995)

* Added a new regression test for HAFS. 
* Fixed the S4 RT template job card.

Co-authored-by: David Huber <david.huber@noaa.gov>
Co-authored-by: Bin Liu <Bin.Liu@noaa.gov>

* CCPP: Update consistency checks and effective radii calculation for Thompson MP (ufs-community#1000)

This PR contains changes in ccpp-physics (Update consistency checks and effective radii calculation for Thompson MP) and adds two new regression tests for both Intel and GNU that use the new prognostic cloud scheme progcld_thompson with GFS v16 and with RAP. The new cloud scheme is activated by setting icloud == 3 in input.nml.

* Rremove nems_datm related code, update CDEPS, and remove run directories that are no longer needed by other tests (ufs-community#1008)

* Add fhzero ort to coupled tests (ufs-community#1009)

* Add fhz ort to cpld tests
* Set ACCNR in rt.sh, not detect_machine.sh. Add some comments explaining where the variable should not be set or used.

Co-authored-by: Samuel Trahan <Samuel.Trahan@noaa.gov>

* Unified write_netcdf, add single file 'cubed_sphere_grid' output (ufs-community#1011)

* Unify write_netcdf and write_netcdf_parallel modules.
* Add support for writing 'cubed_sphere_grid' history files in a single netcdf file.

* Change standard name and unit of CCPP error flag variable in CCPP framework and physics (ufs-community#1013)

* updates the submodule pointers for fv3atm, ccpp-framework and ccpp-physics for:  Change standard name and unit of CCPP error flag variable in CCPP framework and physics.

* ESMF managed threading (ufs-community#1018)

Add code updates to enable the ESMF managed threading in UFS.


Co-authored-by: Gerhard Theurich <gerhard.j.theurich@noaa.gov >

* sync-ww3 (ufs-community#1026)

* sync with ww3 develop branch

* Revisions to repair iovr=5 cloud overlap option (ufs-community#1025)

* This PR addresses part 2 of issue ufs-community#748 to activate the exponential-random cloud overlap method (iovr=5) in RRTMG. RT tests were added for  iovr=4 and iovr=5 options
* change iovr to use the environment variable in control.nml.IN; modify all tests that use control.nml.IN to set env. var. iovr=3 (except for new tests); remove restart files from new tests

* 2D decomposition on write grid, bug fix in FV3GFS_IO.F90, and quad precision in stochastic physics (ufs-community#1028)

- The 2D decomposition is enabled on write grid component.
- FV3atm PR NOAA-EMC/fv3atm#468 from @MicroTed 
- stochastic physics PR NOAA-PSL/stochastic_physics#53 from Dom.

Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Dom Heinzeller <heinzell@ucar.edu>

* Update CMEPS for wave coupling (ufs-community#1034)


* add run phases to allow waves to run in the slow loop
* re-order hafs wave test
* update aliases in fd_nems.yaml
* update CMEPS to prevent advertise conflicts when s2sw is
still using connectors
* add correct masking variables to hafs nems.configure files

* 4DIAU bug fix when iau_filter_increments=T (ufs-community#990)

* Add lsm_cold_start variable for RUC LSM SCM support and gwdps bugfix (combined) (ufs-community#1037)

Co-authored-by: @SMoorthi-emc

* Update esmf 821b04 (ufs-community#1041)

* Update the version of ESMF library to 8.2.1b04
* Add the write group lon info for upp 2d decomposition test.

* Multiple output (nested) grids from fv3atm (ufs-community#1044)

Update fv3atm to support writing multiple history files for parent and nested domains.
Nested HAFS regression tests were updated to write out history files from all nested domains.

Co-authored-by: Bin Liu <bin.liu@noaa.gov>

* GPU-enabled version of Grell-Freitas convection in ccpp-physics (ufs-community#1043)

* Enable Thompson microphysics with UFS-Aerosols and update GOCART (ufs-community#1049)

This update allows the Thompson microphysics scheme to be used with prognostic aerosols (UFS-Aerosols) and updates GOCART to version 2.0.2.

* Add support for Stochastically Perturbed Parameterizations (SPP) in FV3 (ufs-community#982)

* Update CICE; add support for restart test when running single RT (was PR 1058); update MOM6 (was PR 1060) (ufs-community#1045)


* update cice
* (1) pointing MOM6 to testing branch (MOM6 main repo PR: mom-ocean/MOM6#1556 (comment))
    (2) add three new files in mom_cmake file
            core/MOM_porous_barriers.F90
            external/drifters/MOM_particles.F90
            external/drifters/MOM_particles_types.F90
    (3) explicitly set in MOM_input for all resolutions to reflect the default value changes in code to retain answer
        EPBL_MLD_BISECTION = True (default changed from T to F in this MOM6 PR)
        INTERNAL_WAVE_SPEED_BETTER_EST = False (default changed from F to T in this MOM6 PR)
        note BBL_USE_EOS now set to use the value of USE_EOS in this MOM6 PR,
        but we set BBL_USE_EOS explicit so it has no impact

* remove duplicated CHL input setting in MOM_input_template_050
* add support for restart tests while running single RT

* lateral boundary fix for regional runs (ufs-community#1050)

This PR fixes a transient increase in noise caused by incorrect processing of winds along the lateral boundary on the final acoustic time step before fresh boundary information is ingested into the model.

* Update pointer to FV3

* Point to Christina's branches.

* Update RT log for Hera Intel.

* Adding RT log for Hera GNU tests.

* Updating RT log for Jet Intel tests.

* Latest regression tests on Jet.

* Adding regression logs.

* Updating regression test date stamp.

* Point to GSL repositories for FV3 and FV3/ccpp/physics

Co-authored-by: Dusan Jovic <48258889+DusanJovic-NOAA@users.noreply.github.com>
Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com>
Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>
Co-authored-by: BinLi-NOAA <bin.li@noaa.gov>
Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>
Co-authored-by: Dusan Jovic <dusan.jovic@noaa.gov>
Co-authored-by: Grant Firl <grant.firl@noaa.gov>
Co-authored-by: Dom Heinzeller <climbfuji@ymail.com>
Co-authored-by: mzhangw <mzhangw@gmail.com>
Co-authored-by: jiandewang <jiande.wang@noaa.gov>
Co-authored-by: WenMeng-NOAA <48260754+WenMeng-NOAA@users.noreply.github.com>
Co-authored-by: Samuel Trahan (NOAA contractor) <39415369+SamuelTrahanNOAA@users.noreply.github.com>
Co-authored-by: SMoorthi-emc <47667426+SMoorthi-emc@users.noreply.github.com>
Co-authored-by: ChunxiZhang-NOAA <49283036+ChunxiZhang-NOAA@users.noreply.github.com>
Co-authored-by: David Huber <david.huber@noaa.gov>
Co-authored-by: Bin Liu <Bin.Liu@noaa.gov>
Co-authored-by: Samuel Trahan <Samuel.Trahan@noaa.gov>
Co-authored-by: Jun Wang <37633869+junwang-noaa@users.noreply.github.com>
Co-authored-by: Gerhard Theurich <gerhard.j.theurich@noaa.gov >
Co-authored-by: Ali.Abdolali <37336972+aliabdolali@users.noreply.github.com>
Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Dom Heinzeller <heinzell@ucar.edu>
Co-authored-by: Jeff Whitaker <jswhit@fastmail.fm>
Co-authored-by: DomHeinzeller <58610420+DomHeinzeller@users.noreply.github.com>
Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>
Co-authored-by: JeffBeck-NOAA <55201531+JeffBeck-NOAA@users.noreply.github.com>
Co-authored-by: MatthewPyle-NOAA <48285220+MatthewPyle-NOAA@users.noreply.github.com>

* Remove GF Consistency Check & Add SDFs - V2 (#120)

* - Removes consistency check in GF, allows GF to run with different or no shallow scheme
- Add additional RAP based SDF

* - Add regression test logs

* jet intel test

* point to gsl/develop for FV3

Co-authored-by: Samuel Trahan <Samuel.Trahan@noaa.gov>

* Update to top of FV3 to get latest ccpp-physics codeowners (#123)

* FV3 submodule: merge gsl/develop into RRFS_dev

* Hera gnu tests passed.

* Hera intel tests passed.

* jet tests passed

* Put a comment back in the FV3/ccpp/physics submodule.

* point to head of FV3 RRFS_dev branch

Co-authored-by: Christina Holt <56881914+christinaholtNOAA@users.noreply.github.com>
Co-authored-by: Dusan Jovic <48258889+DusanJovic-NOAA@users.noreply.github.com>
Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com>
Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>
Co-authored-by: BinLi-NOAA <bin.li@noaa.gov>
Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>
Co-authored-by: Dusan Jovic <dusan.jovic@noaa.gov>
Co-authored-by: Grant Firl <grant.firl@noaa.gov>
Co-authored-by: Dom Heinzeller <climbfuji@ymail.com>
Co-authored-by: mzhangw <mzhangw@gmail.com>
Co-authored-by: jiandewang <jiande.wang@noaa.gov>
Co-authored-by: WenMeng-NOAA <48260754+WenMeng-NOAA@users.noreply.github.com>
Co-authored-by: SMoorthi-emc <47667426+SMoorthi-emc@users.noreply.github.com>
Co-authored-by: ChunxiZhang-NOAA <49283036+ChunxiZhang-NOAA@users.noreply.github.com>
Co-authored-by: David Huber <david.huber@noaa.gov>
Co-authored-by: Bin Liu <Bin.Liu@noaa.gov>
Co-authored-by: Jun Wang <37633869+junwang-noaa@users.noreply.github.com>
Co-authored-by: Gerhard Theurich <gerhard.j.theurich@noaa.gov >
Co-authored-by: Ali.Abdolali <37336972+aliabdolali@users.noreply.github.com>
Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Dom Heinzeller <heinzell@ucar.edu>
Co-authored-by: Jeff Whitaker <jswhit@fastmail.fm>
Co-authored-by: DomHeinzeller <58610420+DomHeinzeller@users.noreply.github.com>
Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>
Co-authored-by: JeffBeck-NOAA <55201531+JeffBeck-NOAA@users.noreply.github.com>
Co-authored-by: MatthewPyle-NOAA <48285220+MatthewPyle-NOAA@users.noreply.github.com>
Co-authored-by: Hannah C Barnes <38660891+hannahcbarnes@users.noreply.github.com>
SamuelTrahanNOAA added a commit to NOAA-GSL/fv3atm that referenced this pull request May 19, 2022
* Feature/ccpp codeowners (NOAA-EMC#442)

* Per-file CODEOWNERS in ccpp/physics to set up automatic review requests

* Code cleanup. Remove used code/variables. Fix minor inconsistencies. (NOAA-EMC#440)

* Remove ESMF Alarm and TimeInterval variables from module_fv3_config.
* Variables nfhmax, nfhmax_hf are unused outside InitializeAdvertise. Declared them as local variables.
* There is no need to keep duplicates of all time related variables in atm_int_state when we can easily access them from atm_int_state%Atm.
* Remove redundant call to fms_init
* Add few missing ESMF_LogFoundError checks in module_fcst_grid_comp.F90
* Delete time_utils.F90. Unused.
* print only actual errors to stderr, everything else to stdout
* Move realizeConnectedCplFields to module_cplfields from module_cap_cpl
* Declare Atmos as module variable, and remove atmos_internalstate_wrapper
* Move code from clock_cplIntval to InitializeAdvertise
* Removed INTERNAL_FILE_NML from atmos_model.F90

* CCPP cloud cover change for Thompson MP associated with Xu Randall (NOAA-EMC#443)

* Remove old comments from GFS_typedefs.F90.
* Update logic that sets Model%cnvcld = .false.

* Wrapper for ccpp-physics NOAA-EMC#806, NOAA-EMC#807, NOAA-EMC#813 (NOAA-EMC#447)

* CCPP physics updates from PR NOAA-EMC#806(only diagnostic changes for RRTMGP - small diag-only RT impact for tests involving RRTMGP),NOAA-EMC#807 (only aborting model if sfc emis file not present when needed) ,NOAA-EMC#813(only SCM-specific physics changes)

* Wrapper for ccpp-physics NOAA-EMC#808 and 816 (roughness length over ice and NoahMP tsurf bugfix) (NOAA-EMC#452)

This PR contains the ccpp physics PR NOAA-EMC#808 and NOAA-EMC#816. 808 addresses an error in the momentum roughness length over tiles with ice. 816 fixes an occasional segfault bug related to the tsurf variable in NoahMP and updates to "improve snow simulation in NoahMP for P8".

* Use 'model set run clock' routine in FV3 NUOPC cap. (NOAA-EMC#450)

* Add 'SetRunClock' specialization routine to FV3 NUOPC cap, which sets fv3 model clock. FV3 cap does not keep local copy of clock (clock_fv3) as saved module variable anymore.
* Minor code cleanup.

* Clean up iovr=4 (exponential cloud overlapping method) in RRTMG (NOAA-EMC#445)

Clean up the exponential cloud overlapping method in RRTMG, which reflects the practice of the pre-2018 operational HWRF model.

* Bug fix for dimensions of eta level variables and WAM variables in Fortran code and CCPP metadata (NOAA-EMC#431)

* Add additional diagnostic arrays for radiation-microphysics development
* Bug fixes for WAM model runs with levr < levs

* Update inline post with latest UPP release upp_v10.0.11 (NOAA-EMC#449)

* Update UPP revision
* Add foundation temperaure in GFS read interface for inline post.

* Wrapper for ccpp-physics NOAA-EMC#812 (NOAA-EMC#453)

* update submodule pointer for regression testing of ccpp-physics#812

* MYNN sfclay (RAP suite) restart reproducibility, P8 suite definition files (NOAA-EMC#455)

* Fix uninitialized variable zmtnblck in ccpp/data/GFS_typedefs.F90
* Fix typo in CCPP standard name for ncnvwind in ccpp/data/GFS_typedefs.meta
* Add ten 2d variables required for MYNNSFC restart reproducibility to ccpp/driver/GFS_restart.F90
* create initial p8 suites, P8 initial SDFs as copies of FV3_GFS_v16_coupled_nsstNoahmpUGWPv1
and FV3_GFS_v16_nsstNoahmpUGWPv1

Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>

* Fixes on initializing snow depth over ice and changes z0ice (NOAA-EMC#461)

* modify FV3GFS_io.F90 by fixing errors associated with initializing snow depth over ice in the case where both land and water coexist (i.e. fractional grid case)
* z0ice is changed to 1.0 cm from 1.1cm in atmos_model.F90

* Radar-derived microphysics temperature tendencies similar to operational HRRR (NOAA-EMC#457)

- implements a feature of the operational HRRR, radar-derived microphysics temperature tendencies applied in the first N minutes of the forecast to improve clouds in the first few hours.
* HRRR-like radar-derived temperature tendencies
* Give a warning when convection is enabled with radar tten
* Fix uninitialized variable zmtnblck in ccpp/data/GFS_typedefs.F90
* Add ten 2d variables required for MYNNSFC restart reproducibility to ccpp/driver/GFS_restart.F90

* fixing snod bug in atmos_model.F90 (NOAA-EMC#465)

* Thompson MP cloud tuning (NOAA-EMC#463)

*  Improve cloud fraction when using Thompson MP. See NCAR/ccpp-physics#809 for more details.

* Feature/hwrf legacy (NOAA-EMC#459)

* Added a new suite definition file: suite_FV3_HAFS_v0_thompson_tedmf_gfdlsf.xml. This SDF is a legacy hwrf SDF but only the GFDL surface layer scheme and the Thompson scheme are kept.

* CCPP: Update consistency checks and effective radii calculation for Thompson MP (NOAA-EMC#462)

* This PR updates the submodule pointer for ccpp-physics for the changes described in NCAR/ccpp-physics#786 (Update consistency checks and effective radii calculation for Thompson MP)

* Unified write_netcdf, add single file 'cubed_sphere_grid' output (NOAA-EMC#466)

* Unify write_netcdf and write_netcdf_parallel modules.
* Add support for writing 'cubed_sphere_grid' history files in a single netcdf file
* Code refactoring and cleanup.

* Change standard name and unit of CCPP error flag variable in CCPP framework and physics (NOAA-EMC#467)

*Change standard name and unit of CCPP error flag variable in CCPP framework and physics.

* Add code to enable ESMF managed threading (NOAA-EMC#469)

* Implement ESMF-managed-threading for fcstComp and wrtComp's.

* Revisions to repair iovr=5 cloud overlap option (NOAA-EMC#471)

*This PR addresses part 2 of CCPP issue NOAA-EMC#748 to activate the exponential-random cloud overlap method (iovr=5) in RRTMG.

* Add 2d decomposition on the write grid comp (NOAA-EMC#470)

* update write grid comp to have 2D decomposition on output grid.
* combine PR#468: Bug fix in FV3GFS_io.F90 for allocation of temp2d

Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>

* 4DIAU bug when iau_filter_increments=T  (NOAA-EMC#458)

* Add lsm_cold_start variable for RUC LSM SCM support and gwdps bugfix (combined) (NOAA-EMC#475)

* add lsm_cold_start variable in GFS_typedefs.F90/meta and update ccpp/physics for testing
* includes @SMoorthi-emc 's bugfix for gwdps.f.

* add lon info in the write group (NOAA-EMC#476)

* Multiple output grids (NOAA-EMC#480)

Update fv3 cap and write grid component to enable outputting multiple domains.
This is done be creating an array of fcstGrids, and array of rout handles where each element of these arrays correspond to one atm domain.
In the write grid component updates were made to allow grid spec parameters for each output grid to be specified separately.

Co-authored-by: Gerhard Theurich <theurich@sourcespring.net>

* GPU-enabled version of Grell-Freitas convection in ccpp-physics (NOAA-EMC#479)

* Enable Thompson MP when coupling with UFS-Aerosols (NOAA-EMC#484)

* Add support for Stochastically Perturbed Parameterizations (SPP) in FV3 and add the FV3_RRFS_v1alpha SDF. (NOAA-EMC#454)

Adds the necessary code in fv3atm to allow for Stochastically Perturbed Parameterizations (SPP) in a set of RAP/HRRR-based physics parameterizations. Specific to the fv3atm repository, code in this PR defines the necessary variables associated with various SPP-related fields (e.g., logical to activate SPP, parameterization-specific SPP variables, etc.) that are then passed to ccpp-physics.

* lateral boundary fix for regional runs (NOAA-EMC#482)

* Feature: NSSL Microphysics support (NOAA-EMC#472)

Supports CCPP with new NSSL microphysics scheme, which has separate hydrometeor categories for graupel and hail (Mansell et al. 2010, JAS). This PR allows fv3atm to compile correctly with CCPP+NSSL, but a further update to atmos_cubed_sphere is needed to run correctly. The NSSL scheme is used in NSSL's daily FV3 convection-allowing forecasts, and the scheme itself is aimed at this scale (dx <= 4km). Testing is underway also with a preliminary Warn-on-Forecast FV3 ensemble.

* Add p8b SDF (NOAA-EMC#491)

* add p8 suite files
* fix 2phases intermediate restart

Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>
Co-authored-by: Jun Wang <jun.wang@noaa.gov>

* Split CCPP finalize into physics_finalize and (framework) finalize; minor CCPP cleanup and documentation updates (NOAA-EMC#487)

* Minor cleanup and documentation updates for CCPP; split CCPP finalize into physics_finalize and (framework) finalize

* Add 'valid time' variable using ISO string format (NOAA-EMC#492)

Add 'valid time' variable using ISO string format to netcdf history files.
Use double precision variable to set value of 'time' attribute in wrt comp import state
Update ccpp/physics (setting surface-related interstitial variables for SCM prescribed-surface-flux mode)
Update inline_post_stub.F90 subroutine interfaces to match inline_post.F90

Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Grant Firl <grant.firl@noaa.gov>

* Fix Lambert Conformal Conic mapping in the write component on the southern hemisphere (NOAA-EMC#497)

Subroutine `lambert` in the write component has been fixed to do the mapping on the southern hemisphere correctly.

* Rectify total number of Thompson MP species for PBL mixing (NOAA-EMC#509)

* update upp revision (NOAA-EMC#505)

* move some parameters in mynn surface layer to namelist options

* land surface upgrades for p8c (NOAA-EMC#499)

* Added capability for cdmbgwd(1) to scale GSL blocking drag (NOAA-EMC#502)

* release/p8c: it is related to ccpp-physics PBL and cumulus updates plus a bug fix to the Thompson scheme PR #882 (NOAA-EMC#503)

* change ccpp-physics branch to feature/p8c_pbl_cp branch

* change ccpp/physics in feature/p8c_pbl_cp branch

* revised GFS_typedefs.F90 for the paramters in the saSAS scheme

* sync upp

* Revert "sync upp"

This reverts commit 429d046.

* sync upp to 0185bb9

* Merge remote-tracking branch remote/main for ccpp-physics and update .gitmodules

* Bring AQM model into UFS (NOAA-EMC#511)

* Reset to zero coupling arrays for accumulated snow,  large scale rain, and convective rain at the end of each coupling step if coupling with chemistry model.
* Properly set kind type of literal constants defining zero and one.
* Initialize to zero canopy resistance output variablein noah/osu land-surface model subdriver.
* Re-implement radiation diagnostic output involving spectral band layer cloud optical depths (0.55 and 10 mu channels)
to prevent floating invalid errors due to uninitialized optical depth arrays.
* Temporarily disable filling export fields during the NUOPC Realize phase since it breaks coupling with aerosol component.
* Increase maximum number of input aerosol scavenging factors to accommodate AQM/CMAQ 5.2.1 chemical tracers.
* Remove inst_pres_height_surface from chemistryFieldNames as its imported already elsewhere

Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>

* RRTMGP for prototype 8 (NOAA-EMC#498)

* Add explict treatment of convective cloud to RRTMGP.
* Some cleanup and bug fixes from previous commit. working now with Thompson MP plus radiatively active convective cloud.
* Updated physics. New interstitials.
* Added explicit MYNN-EDMF cloud coupling to RRTMGP.
* Added switches for sgs clouds in GP.
* Replaced cld_mynn_ naming convention with cld_pbl_
* Fixed inconsistency between G/GP in Thompson MP.
* Some more cleanup of cloud-fraction...
* Pulled in Qingfu's changes to ccpp-physics
* Synced physics with main
* Updated physics.
* Revert change from previous commits (sampling of different cloud types). New SDFs for p8c
* Changed arguments to implied shape.
* Renamed scheme file in SDF. Now common to both RRTMG and RRTMGP SDFs
* Syntax error in ccpp-physics
* Revert changes to names
* Renamed p8_rrtmgp SDFs
* Added bounding to temperature at layer-interface used by RRTMGP.
* Reorder loop
* Combined gp sw and lw aerosol routines. Modest speedup (~4%)

* HAFS related moving-nesting, nest coupling, and movable output grid capabilities (NOAA-EMC#501)

* Add the storm-following or specified moving-nesting capability for both the regional and global configuration.
* Expand the UFS/HAFS FV3ATM-HYCOM ocean coupling to support the static and moving nesting configuration.
* Expand the write grid component to support outputting the moving nest domain properly.
* Five HAFS related moving-nesting regression tests are added to test these newly added features. The regional_hafs RT was removed.

Co-authored-by: Dom Heinzeller <climbfuji@ymail.com>
Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>
Co-authored-by: Grant Firl <grantf@ucar.edu>
Co-authored-by: Daniel Rosen <daniel.rosen@noaa.gov>
Co-authored-by: man.zhang <Man.Zhang@noaa.gov>
Co-authored-by: William Ramstrom <William.Ramstrom@noaa.gov>
Co-authored-by: Ufuk Turuncoglu <ufuk.turuncoglu@noaa.gov>
Co-authored-by: Dusan Jovic <dusan.jovic@noaa.gov>
Co-authored-by: A-Kyle <kyle.ahern@noaa.gov>
Co-authored-by: Gerhard Theurich <theurich@sourcespring.net>
Co-authored-by: Dusan Jovic <48258889+DusanJovic-NOAA@users.noreply.github.com>

* Split GFS_typedefs.F90, rename CCPP_interstitial, import ccpp_t from ccpp_types (NOAA-EMC#508)

* Split GFS_typedefs.F90: move GFS_interstitial_type to CCPP_typedefs.F90; Rename CCPP_interstitial to GFDL_interstitial; Import ccpp_t from ccpp_types instead of ccpp_api
* Add bugfix in ccpp/data/GFS_typedefs.meta: mark input_nml_file as active only if the number of lines in the file is greater than zero

* Fix uninitialized min_rand variable in Thompson MP when using SPP (NOAA-EMC#516)

* Pass SPP namelist entries outside of stochastic physics routines.

* Fixes to SPP metadata

* Add active attribute to spp_prt_list

* Add active metadata flag to spp_stddev_cutoff

* Add active metadata flag for spp_var_list

* A bug fix for wet scavenging of aerosols and a minor modification for moisture property calculation  (NOAA-EMC#520)

* Update ccpp_prebuild_config.py for one-to-one CCPP physics schemes/files (NOAA-EMC#519)

* point to gsl/merge-develop branch

* Point to gsl/merge-develop-to-community branch for ccpp/physics

* Corrections to merge + joe updates

* Point .gitmodules to Sam's fork

* Correct issues found by reviewers

* Rename mynnpbl to mynnedmf

* Remove some empty _init and _finalize routines and update a comment in sgscloud_radpre

* Feature/lndp noahmp2 : add land perturbation scheme for Noah-MP fractional veg  (NOAA-EMC#513)

Updates to allow land perturbation scheme to be applied for Noah-MP.

Main code changes:
-added lsm_noahmp model option to lndp_type==2 land perturbation scheme (fv3-atm, stochastic_physics)
-updated comments
-cleaned up the namelists to make the different options clearer. Removed misleading lndp_each_step variable from gfs_physics_nml, and replaced it with "lndp_model_type" in the nam_sfcperts namelist for the different forecast types (cycling DA, short forecasts, perturbing only initial conditions) (fv3-atm, stochastic_physics, ufs-weather-model)
NOTE: see note below, re: specification of smc perturbation for RAP/HRRR/etc (lndp_model_type==2)
-revised code in stochastic_physics_wrapper to only allocate arrays that will be used when passed into lndp_apply_perts (fv3-atm)
-deleted unused albedo arrays in stochastic_physics_wrapper and lndp_apply_perts (fv3-atm, stochastic_physics)
-fixed bug causing precision errors when calculating the soil moisture ice content (often ~0.0; stochastic_physics)
-fixed bug in which lndp_apply_perts wasn't called for nscyc = 0. (fv3-atm)
-increased max_nvar_lndp at Jeff Ator's request (fv3-atm)
-added a new test for the Noah-MP and lndp==2 combination (ufs-weather-model)

* Add \file to smoke files

* Add hailcast code in atmos_cubed_sphere  (NOAA-EMC#528)

* Combine PROD and REPRO build options into 'Release' build type (NOAA-EMC#524)

Combined PROD and REPRO build modes into 'Release' build type. Now only 'Debug' and 'Release' build type are supported.
Both build types must produce bit-for-bit reproducible outputs using different number of threads, mpi tasks, different domain decomposition, have reproducible restarts etc.

* Unify inline post (NOAA-EMC#521)

* Unify global and regional inline posts and add bug fix for dx/dy computation.
* Update upp revision to dc8bc68
* Set up different configurations of read max/min 2m T for global and regional FV3.

* adding bug fix from jili dong and removing U* averaging

* adding bug fix from jili dong and removing U* averaging

* Remove some commented-out code from cu_gf_driver in ccpp-physics.

* Turn rrtmg smoke band 10 into a model namelist variable

* Switch ccpp-physics smoke arguments to implied shape

* Remove some stops and gotos from ccpp-physics smoke

* Remove some hard-coded constants and rename some meta entries

* Change RRTMGP to RRTMG in two P8 SDFs (NOAA-EMC#527)

* Change RRTMGP to RRTMG in suite_FV3_GFS_v17_p8 and suite_FV3_GFS_v17_coupled_p8
* deleted or modified some SDFs related to RRTMGP or Thompson schemes
* added a new SDF file for P8 with rrtmgp

* Rename some standard_names, change a unit, and remove a "use physcons"

* NSSL ccpp-physics bugfixes and new ccpp-framework debugging feature (NOAA-EMC#529)

* Set num3d correctly when using radar_tten

* Only add cnv_3d_ud_mf to the restart files when needed

* Pass some chemistry varibles to mynn_bl_driver correctly

* Correct argument passing within mynn edmf

* point to NCAR main ccpp/physics

* merge gsl institutional fork (NOAA-EMC#526)

Merge in GSL institutional fork

* point to sam's repo

* ccpp/physics: Change to GSL CODEOWNERS

* Returned the comments on soil resistance back.

* point to gsl/develop for ccpp/physics

Co-authored-by: Dusan Jovic <48258889+DusanJovic-NOAA@users.noreply.github.com>
Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>
Co-authored-by: Grant Firl <grant.firl@noaa.gov>
Co-authored-by: mzhangw <mzhangw@gmail.com>
Co-authored-by: WenMeng-NOAA <48260754+WenMeng-NOAA@users.noreply.github.com>
Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>
Co-authored-by: SMoorthi-emc <47667426+SMoorthi-emc@users.noreply.github.com>
Co-authored-by: ChunxiZhang-NOAA <49283036+ChunxiZhang-NOAA@users.noreply.github.com>
Co-authored-by: Jun Wang <37633869+junwang-noaa@users.noreply.github.com>
Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Jeff Whitaker <jswhit@fastmail.fm>
Co-authored-by: Gerhard Theurich <theurich@sourcespring.net>
Co-authored-by: DomHeinzeller <58610420+DomHeinzeller@users.noreply.github.com>
Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>
Co-authored-by: JeffBeck-NOAA <55201531+JeffBeck-NOAA@users.noreply.github.com>
Co-authored-by: MatthewPyle-NOAA <48285220+MatthewPyle-NOAA@users.noreply.github.com>
Co-authored-by: Ted Mansell <37668594+MicroTed@users.noreply.github.com>
Co-authored-by: Jessica Meixner <jessica.meixner@noaa.gov>
Co-authored-by: Jun Wang <jun.wang@noaa.gov>
Co-authored-by: joeolson42 <Joseph.B.Olson@noaa.gov>
Co-authored-by: HelinWei-NOAA <48133472+HelinWei-NOAA@users.noreply.github.com>
Co-authored-by: mdtoyNOAA <73618848+mdtoyNOAA@users.noreply.github.com>
Co-authored-by: Brian Curtis <64433609+BrianCurtis-NOAA@users.noreply.github.com>
Co-authored-by: dustinswales <dustin.swales@noaa.gov>
Co-authored-by: Bin Liu <bin.liu@noaa.gov>
Co-authored-by: Dom Heinzeller <climbfuji@ymail.com>
Co-authored-by: Grant Firl <grantf@ucar.edu>
Co-authored-by: Daniel Rosen <daniel.rosen@noaa.gov>
Co-authored-by: man.zhang <Man.Zhang@noaa.gov>
Co-authored-by: William Ramstrom <William.Ramstrom@noaa.gov>
Co-authored-by: Ufuk Turuncoglu <ufuk.turuncoglu@noaa.gov>
Co-authored-by: Dusan Jovic <dusan.jovic@noaa.gov>
Co-authored-by: A-Kyle <kyle.ahern@noaa.gov>
Co-authored-by: ClaraDraper-NOAA <33430543+ClaraDraper-NOAA@users.noreply.github.com>
Co-authored-by: BinLi-NOAA <bin.li@noaa.gov>
SamuelTrahanNOAA added a commit to NOAA-GSL/ufs-weather-model that referenced this pull request May 19, 2022
* Add ufs driver. Remove NEMS submodule. Update CMEPS. Update CDEPS. Feature/ort addition.  (ufs-community#931)

* Add UFS driver, a copy of two fortran files from the NEMS repository. Remove NEMS submodule.

* Update CMEPS for current escomp/master. Implement the ability to write mediator history files for ATM on tiles rather than mesh

* Update CDEPS component. The updated CDEPS includes code updates for GFS v16 data mode.

* Operation requirement test (ORT) for:
  - cpld_control_p7 thread and decomposition tests
  - cpld_bmark_p7 thread and decomposition tests
  - Due to the wave component in these tests, restart and debug do not work

Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com>
Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>
Co-authored-by: BinLi-NOAA <bin.li@noaa.gov>

* fv3atm cap cleanup (ufs-community#954)

FV3 nuopc cap code cleanup. Remove used code/variables. Fix minor inconsistencies.

* CCPP cloud cover change for Thompson MP associated with Xu Randall, several bug fixes for regression testing scripts (ufs-community#962)

This PR contains code changes 1) to remove old comments in GFS_typedefs.F90, increase cloud cover for Thompson MP by passing convective cloud water mixing ratio to progcld6. 2) Updates for auto fixes

Co-authored-by: Dusan Jovic <dusan.jovic@noaa.gov>

* ccpp-physics ufs-community#806, #807, ufs-community#813, and ort modifications (ufs-community#970)

- CCPP Physics changes: see Wrapper for ccpp-physics ufs-community#806, #807, ufs-community#813, NOAA-EMC/fv3atm#447.
- UFS weather model changes for ORT updates following changes in Fix Auto-RT on Gaea, Jet and Orion ufs-community#965

* Wrapper for ccpp-physics ufs-community#808 and 816 (roughness length over ice and NoahMP tsurf bugfix) (ufs-community#971)

* update submodule pointer for regression testing for ccpp-physics ufs-community#808,816: roughness length over ice and NoahMP tsurf bugfix
* Reduce time step from 600s to 450s for tests control_csawmg and control_csawmg_debug

Co-authored-by: Dom Heinzeller <climbfuji@ymail.com>
Co-authored-by: climbfuji <dom.heinzeller@icloud.com>

* Use 'model set run clock' routine in FV3 NUOPC cap. (ufs-community#967)

* Update FV3 cap by adding 'model set run clock' specialization routine. Because FV3 cap does not modify 'Earth clock' anymore in case of a restarted run, that adjustment is now done in the UFS driver.
* Fix model_configure file for two tests to define `fhrot` parameter.

* Revert iovr=4 (exponential cloud overlap method) in RRTMG (ufs-community#963)

Revert/cleanup iovr=4 in RRTMG per developer's request. The current exponential cloud overlapping method used in RRTMG represents the method used in the pre-2018 operational HWRF model.

* Add fhzero ort test (ufs-community#977)

* Fhzero ORT applies to the two tests `control` and `control_p7`
* Test can be run by invoking, for example, `./opnReqTest -n control -c fhz`

* Add a new CDEPS test to use GSFv16 data mode in CDEPS ,update rt_weekly.conf and update MOM6 to main branch (20211220) (ufs-community#976)

* add a new cdeps test using GFS data mode
* update rt_weekly.conf to include cpld_bmark_p7_35d tests
* update MOM6

Co-authored-by: jiandewang <jiande.wang@noaa.gov>

* Bug fixes for WAM physics and regression tests, remove duplicate modules from ccpp-physics CMakeLists.txt (ufs-community#947)

*adds a new regression test control_wam_debug and fixes problems with the existing control_wam test (e.g. IAER was not set as intended).

* Update global inline post for outputting foundation temperature (ufs-community#968)

Update upp submodule in fv3atm with latest UPP version for outputting foundation temperature and fixing land-sea mask issue in global inline post.

* Regression test bug fixes for running on Jet or with Rocoto (ufs-community#981)

* Bug fixes for running on jet or with rocoto
1. Correctly detect Jet when deciding build job runtimes
2. Do not delete out and err files in compile & test jobs
3. Correct dependencies for WW3 in Rocoto
4. Do not hard-code ACCNR to h-nems
This PR also contains fv3/ccpp physics (CCPP PR#812) updates that implements a vertically-integrated formula to properly compute tracer column mass and ensure mass conservation for all tracers except TKE. See issue NCAR/ccpp-physics#811 for details

* MYNN sfclay (RAP suite) restart reproducibility (ufs-community#984)

*This PR turns on the rap_restart tests. Bit-for-bit reproducibility is achieved with the bug fixes for MYNN sfclay in the associated fv3atm and ccpp-physics PRs.

* fixes on initializing snow over fractional ice and GFS_surface_composite and z0ice value change (ufs-community#996)

* Radar-derived microphysics temperature tendencies similar to operational HRRR (ufs-community#986)

This PR implements a feature of the operational HRRR, radar-derived microphysics temperature tendencies applied in the first N minutes of the forecast to improve clouds in the first few hours. Two regression tests are added.

* Turn on CICE albedo use by ATM for coupled model; update Prototype test suite to initial P8 configuration;  (ufs-community#944)

* add updates for suites, tests for initial P8 configuration
* add IOPT_SFC=3 as default; set IOPT_SFC=1 in p7 tests
* change min_seaice to 1e-6; set to 1e-11 in p7 tests
* set dddmp=0.1 in both P7 and P8. This matches the final
P7 configuration. It will change P7 baselines.
* set cdmbwd in both P7 and P8 to the final P7 configuration.
(1.0,2.2,1.0,1.0) for all resolutions. It will change P7 baselines.
* add dusan's ecflow fix for wcoss2
* update FV3 with Moorthis' snod fix
* update rt_weekly to use p8 tests
* change integration length to 5 days

* Thompson MP cloud tuning (ufs-community#1005)

* improves cloud fraction when using Thompson MP. See NCAR/ccpp-physics#809 for more details.

* Add a new HAFS regression test with Thompson MP and GFDL surface layer schemes and Corrected S4 template regression testing job card (ufs-community#995)

* Added a new regression test for HAFS. 
* Fixed the S4 RT template job card.

Co-authored-by: David Huber <david.huber@noaa.gov>
Co-authored-by: Bin Liu <Bin.Liu@noaa.gov>

* CCPP: Update consistency checks and effective radii calculation for Thompson MP (ufs-community#1000)

This PR contains changes in ccpp-physics (Update consistency checks and effective radii calculation for Thompson MP) and adds two new regression tests for both Intel and GNU that use the new prognostic cloud scheme progcld_thompson with GFS v16 and with RAP. The new cloud scheme is activated by setting icloud == 3 in input.nml.

* Rremove nems_datm related code, update CDEPS, and remove run directories that are no longer needed by other tests (ufs-community#1008)

* Add fhzero ort to coupled tests (ufs-community#1009)

* Add fhz ort to cpld tests
* Set ACCNR in rt.sh, not detect_machine.sh. Add some comments explaining where the variable should not be set or used.

Co-authored-by: Samuel Trahan <Samuel.Trahan@noaa.gov>

* Unified write_netcdf, add single file 'cubed_sphere_grid' output (ufs-community#1011)

* Unify write_netcdf and write_netcdf_parallel modules.
* Add support for writing 'cubed_sphere_grid' history files in a single netcdf file.

* Change standard name and unit of CCPP error flag variable in CCPP framework and physics (ufs-community#1013)

* updates the submodule pointers for fv3atm, ccpp-framework and ccpp-physics for:  Change standard name and unit of CCPP error flag variable in CCPP framework and physics.

* ESMF managed threading (ufs-community#1018)

Add code updates to enable the ESMF managed threading in UFS.


Co-authored-by: Gerhard Theurich <gerhard.j.theurich@noaa.gov >

* sync-ww3 (ufs-community#1026)

* sync with ww3 develop branch

* Revisions to repair iovr=5 cloud overlap option (ufs-community#1025)

* This PR addresses part 2 of issue ufs-community#748 to activate the exponential-random cloud overlap method (iovr=5) in RRTMG. RT tests were added for  iovr=4 and iovr=5 options
* change iovr to use the environment variable in control.nml.IN; modify all tests that use control.nml.IN to set env. var. iovr=3 (except for new tests); remove restart files from new tests

* 2D decomposition on write grid, bug fix in FV3GFS_IO.F90, and quad precision in stochastic physics (ufs-community#1028)

- The 2D decomposition is enabled on write grid component.
- FV3atm PR NOAA-EMC/fv3atm#468 from @MicroTed 
- stochastic physics PR NOAA-PSL/stochastic_physics#53 from Dom.

Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Dom Heinzeller <heinzell@ucar.edu>

* Update CMEPS for wave coupling (ufs-community#1034)


* add run phases to allow waves to run in the slow loop
* re-order hafs wave test
* update aliases in fd_nems.yaml
* update CMEPS to prevent advertise conflicts when s2sw is
still using connectors
* add correct masking variables to hafs nems.configure files

* 4DIAU bug fix when iau_filter_increments=T (ufs-community#990)

* Add lsm_cold_start variable for RUC LSM SCM support and gwdps bugfix (combined) (ufs-community#1037)

Co-authored-by: @SMoorthi-emc

* Update esmf 821b04 (ufs-community#1041)

* Update the version of ESMF library to 8.2.1b04
* Add the write group lon info for upp 2d decomposition test.

* Multiple output (nested) grids from fv3atm (ufs-community#1044)

Update fv3atm to support writing multiple history files for parent and nested domains.
Nested HAFS regression tests were updated to write out history files from all nested domains.

Co-authored-by: Bin Liu <bin.liu@noaa.gov>

* GPU-enabled version of Grell-Freitas convection in ccpp-physics (ufs-community#1043)

* Enable Thompson microphysics with UFS-Aerosols and update GOCART (ufs-community#1049)

This update allows the Thompson microphysics scheme to be used with prognostic aerosols (UFS-Aerosols) and updates GOCART to version 2.0.2.

* Add support for Stochastically Perturbed Parameterizations (SPP) in FV3 (ufs-community#982)

* Update CICE; add support for restart test when running single RT (was PR 1058); update MOM6 (was PR 1060) (ufs-community#1045)


* update cice
* (1) pointing MOM6 to testing branch (MOM6 main repo PR: mom-ocean/MOM6#1556 (comment))
    (2) add three new files in mom_cmake file
            core/MOM_porous_barriers.F90
            external/drifters/MOM_particles.F90
            external/drifters/MOM_particles_types.F90
    (3) explicitly set in MOM_input for all resolutions to reflect the default value changes in code to retain answer
        EPBL_MLD_BISECTION = True (default changed from T to F in this MOM6 PR)
        INTERNAL_WAVE_SPEED_BETTER_EST = False (default changed from F to T in this MOM6 PR)
        note BBL_USE_EOS now set to use the value of USE_EOS in this MOM6 PR,
        but we set BBL_USE_EOS explicit so it has no impact

* remove duplicated CHL input setting in MOM_input_template_050
* add support for restart tests while running single RT

* lateral boundary fix for regional runs (ufs-community#1050)

This PR fixes a transient increase in noise caused by incorrect processing of winds along the lateral boundary on the final acoustic time step before fresh boundary information is ingested into the model.

* Feature: NSSL Microphysics support (ufs-community#1032)

* add new RT that uses NSSL MP; add new NSSL vars to default_vars.sh and rap.nml.IN; add new field table for using NSSL MP with hail on

* p8b (with aerosols)  (ufs-community#1071)


updates all p8 tests to the p8b settings, which includes GOCART

Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>

* Split CCPP finalize into physics_finalize and (framework) finalize, fix CCPP's metadata2html.py (ufs-community#1061)

* add a cdeps test using GFSv16 input data (ufs-community#1070)

* CMAKE for WW3  (ufs-community#1089)


* Use WW3 CMake build

Co-authored-by: kgerheiser <kgerheiser@icloud.com>

* Add 'valid time' variable using ISO string format (ufs-community#1073)

* Require cmake version 3.19
* Update FV3:
   - Add 'valid time' variable using ISO string format to netcdf history files.
   - Use double precision variable to set value of 'time' attribute in wrt comp import state
   - Update ccpp/physics (setting surface-related interstitial variables for SCM prescribed-surface-flux mode)
   - Update inline_post_stub.F90 subroutine interfaces to match inline_post.F90

* Add hi-res regional test on small domain (ufs-community#1084)


* Add hi-res regional test on small domain

* Fix Lambert Conformal Conic mapping in the write component on the southern hemisphere (ufs-community#1087)

* Subroutine `lambert` in the FV3 write component has been fixed to do the mapping on the southern hemisphere correctly.
* CI fix
  - Docker container library updated. CI input data updated. Add chm_petlist_bounds.
  - Change gocart output file name for debug test
  - Changes for opnReqTest to also scan rt_gnu.conf
  - Change CI tests to noaero

Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com>

* Rectify total number of Thompson MP species and add UFS-Aerosols to S2S and S2SW (ufs-community#1118)

* Rectify total number of Thompson MP species
* Implement S2SWA/S2SA build app as the only one enabling prognostic aerosols in fully coupled model.
* Update output_fh documentation.

* Add a cdeps test and update mom6, fv3, and weekly tests (ufs-community#1136)

Co-authored-by: jiandewang <jiande.wang@noaa.gov>
Co-authored-by: WenMeng-NOAA <48260754+WenMeng-NOAA@users.noreply.github.com>
Co-authored-by: DusanJovic-NOAA <dusan.jovic@noaa.gov>

* update submodule pointer for fv3atm

* Update fms to 2021.04 (ufs-community#1135)

* Update fms to 2021.04
* Add RT for ocean IAU
* Update documentation

Co-authored-by: Phil Pegion <38869668+pjpegion@users.noreply.github.com>
Co-authored-by: Jun Wang <37633869+junwang-noaa@users.noreply.github.com>

* land surface upgrades for p8 (ufs-community#1095)

* reset the Noah-MP default options for those well tested

* P8c: Added capability for cdmbgwd(1) to scale GSL blocking drag (ufs-community#1110)

* Update ESMF to version 8.3.0bs09 (ufs-community#1146)

* Update ESMF to version 8.3.0b09
* Increase wall clock limit for tests/regional_3km to 15 min
* Update AutoRT to use tfe01 ecflow node on Jet
* Updated Cheyenne baseline epicufsrt path
* Updated Cheyenne epicufsrt hpc-stack build

Co-authored-by: JONG KIM <jong.kim@noaa.gov>
Co-authored-by: Brian Curtis <brian.curtis@noaa.gov>

* Update rrfs conus regression tests to compute fluxes.

* Remove nst_anl and a comment from the rrfs_conus13km_hrrr.nml.IN namelist template

* release/p8c: updated the TKE-EDMF PBL and saSAS cumulus scheme plus bug fix for the Thompson scheme (issue ufs-community#881) (ufs-community#1120)

* release/p8c: it is related to ccpp-physics PBL and cumulus updates plus a bug fix to the Thompson scheme

* Bring AQM model into UFS & Update MOM6 mixed FMS (ufs-community#1137)

* Add AQM submodule to integrate NOAA's Air Quality Model embedding
the U.S. EPA Community Multiscale Air Quality Model (CMAQ).

* MOM6: An alternate fix to class(*) issues with FMS 2022-01

Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>
Co-authored-by: jiandewang <jiande.wang@noaa.gov>

* Radiation changes for prototype 8 (ufs-community#1090)

* New feature in RRTMGP enabled ccpp-physics.
* Updated physics. New FV3 interstitials.
* Cleanup of RRTMGP regression tests.
* Replaced cld_mynn_ naming convention with cld_pbl_
* Some more cleanup of cloud-fraction...
* Pulled in Qingfu's changes to ccpp-physics
* New RT's for RRTMGP p8. Revert change in ccpp-physics
* Changed arguments to implied shape. in ccpp-physics
* Updated RRTMGP regression tests
* Changes to FV3/ccpp-physics.
* RRTMGP RT cleanup, again. Bug-fix in FV3/ccpp-physics
* Revert changes to names in FV3/ccpp-physics
* Cleanup p8 tests.
* More RT cleanup for p8.
* Add GP setting to non-coupled p8 tests.
* Added bounding to temperature at layer-interface used by RRTMGP.
* Reorder loop
* Combined gp sw and lw aerosol routines. Modest speedup (~4%). Add GP setting to test setup.
* Add GP settings to cpld_control_c96_noaero_p8 test.
* update test resources default_vars.sh

* HAFS moving-nesting related developments (ufs-community#1104)

* Add the storm-following or specified moving-nesting capability for both the regional and global configuration.
* Expand the UFS/HAFS FV3ATM-HYCOM ocean coupling to support the static and moving nesting configuration.
* Expand the write grid component to support outputting the moving nest domain properly.
* Five HAFS related moving-nesting regression tests are added to test these newly added features. The regional_hafs RT was removed.

Co-authored-by: Daniel Rosen <daniel.rosen@noaa.gov>
Co-authored-by: Ufuk Turuncoglu <ufuk.turuncoglu@noaa.gov>
Co-authored-by: uturuncoglu <turuncu@ucar.edu>
Co-authored-by: Dusan Jovic <dusan.jovic@noaa.gov>
Co-authored-by: William Ramstrom <William.Ramstrom@noaa.gov>
Co-authored-by: A-Kyle <kyle.ahern@noaa.gov>

* fv3atm: Split GFS_typedefs.F90, rename CCPP_interstitial, import ccpp_t from ccpp_types; contains bug fix for WW3 debug (ufs-community#1130)


Co-authored-by: Jessica Meixner <Jessica.Meixner@noaa.gov>

* Close MOM6 1deg at j=1; Update input-data for P8c (ufs-community#909)


* update all tests using 1deg MOM6/CICE6
* pre-pend ufs to existing topo edits filename when
creating the new file
* update input-data directory

* Fix uninitialized min_rand variable in Thompson MP when using SPP (ufs-community#1152)

* Updates to stochastic_physics and fv3 and CI

* A bug fix for wet scavenging of aerosols and a minor modification for moisture property calculation (ufs-community#1176)

* point to one_scheme_to_one_file PR branch (ufs-community#1175)

* merge ufs-community develop

* Point to gsl/merge-develop for FV3

* Changes missing from prior commit

* use community version of hafs_regional_atm

* merge rt.sh changes

* Point to gsl/merge-develop-to-community branch for FV3

* Revert gsl-specific changes to rt.sh

* Corrections to prior merge

* corrections to FV3_HRRR tests

* Add rrfs_smoke_conus13km_hrrr_warm test

* Add FV3_HRRR_smoke to rt builds

* Correct nst and domains_stack_size in rrfs_conus13km_hrrr.nml.IN

* Set SFCLAY_COMPUTE_FLUX=.true. for rrfs_smoke tests

* hera.gnu tests passed

* Hera BUILD_WALLTIME of 1 hour due to occasional extreme contention for intel licenses

* hera intel tests passed

* Jet tests passed

* Revert gsl-specific change to rt.sh

* Point FV3/.gitmodules to Sam's fork

* Correct issues found by reviewers in ccpp-physics and fv3atm

* Rename mynnpbl to mynnedmf in fv3atm and ccpp-physics

* FV3: Remove some empty _init and _finalize routines and update a comment in sgscloud_radpre

* Feature/lndp noahmp2 : add land perturbation scheme for Noah-MP fractional veg (ufs-community#1143)

Updates to allow land perturbation scheme to be applied for Noah-MP. Currently, for Noah-MP model parameters can perturb only vegetation fraction.

Main code changes:
-added lsm_noahmp model option to lndp_type==2 land perturbation scheme (fv3-atm, stochastic_physics)
-updated comments
-cleaned up the namelists to make the different options clearer. Removed misleading lndp_each_step variable from gfs_physics_nml, and replaced it with "lndp_model_type" in the nam_sfcperts namelist for the different forecast types (cycling DA, short forecasts, perturbing only initial conditions) (fv3-atm, stochastic_physics, ufs-weather-model)
NOTE: see note below, re: specification of smc perturbation for RAP/HRRR/etc (lndp_model_type==2)
-revised code in stochastic_physics_wrapper to only allocate arrays that will be used when passed into lndp_apply_perts (fv3-atm)
-deleted unused albedo arrays in stochastic_physics_wrapper and lndp_apply_perts (fv3-atm, stochastic_physics)
-fixed bug causing precision errors when calculating the soil moisture ice content (often ~0.0; stochastic_physics)
-fixed bug in which lndp_apply_perts wasn't called for nscyc = 0. (fv3-atm)
-increased max_nvar_lndp at Jeff Ator's request (fv3-atm)
-added a new test for the Noah-MP and lndp==2 combination (ufs-weather-model)

* Add \file to smoke files

* Update CDEPS component, add hailcast code in FV3/atmos_cubed_sphere and reduce wall clock time used for the cpld_control_c192_p8 test on Jet (ufs-community#1156)

Co-authored-by: Yunheng Wang

* Update CMEPS; Add gocart userguide (was ufs-community#1182) (ufs-community#1189)


* Update CMEPS submodule
* Add GOCART documentation

Co-authored-by: Li Pan <Li.Pan@noaa.gov>

* Combine PROD and REPRO build options into 'Release' build type (ufs-community#1171)

Combined PROD and REPRO build modes into 'Release' build type. Now only 'Debug' and 'Release' build type are supported.
Both build types must produce bit-for-bit reproducible outputs using different number of threads, mpi tasks, different domain decomposition, have reproducible restarts etc.

* Unify inline post interfaces (ufs-community#1181)

* replace pwatclm to pwat in the all diag tables under tests/parm/diag_table.

* adding bug fix from jili dong and removing U* averaging

* adding bug fix from jili dong and removing U* averaging

* Hera tests passed.

* Remove some commented-out code from cu_gf_driver in ccpp-physics.

* In FV3, turn rrtmg smoke band 10 into a model namelist variable

* Switch ccpp-physics smoke arguments to implied shape

* Remove some stops and gotos from ccpp-physics rrfs-smoke

* In FV3, remove some hard-coded constants and rename some meta entries

* In FV3, rename some standard_names, change a unit, and remove a "use physcons"

* Updated settings for NoahMP, Thompson and RRTMGP in RTs (ufs-community#1196)

* add NoahMP opt_trs to namelist
* Updated namelist settings related to NoahMP scheme, Thompson scheme and RRTMGP (issue ufs-community#1173 and ufs-community#1185)
* Updated some namelist settings and add/remove some RTs related to RRTMGP

* Compute petlist bounds for each subcomponent from number of tasks. Update CICE (ufs-community#1200)

* Instead of specifying petlist bounds (lower and upper bound), compute them using the number of tasks for each component. The function that computes petlist bounds assumes that the mediator always runs on the same tasks as ATM component, and that the Chemistry component also always runs on the same tasks as ATM.
* Update CICE submodule (@DeniseWorthen)

Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com>

* NSSL ccpp-physics bugfixes and new ccpp-framework debugging feature (ufs-community#1202)

* Merge develop

* hera.gnu tests passed

* hera.intel tests passed

* jet intel tests passed

* hera.gnu tests passed.

* fv3atm: Only add cnv_3d_ud_mf to the restart files when needed

* More warnings not to change the conus13km tests

* adjust BL_DATE in rt.sh to 20220516

* WCOSS Cray tests passed

* fv3atm: pass some chemistry varibles to mynn_bl_driver correctly

* [AutoRT] hera.gnu Job Completed.

on-behalf-of @ufs-community <brian.curtis@noaa.gov>

* [AutoRT] hera.intel Job Completed.

on-behalf-of @ufs-community <brian.curtis@noaa.gov>

* [AutoRT] cheyenne.intel Job Completed.

on-behalf-of @ufs-community <brian.curtis@noaa.gov>

* WCOSS Dell P3 tests passed

* Correct argument passing within mynn edmf

* [AutoRT] gaea.intel Job Completed.

on-behalf-of @ufs-community <brian.curtis@noaa.gov>

* [AutoRT] cheyenne.gnu Job Completed.

on-behalf-of @ufs-community <brian.curtis@noaa.gov>

* Orion RT Passed

* [AutoRT] jet.intel Job Completed.

on-behalf-of @ufs-community <brian.curtis@noaa.gov>

* Point to NOAA-EMC develop for FV3

* merge gsl institutional fork (ufs-community#1195)

Merge in GSL institutional fork.

* update FV3 to top of gsl/merge-community-to-develop

* FV3: point to sam's repo

* Point to GSL baseline areas

* FV3/ccpp/physics: Change to GSL CODEOWNERS

* hera.gnu tests passed

* hera.intel passed

* jet.intel tests passed

* Set regression test path to GSL areas on Hera

* FV3: Returned the comments on soil resistance back.

Co-authored-by: Dusan Jovic <48258889+DusanJovic-NOAA@users.noreply.github.com>
Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com>
Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>
Co-authored-by: BinLi-NOAA <bin.li@noaa.gov>
Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>
Co-authored-by: Dusan Jovic <dusan.jovic@noaa.gov>
Co-authored-by: Grant Firl <grant.firl@noaa.gov>
Co-authored-by: Dom Heinzeller <climbfuji@ymail.com>
Co-authored-by: mzhangw <mzhangw@gmail.com>
Co-authored-by: jiandewang <jiande.wang@noaa.gov>
Co-authored-by: WenMeng-NOAA <48260754+WenMeng-NOAA@users.noreply.github.com>
Co-authored-by: SMoorthi-emc <47667426+SMoorthi-emc@users.noreply.github.com>
Co-authored-by: ChunxiZhang-NOAA <49283036+ChunxiZhang-NOAA@users.noreply.github.com>
Co-authored-by: David Huber <david.huber@noaa.gov>
Co-authored-by: Bin Liu <Bin.Liu@noaa.gov>
Co-authored-by: Jun Wang <37633869+junwang-noaa@users.noreply.github.com>
Co-authored-by: Gerhard Theurich <gerhard.j.theurich@noaa.gov >
Co-authored-by: Ali.Abdolali <37336972+aliabdolali@users.noreply.github.com>
Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Dom Heinzeller <heinzell@ucar.edu>
Co-authored-by: Jeff Whitaker <jswhit@fastmail.fm>
Co-authored-by: DomHeinzeller <58610420+DomHeinzeller@users.noreply.github.com>
Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>
Co-authored-by: JeffBeck-NOAA <55201531+JeffBeck-NOAA@users.noreply.github.com>
Co-authored-by: MatthewPyle-NOAA <48285220+MatthewPyle-NOAA@users.noreply.github.com>
Co-authored-by: Jessica Meixner <jessica.meixner@noaa.gov>
Co-authored-by: kgerheiser <kgerheiser@icloud.com>
Co-authored-by: RatkoVasic-NOAA <37597874+RatkoVasic-NOAA@users.noreply.github.com>
Co-authored-by: joeolson42 <Joseph.B.Olson@noaa.gov>
Co-authored-by: Phil Pegion <38869668+pjpegion@users.noreply.github.com>
Co-authored-by: HelinWei-NOAA <48133472+HelinWei-NOAA@users.noreply.github.com>
Co-authored-by: mdtoyNOAA <73618848+mdtoyNOAA@users.noreply.github.com>
Co-authored-by: JONG KIM <jong.kim@noaa.gov>
Co-authored-by: Brian Curtis <brian.curtis@noaa.gov>
Co-authored-by: Brian Curtis <64433609+BrianCurtis-NOAA@users.noreply.github.com>
Co-authored-by: dustinswales <dustin.swales@noaa.gov>
Co-authored-by: Daniel Rosen <daniel.rosen@noaa.gov>
Co-authored-by: Ufuk Turuncoglu <ufuk.turuncoglu@noaa.gov>
Co-authored-by: uturuncoglu <turuncu@ucar.edu>
Co-authored-by: William Ramstrom <William.Ramstrom@noaa.gov>
Co-authored-by: A-Kyle <kyle.ahern@noaa.gov>
Co-authored-by: ClaraDraper-NOAA <33430543+ClaraDraper-NOAA@users.noreply.github.com>
Co-authored-by: Li Pan <Li.Pan@noaa.gov>
SamuelTrahanNOAA added a commit to NOAA-GSL/fv3atm that referenced this pull request Jun 27, 2022
* Merge NOAA-EMC develop into gsl/develop (#126)

* Feature/ccpp codeowners (NOAA-EMC#442)

* Per-file CODEOWNERS in ccpp/physics to set up automatic review requests

* Code cleanup. Remove used code/variables. Fix minor inconsistencies. (NOAA-EMC#440)

* Remove ESMF Alarm and TimeInterval variables from module_fv3_config.
* Variables nfhmax, nfhmax_hf are unused outside InitializeAdvertise. Declared them as local variables.
* There is no need to keep duplicates of all time related variables in atm_int_state when we can easily access them from atm_int_state%Atm.
* Remove redundant call to fms_init
* Add few missing ESMF_LogFoundError checks in module_fcst_grid_comp.F90
* Delete time_utils.F90. Unused.
* print only actual errors to stderr, everything else to stdout
* Move realizeConnectedCplFields to module_cplfields from module_cap_cpl
* Declare Atmos as module variable, and remove atmos_internalstate_wrapper
* Move code from clock_cplIntval to InitializeAdvertise
* Removed INTERNAL_FILE_NML from atmos_model.F90

* CCPP cloud cover change for Thompson MP associated with Xu Randall (NOAA-EMC#443)

* Remove old comments from GFS_typedefs.F90.
* Update logic that sets Model%cnvcld = .false.

* Wrapper for ccpp-physics NOAA-EMC#806, NOAA-EMC#807, NOAA-EMC#813 (NOAA-EMC#447)

* CCPP physics updates from PR NOAA-EMC#806(only diagnostic changes for RRTMGP - small diag-only RT impact for tests involving RRTMGP),NOAA-EMC#807 (only aborting model if sfc emis file not present when needed) ,NOAA-EMC#813(only SCM-specific physics changes)

* Wrapper for ccpp-physics NOAA-EMC#808 and 816 (roughness length over ice and NoahMP tsurf bugfix) (NOAA-EMC#452)

This PR contains the ccpp physics PR NOAA-EMC#808 and NOAA-EMC#816. 808 addresses an error in the momentum roughness length over tiles with ice. 816 fixes an occasional segfault bug related to the tsurf variable in NoahMP and updates to "improve snow simulation in NoahMP for P8".

* Use 'model set run clock' routine in FV3 NUOPC cap. (NOAA-EMC#450)

* Add 'SetRunClock' specialization routine to FV3 NUOPC cap, which sets fv3 model clock. FV3 cap does not keep local copy of clock (clock_fv3) as saved module variable anymore.
* Minor code cleanup.

* Clean up iovr=4 (exponential cloud overlapping method) in RRTMG (NOAA-EMC#445)

Clean up the exponential cloud overlapping method in RRTMG, which reflects the practice of the pre-2018 operational HWRF model.

* Bug fix for dimensions of eta level variables and WAM variables in Fortran code and CCPP metadata (NOAA-EMC#431)

* Add additional diagnostic arrays for radiation-microphysics development
* Bug fixes for WAM model runs with levr < levs

* Update inline post with latest UPP release upp_v10.0.11 (NOAA-EMC#449)

* Update UPP revision
* Add foundation temperaure in GFS read interface for inline post.

* Wrapper for ccpp-physics NOAA-EMC#812 (NOAA-EMC#453)

* update submodule pointer for regression testing of ccpp-physics#812

* MYNN sfclay (RAP suite) restart reproducibility, P8 suite definition files (NOAA-EMC#455)

* Fix uninitialized variable zmtnblck in ccpp/data/GFS_typedefs.F90
* Fix typo in CCPP standard name for ncnvwind in ccpp/data/GFS_typedefs.meta
* Add ten 2d variables required for MYNNSFC restart reproducibility to ccpp/driver/GFS_restart.F90
* create initial p8 suites, P8 initial SDFs as copies of FV3_GFS_v16_coupled_nsstNoahmpUGWPv1
and FV3_GFS_v16_nsstNoahmpUGWPv1

Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>

* Fixes on initializing snow depth over ice and changes z0ice (NOAA-EMC#461)

* modify FV3GFS_io.F90 by fixing errors associated with initializing snow depth over ice in the case where both land and water coexist (i.e. fractional grid case)
* z0ice is changed to 1.0 cm from 1.1cm in atmos_model.F90

* Radar-derived microphysics temperature tendencies similar to operational HRRR (NOAA-EMC#457)

- implements a feature of the operational HRRR, radar-derived microphysics temperature tendencies applied in the first N minutes of the forecast to improve clouds in the first few hours.
* HRRR-like radar-derived temperature tendencies
* Give a warning when convection is enabled with radar tten
* Fix uninitialized variable zmtnblck in ccpp/data/GFS_typedefs.F90
* Add ten 2d variables required for MYNNSFC restart reproducibility to ccpp/driver/GFS_restart.F90

* fixing snod bug in atmos_model.F90 (NOAA-EMC#465)

* Thompson MP cloud tuning (NOAA-EMC#463)

*  Improve cloud fraction when using Thompson MP. See NCAR/ccpp-physics#809 for more details.

* Feature/hwrf legacy (NOAA-EMC#459)

* Added a new suite definition file: suite_FV3_HAFS_v0_thompson_tedmf_gfdlsf.xml. This SDF is a legacy hwrf SDF but only the GFDL surface layer scheme and the Thompson scheme are kept.

* CCPP: Update consistency checks and effective radii calculation for Thompson MP (NOAA-EMC#462)

* This PR updates the submodule pointer for ccpp-physics for the changes described in NCAR/ccpp-physics#786 (Update consistency checks and effective radii calculation for Thompson MP)

* Unified write_netcdf, add single file 'cubed_sphere_grid' output (NOAA-EMC#466)

* Unify write_netcdf and write_netcdf_parallel modules.
* Add support for writing 'cubed_sphere_grid' history files in a single netcdf file
* Code refactoring and cleanup.

* Change standard name and unit of CCPP error flag variable in CCPP framework and physics (NOAA-EMC#467)

*Change standard name and unit of CCPP error flag variable in CCPP framework and physics.

* Add code to enable ESMF managed threading (NOAA-EMC#469)

* Implement ESMF-managed-threading for fcstComp and wrtComp's.

* Revisions to repair iovr=5 cloud overlap option (NOAA-EMC#471)

*This PR addresses part 2 of CCPP issue NOAA-EMC#748 to activate the exponential-random cloud overlap method (iovr=5) in RRTMG.

* Add 2d decomposition on the write grid comp (NOAA-EMC#470)

* update write grid comp to have 2D decomposition on output grid.
* combine PR#468: Bug fix in FV3GFS_io.F90 for allocation of temp2d

Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>

* 4DIAU bug when iau_filter_increments=T  (NOAA-EMC#458)

* Add lsm_cold_start variable for RUC LSM SCM support and gwdps bugfix (combined) (NOAA-EMC#475)

* add lsm_cold_start variable in GFS_typedefs.F90/meta and update ccpp/physics for testing
* includes @SMoorthi-emc 's bugfix for gwdps.f.

* add lon info in the write group (NOAA-EMC#476)

* Multiple output grids (NOAA-EMC#480)

Update fv3 cap and write grid component to enable outputting multiple domains.
This is done be creating an array of fcstGrids, and array of rout handles where each element of these arrays correspond to one atm domain.
In the write grid component updates were made to allow grid spec parameters for each output grid to be specified separately.

Co-authored-by: Gerhard Theurich <theurich@sourcespring.net>

* GPU-enabled version of Grell-Freitas convection in ccpp-physics (NOAA-EMC#479)

* Enable Thompson MP when coupling with UFS-Aerosols (NOAA-EMC#484)

* Add support for Stochastically Perturbed Parameterizations (SPP) in FV3 and add the FV3_RRFS_v1alpha SDF. (NOAA-EMC#454)

Adds the necessary code in fv3atm to allow for Stochastically Perturbed Parameterizations (SPP) in a set of RAP/HRRR-based physics parameterizations. Specific to the fv3atm repository, code in this PR defines the necessary variables associated with various SPP-related fields (e.g., logical to activate SPP, parameterization-specific SPP variables, etc.) that are then passed to ccpp-physics.

* lateral boundary fix for regional runs (NOAA-EMC#482)

* Updating pointer to ccpp/physics.

* Point to Christina's branches.

* Point to NOAA-GSL gsl/develop for ccpp/physics

Co-authored-by: Samuel Trahan (NOAA contractor) <39415369+SamuelTrahanNOAA@users.noreply.github.com>
Co-authored-by: Dusan Jovic <48258889+DusanJovic-NOAA@users.noreply.github.com>
Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>
Co-authored-by: Grant Firl <grant.firl@noaa.gov>
Co-authored-by: mzhangw <mzhangw@gmail.com>
Co-authored-by: WenMeng-NOAA <48260754+WenMeng-NOAA@users.noreply.github.com>
Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>
Co-authored-by: SMoorthi-emc <47667426+SMoorthi-emc@users.noreply.github.com>
Co-authored-by: ChunxiZhang-NOAA <49283036+ChunxiZhang-NOAA@users.noreply.github.com>
Co-authored-by: Jun Wang <37633869+junwang-noaa@users.noreply.github.com>
Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Jeff Whitaker <jswhit@fastmail.fm>
Co-authored-by: Gerhard Theurich <theurich@sourcespring.net>
Co-authored-by: DomHeinzeller <58610420+DomHeinzeller@users.noreply.github.com>
Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>
Co-authored-by: JeffBeck-NOAA <55201531+JeffBeck-NOAA@users.noreply.github.com>
Co-authored-by: MatthewPyle-NOAA <48285220+MatthewPyle-NOAA@users.noreply.github.com>
Co-authored-by: samuel.trahan <Samuel.Trahan@noaa.gov>

* Remove GF Consistency Check & Add SDFs - V2 (#127)

* - Removes consistency check in GF, allows GF to run with different or no shallow scheme
- Add additional RAP based SDF

* Point to gsl/develop for ccpp/physics

Co-authored-by: Samuel Trahan <Samuel.Trahan@noaa.gov>

* Update to top of gsl/physics to get codeowners (#130)

* updates related to MYNN-EDMF (#135)

* adding new variable (Sm3D) stability function for momentum

* update submodule pointer for ccpp-physics

* Point to gsl/develop for ccpp/physics

Co-authored-by: samuel.trahan <Samuel.Trahan@noaa.gov>

* GWD, LSM and MYNN physics updates from RRFS_dev branch (#136)

* GWD, LSM and MYNN physics updates from RRFS_dev branch

* Point to gsl/develop for ccpp/physics

Co-authored-by: samuel.trahan <Samuel.Trahan@noaa.gov>

* Point to top of gsl/develop for ccpp-physics

* gsl/develop RRFS-Smoke (#141)

* "add for smoke"

* "update to pass GNU compiler"

* "following Joe's comments to add nchem,ndvel as input to MYNN"

* "update fv3atm to point to gsl/develop-smoke"

* "to point to physics branch of gsl/develop-smoke"

* Point to gsl/develop-smoke for ccpp-physics

* "point ot gsl/develop-smoke branch"

* "update GFS_typedefs.meta with rrfs_smoke active flag"

* "update the 3rd dimension readin of GBBEPx for smoke"

* Remove an extra read that went out of bounds

* Point to NOAA-GSL gsl/develop for ccpp-physics

Co-authored-by: samuel.trahan <Samuel.Trahan@noaa.gov>

* Correction to prior commit: point ccpp-physics to NOAA-GSL gsl/develop

* Merge May 16 EMC develop to gsl/develop (#144)

* Feature/ccpp codeowners (NOAA-EMC#442)

* Per-file CODEOWNERS in ccpp/physics to set up automatic review requests

* Code cleanup. Remove used code/variables. Fix minor inconsistencies. (NOAA-EMC#440)

* Remove ESMF Alarm and TimeInterval variables from module_fv3_config.
* Variables nfhmax, nfhmax_hf are unused outside InitializeAdvertise. Declared them as local variables.
* There is no need to keep duplicates of all time related variables in atm_int_state when we can easily access them from atm_int_state%Atm.
* Remove redundant call to fms_init
* Add few missing ESMF_LogFoundError checks in module_fcst_grid_comp.F90
* Delete time_utils.F90. Unused.
* print only actual errors to stderr, everything else to stdout
* Move realizeConnectedCplFields to module_cplfields from module_cap_cpl
* Declare Atmos as module variable, and remove atmos_internalstate_wrapper
* Move code from clock_cplIntval to InitializeAdvertise
* Removed INTERNAL_FILE_NML from atmos_model.F90

* CCPP cloud cover change for Thompson MP associated with Xu Randall (NOAA-EMC#443)

* Remove old comments from GFS_typedefs.F90.
* Update logic that sets Model%cnvcld = .false.

* Wrapper for ccpp-physics NOAA-EMC#806, NOAA-EMC#807, NOAA-EMC#813 (NOAA-EMC#447)

* CCPP physics updates from PR NOAA-EMC#806(only diagnostic changes for RRTMGP - small diag-only RT impact for tests involving RRTMGP),NOAA-EMC#807 (only aborting model if sfc emis file not present when needed) ,NOAA-EMC#813(only SCM-specific physics changes)

* Wrapper for ccpp-physics NOAA-EMC#808 and 816 (roughness length over ice and NoahMP tsurf bugfix) (NOAA-EMC#452)

This PR contains the ccpp physics PR NOAA-EMC#808 and NOAA-EMC#816. 808 addresses an error in the momentum roughness length over tiles with ice. 816 fixes an occasional segfault bug related to the tsurf variable in NoahMP and updates to "improve snow simulation in NoahMP for P8".

* Use 'model set run clock' routine in FV3 NUOPC cap. (NOAA-EMC#450)

* Add 'SetRunClock' specialization routine to FV3 NUOPC cap, which sets fv3 model clock. FV3 cap does not keep local copy of clock (clock_fv3) as saved module variable anymore.
* Minor code cleanup.

* Clean up iovr=4 (exponential cloud overlapping method) in RRTMG (NOAA-EMC#445)

Clean up the exponential cloud overlapping method in RRTMG, which reflects the practice of the pre-2018 operational HWRF model.

* Bug fix for dimensions of eta level variables and WAM variables in Fortran code and CCPP metadata (NOAA-EMC#431)

* Add additional diagnostic arrays for radiation-microphysics development
* Bug fixes for WAM model runs with levr < levs

* Update inline post with latest UPP release upp_v10.0.11 (NOAA-EMC#449)

* Update UPP revision
* Add foundation temperaure in GFS read interface for inline post.

* Wrapper for ccpp-physics NOAA-EMC#812 (NOAA-EMC#453)

* update submodule pointer for regression testing of ccpp-physics#812

* MYNN sfclay (RAP suite) restart reproducibility, P8 suite definition files (NOAA-EMC#455)

* Fix uninitialized variable zmtnblck in ccpp/data/GFS_typedefs.F90
* Fix typo in CCPP standard name for ncnvwind in ccpp/data/GFS_typedefs.meta
* Add ten 2d variables required for MYNNSFC restart reproducibility to ccpp/driver/GFS_restart.F90
* create initial p8 suites, P8 initial SDFs as copies of FV3_GFS_v16_coupled_nsstNoahmpUGWPv1
and FV3_GFS_v16_nsstNoahmpUGWPv1

Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>

* Fixes on initializing snow depth over ice and changes z0ice (NOAA-EMC#461)

* modify FV3GFS_io.F90 by fixing errors associated with initializing snow depth over ice in the case where both land and water coexist (i.e. fractional grid case)
* z0ice is changed to 1.0 cm from 1.1cm in atmos_model.F90

* Radar-derived microphysics temperature tendencies similar to operational HRRR (NOAA-EMC#457)

- implements a feature of the operational HRRR, radar-derived microphysics temperature tendencies applied in the first N minutes of the forecast to improve clouds in the first few hours.
* HRRR-like radar-derived temperature tendencies
* Give a warning when convection is enabled with radar tten
* Fix uninitialized variable zmtnblck in ccpp/data/GFS_typedefs.F90
* Add ten 2d variables required for MYNNSFC restart reproducibility to ccpp/driver/GFS_restart.F90

* fixing snod bug in atmos_model.F90 (NOAA-EMC#465)

* Thompson MP cloud tuning (NOAA-EMC#463)

*  Improve cloud fraction when using Thompson MP. See NCAR/ccpp-physics#809 for more details.

* Feature/hwrf legacy (NOAA-EMC#459)

* Added a new suite definition file: suite_FV3_HAFS_v0_thompson_tedmf_gfdlsf.xml. This SDF is a legacy hwrf SDF but only the GFDL surface layer scheme and the Thompson scheme are kept.

* CCPP: Update consistency checks and effective radii calculation for Thompson MP (NOAA-EMC#462)

* This PR updates the submodule pointer for ccpp-physics for the changes described in NCAR/ccpp-physics#786 (Update consistency checks and effective radii calculation for Thompson MP)

* Unified write_netcdf, add single file 'cubed_sphere_grid' output (NOAA-EMC#466)

* Unify write_netcdf and write_netcdf_parallel modules.
* Add support for writing 'cubed_sphere_grid' history files in a single netcdf file
* Code refactoring and cleanup.

* Change standard name and unit of CCPP error flag variable in CCPP framework and physics (NOAA-EMC#467)

*Change standard name and unit of CCPP error flag variable in CCPP framework and physics.

* Add code to enable ESMF managed threading (NOAA-EMC#469)

* Implement ESMF-managed-threading for fcstComp and wrtComp's.

* Revisions to repair iovr=5 cloud overlap option (NOAA-EMC#471)

*This PR addresses part 2 of CCPP issue NOAA-EMC#748 to activate the exponential-random cloud overlap method (iovr=5) in RRTMG.

* Add 2d decomposition on the write grid comp (NOAA-EMC#470)

* update write grid comp to have 2D decomposition on output grid.
* combine PR#468: Bug fix in FV3GFS_io.F90 for allocation of temp2d

Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>

* 4DIAU bug when iau_filter_increments=T  (NOAA-EMC#458)

* Add lsm_cold_start variable for RUC LSM SCM support and gwdps bugfix (combined) (NOAA-EMC#475)

* add lsm_cold_start variable in GFS_typedefs.F90/meta and update ccpp/physics for testing
* includes @SMoorthi-emc 's bugfix for gwdps.f.

* add lon info in the write group (NOAA-EMC#476)

* Multiple output grids (NOAA-EMC#480)

Update fv3 cap and write grid component to enable outputting multiple domains.
This is done be creating an array of fcstGrids, and array of rout handles where each element of these arrays correspond to one atm domain.
In the write grid component updates were made to allow grid spec parameters for each output grid to be specified separately.

Co-authored-by: Gerhard Theurich <theurich@sourcespring.net>

* GPU-enabled version of Grell-Freitas convection in ccpp-physics (NOAA-EMC#479)

* Enable Thompson MP when coupling with UFS-Aerosols (NOAA-EMC#484)

* Add support for Stochastically Perturbed Parameterizations (SPP) in FV3 and add the FV3_RRFS_v1alpha SDF. (NOAA-EMC#454)

Adds the necessary code in fv3atm to allow for Stochastically Perturbed Parameterizations (SPP) in a set of RAP/HRRR-based physics parameterizations. Specific to the fv3atm repository, code in this PR defines the necessary variables associated with various SPP-related fields (e.g., logical to activate SPP, parameterization-specific SPP variables, etc.) that are then passed to ccpp-physics.

* lateral boundary fix for regional runs (NOAA-EMC#482)

* Feature: NSSL Microphysics support (NOAA-EMC#472)

Supports CCPP with new NSSL microphysics scheme, which has separate hydrometeor categories for graupel and hail (Mansell et al. 2010, JAS). This PR allows fv3atm to compile correctly with CCPP+NSSL, but a further update to atmos_cubed_sphere is needed to run correctly. The NSSL scheme is used in NSSL's daily FV3 convection-allowing forecasts, and the scheme itself is aimed at this scale (dx <= 4km). Testing is underway also with a preliminary Warn-on-Forecast FV3 ensemble.

* Add p8b SDF (NOAA-EMC#491)

* add p8 suite files
* fix 2phases intermediate restart

Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>
Co-authored-by: Jun Wang <jun.wang@noaa.gov>

* Split CCPP finalize into physics_finalize and (framework) finalize; minor CCPP cleanup and documentation updates (NOAA-EMC#487)

* Minor cleanup and documentation updates for CCPP; split CCPP finalize into physics_finalize and (framework) finalize

* Add 'valid time' variable using ISO string format (NOAA-EMC#492)

Add 'valid time' variable using ISO string format to netcdf history files.
Use double precision variable to set value of 'time' attribute in wrt comp import state
Update ccpp/physics (setting surface-related interstitial variables for SCM prescribed-surface-flux mode)
Update inline_post_stub.F90 subroutine interfaces to match inline_post.F90

Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Grant Firl <grant.firl@noaa.gov>

* Fix Lambert Conformal Conic mapping in the write component on the southern hemisphere (NOAA-EMC#497)

Subroutine `lambert` in the write component has been fixed to do the mapping on the southern hemisphere correctly.

* Rectify total number of Thompson MP species for PBL mixing (NOAA-EMC#509)

* update upp revision (NOAA-EMC#505)

* move some parameters in mynn surface layer to namelist options

* land surface upgrades for p8c (NOAA-EMC#499)

* Added capability for cdmbgwd(1) to scale GSL blocking drag (NOAA-EMC#502)

* release/p8c: it is related to ccpp-physics PBL and cumulus updates plus a bug fix to the Thompson scheme PR #882 (NOAA-EMC#503)

* change ccpp-physics branch to feature/p8c_pbl_cp branch

* change ccpp/physics in feature/p8c_pbl_cp branch

* revised GFS_typedefs.F90 for the paramters in the saSAS scheme

* sync upp

* Revert "sync upp"

This reverts commit 429d046.

* sync upp to 0185bb9

* Merge remote-tracking branch remote/main for ccpp-physics and update .gitmodules

* Bring AQM model into UFS (NOAA-EMC#511)

* Reset to zero coupling arrays for accumulated snow,  large scale rain, and convective rain at the end of each coupling step if coupling with chemistry model.
* Properly set kind type of literal constants defining zero and one.
* Initialize to zero canopy resistance output variablein noah/osu land-surface model subdriver.
* Re-implement radiation diagnostic output involving spectral band layer cloud optical depths (0.55 and 10 mu channels)
to prevent floating invalid errors due to uninitialized optical depth arrays.
* Temporarily disable filling export fields during the NUOPC Realize phase since it breaks coupling with aerosol component.
* Increase maximum number of input aerosol scavenging factors to accommodate AQM/CMAQ 5.2.1 chemical tracers.
* Remove inst_pres_height_surface from chemistryFieldNames as its imported already elsewhere

Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>

* RRTMGP for prototype 8 (NOAA-EMC#498)

* Add explict treatment of convective cloud to RRTMGP.
* Some cleanup and bug fixes from previous commit. working now with Thompson MP plus radiatively active convective cloud.
* Updated physics. New interstitials.
* Added explicit MYNN-EDMF cloud coupling to RRTMGP.
* Added switches for sgs clouds in GP.
* Replaced cld_mynn_ naming convention with cld_pbl_
* Fixed inconsistency between G/GP in Thompson MP.
* Some more cleanup of cloud-fraction...
* Pulled in Qingfu's changes to ccpp-physics
* Synced physics with main
* Updated physics.
* Revert change from previous commits (sampling of different cloud types). New SDFs for p8c
* Changed arguments to implied shape.
* Renamed scheme file in SDF. Now common to both RRTMG and RRTMGP SDFs
* Syntax error in ccpp-physics
* Revert changes to names
* Renamed p8_rrtmgp SDFs
* Added bounding to temperature at layer-interface used by RRTMGP.
* Reorder loop
* Combined gp sw and lw aerosol routines. Modest speedup (~4%)

* HAFS related moving-nesting, nest coupling, and movable output grid capabilities (NOAA-EMC#501)

* Add the storm-following or specified moving-nesting capability for both the regional and global configuration.
* Expand the UFS/HAFS FV3ATM-HYCOM ocean coupling to support the static and moving nesting configuration.
* Expand the write grid component to support outputting the moving nest domain properly.
* Five HAFS related moving-nesting regression tests are added to test these newly added features. The regional_hafs RT was removed.

Co-authored-by: Dom Heinzeller <climbfuji@ymail.com>
Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>
Co-authored-by: Grant Firl <grantf@ucar.edu>
Co-authored-by: Daniel Rosen <daniel.rosen@noaa.gov>
Co-authored-by: man.zhang <Man.Zhang@noaa.gov>
Co-authored-by: William Ramstrom <William.Ramstrom@noaa.gov>
Co-authored-by: Ufuk Turuncoglu <ufuk.turuncoglu@noaa.gov>
Co-authored-by: Dusan Jovic <dusan.jovic@noaa.gov>
Co-authored-by: A-Kyle <kyle.ahern@noaa.gov>
Co-authored-by: Gerhard Theurich <theurich@sourcespring.net>
Co-authored-by: Dusan Jovic <48258889+DusanJovic-NOAA@users.noreply.github.com>

* Split GFS_typedefs.F90, rename CCPP_interstitial, import ccpp_t from ccpp_types (NOAA-EMC#508)

* Split GFS_typedefs.F90: move GFS_interstitial_type to CCPP_typedefs.F90; Rename CCPP_interstitial to GFDL_interstitial; Import ccpp_t from ccpp_types instead of ccpp_api
* Add bugfix in ccpp/data/GFS_typedefs.meta: mark input_nml_file as active only if the number of lines in the file is greater than zero

* Fix uninitialized min_rand variable in Thompson MP when using SPP (NOAA-EMC#516)

* Pass SPP namelist entries outside of stochastic physics routines.

* Fixes to SPP metadata

* Add active attribute to spp_prt_list

* Add active metadata flag to spp_stddev_cutoff

* Add active metadata flag for spp_var_list

* A bug fix for wet scavenging of aerosols and a minor modification for moisture property calculation  (NOAA-EMC#520)

* Update ccpp_prebuild_config.py for one-to-one CCPP physics schemes/files (NOAA-EMC#519)

* point to gsl/merge-develop branch

* Point to gsl/merge-develop-to-community branch for ccpp/physics

* Corrections to merge + joe updates

* Point .gitmodules to Sam's fork

* Correct issues found by reviewers

* Rename mynnpbl to mynnedmf

* Remove some empty _init and _finalize routines and update a comment in sgscloud_radpre

* Feature/lndp noahmp2 : add land perturbation scheme for Noah-MP fractional veg  (NOAA-EMC#513)

Updates to allow land perturbation scheme to be applied for Noah-MP.

Main code changes:
-added lsm_noahmp model option to lndp_type==2 land perturbation scheme (fv3-atm, stochastic_physics)
-updated comments
-cleaned up the namelists to make the different options clearer. Removed misleading lndp_each_step variable from gfs_physics_nml, and replaced it with "lndp_model_type" in the nam_sfcperts namelist for the different forecast types (cycling DA, short forecasts, perturbing only initial conditions) (fv3-atm, stochastic_physics, ufs-weather-model)
NOTE: see note below, re: specification of smc perturbation for RAP/HRRR/etc (lndp_model_type==2)
-revised code in stochastic_physics_wrapper to only allocate arrays that will be used when passed into lndp_apply_perts (fv3-atm)
-deleted unused albedo arrays in stochastic_physics_wrapper and lndp_apply_perts (fv3-atm, stochastic_physics)
-fixed bug causing precision errors when calculating the soil moisture ice content (often ~0.0; stochastic_physics)
-fixed bug in which lndp_apply_perts wasn't called for nscyc = 0. (fv3-atm)
-increased max_nvar_lndp at Jeff Ator's request (fv3-atm)
-added a new test for the Noah-MP and lndp==2 combination (ufs-weather-model)

* Add \file to smoke files

* Add hailcast code in atmos_cubed_sphere  (NOAA-EMC#528)

* Combine PROD and REPRO build options into 'Release' build type (NOAA-EMC#524)

Combined PROD and REPRO build modes into 'Release' build type. Now only 'Debug' and 'Release' build type are supported.
Both build types must produce bit-for-bit reproducible outputs using different number of threads, mpi tasks, different domain decomposition, have reproducible restarts etc.

* Unify inline post (NOAA-EMC#521)

* Unify global and regional inline posts and add bug fix for dx/dy computation.
* Update upp revision to dc8bc68
* Set up different configurations of read max/min 2m T for global and regional FV3.

* adding bug fix from jili dong and removing U* averaging

* adding bug fix from jili dong and removing U* averaging

* Remove some commented-out code from cu_gf_driver in ccpp-physics.

* Turn rrtmg smoke band 10 into a model namelist variable

* Switch ccpp-physics smoke arguments to implied shape

* Remove some stops and gotos from ccpp-physics smoke

* Remove some hard-coded constants and rename some meta entries

* Change RRTMGP to RRTMG in two P8 SDFs (NOAA-EMC#527)

* Change RRTMGP to RRTMG in suite_FV3_GFS_v17_p8 and suite_FV3_GFS_v17_coupled_p8
* deleted or modified some SDFs related to RRTMGP or Thompson schemes
* added a new SDF file for P8 with rrtmgp

* Rename some standard_names, change a unit, and remove a "use physcons"

* NSSL ccpp-physics bugfixes and new ccpp-framework debugging feature (NOAA-EMC#529)

* Set num3d correctly when using radar_tten

* Only add cnv_3d_ud_mf to the restart files when needed

* Pass some chemistry varibles to mynn_bl_driver correctly

* Correct argument passing within mynn edmf

* point to NCAR main ccpp/physics

* merge gsl institutional fork (NOAA-EMC#526)

Merge in GSL institutional fork

* point to sam's repo

* ccpp/physics: Change to GSL CODEOWNERS

* Returned the comments on soil resistance back.

* point to gsl/develop for ccpp/physics

Co-authored-by: Dusan Jovic <48258889+DusanJovic-NOAA@users.noreply.github.com>
Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>
Co-authored-by: Grant Firl <grant.firl@noaa.gov>
Co-authored-by: mzhangw <mzhangw@gmail.com>
Co-authored-by: WenMeng-NOAA <48260754+WenMeng-NOAA@users.noreply.github.com>
Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>
Co-authored-by: SMoorthi-emc <47667426+SMoorthi-emc@users.noreply.github.com>
Co-authored-by: ChunxiZhang-NOAA <49283036+ChunxiZhang-NOAA@users.noreply.github.com>
Co-authored-by: Jun Wang <37633869+junwang-noaa@users.noreply.github.com>
Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Jeff Whitaker <jswhit@fastmail.fm>
Co-authored-by: Gerhard Theurich <theurich@sourcespring.net>
Co-authored-by: DomHeinzeller <58610420+DomHeinzeller@users.noreply.github.com>
Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>
Co-authored-by: JeffBeck-NOAA <55201531+JeffBeck-NOAA@users.noreply.github.com>
Co-authored-by: MatthewPyle-NOAA <48285220+MatthewPyle-NOAA@users.noreply.github.com>
Co-authored-by: Ted Mansell <37668594+MicroTed@users.noreply.github.com>
Co-authored-by: Jessica Meixner <jessica.meixner@noaa.gov>
Co-authored-by: Jun Wang <jun.wang@noaa.gov>
Co-authored-by: joeolson42 <Joseph.B.Olson@noaa.gov>
Co-authored-by: HelinWei-NOAA <48133472+HelinWei-NOAA@users.noreply.github.com>
Co-authored-by: mdtoyNOAA <73618848+mdtoyNOAA@users.noreply.github.com>
Co-authored-by: Brian Curtis <64433609+BrianCurtis-NOAA@users.noreply.github.com>
Co-authored-by: dustinswales <dustin.swales@noaa.gov>
Co-authored-by: Bin Liu <bin.liu@noaa.gov>
Co-authored-by: Dom Heinzeller <climbfuji@ymail.com>
Co-authored-by: Grant Firl <grantf@ucar.edu>
Co-authored-by: Daniel Rosen <daniel.rosen@noaa.gov>
Co-authored-by: man.zhang <Man.Zhang@noaa.gov>
Co-authored-by: William Ramstrom <William.Ramstrom@noaa.gov>
Co-authored-by: Ufuk Turuncoglu <ufuk.turuncoglu@noaa.gov>
Co-authored-by: Dusan Jovic <dusan.jovic@noaa.gov>
Co-authored-by: A-Kyle <kyle.ahern@noaa.gov>
Co-authored-by: ClaraDraper-NOAA <33430543+ClaraDraper-NOAA@users.noreply.github.com>
Co-authored-by: BinLi-NOAA <bin.li@noaa.gov>

* followup to gsl/develop PR (#145)

* Fix from Joe to remove code that was added back in by the big merge.

* Change suite_FV3_GFS_v15_thompson_mynn to not use mynn surface layer scheme (from Tanya Smirnova)

* move physics/smoke to physics/physics/smoke

* Switch to rad_sw_pre in ccpp/suites/suite_FV3_GFS_v15_thompson_mynn.xml which replaced rrtmg_sw_pre.F90 a while ago

* point to gsl/develop for ccpp/physics

* update submodule pointer for ccpp-physics

* emergency fix to mynn theta calculation

* Merge dom's rap GF b4b fix

* point to gsl/develop

* merge gsl/develop to ccpp/physics

* changes to get this to compile

* Convince FV3GFS_io to use FV3GFS_io_generic

* Remove accidental changes to ccpp/suites/suite_FV3_HAFS_v0_thompson_tedmf_gfdlsf.xml

* FV3GFS NetCDF I/O works again

* Remove unneeded code in ccpp/physics

* fix typos

* Bugfix: writing to pointer before allocating it

* Default diag_flux is always false

* point to RRFS_dev for ccpp/physics

Co-authored-by: Christina Holt <56881914+christinaholtNOAA@users.noreply.github.com>
Co-authored-by: Dusan Jovic <48258889+DusanJovic-NOAA@users.noreply.github.com>
Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>
Co-authored-by: Grant Firl <grant.firl@noaa.gov>
Co-authored-by: mzhangw <mzhangw@gmail.com>
Co-authored-by: WenMeng-NOAA <48260754+WenMeng-NOAA@users.noreply.github.com>
Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>
Co-authored-by: SMoorthi-emc <47667426+SMoorthi-emc@users.noreply.github.com>
Co-authored-by: ChunxiZhang-NOAA <49283036+ChunxiZhang-NOAA@users.noreply.github.com>
Co-authored-by: Jun Wang <37633869+junwang-noaa@users.noreply.github.com>
Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Jeff Whitaker <jswhit@fastmail.fm>
Co-authored-by: Gerhard Theurich <theurich@sourcespring.net>
Co-authored-by: DomHeinzeller <58610420+DomHeinzeller@users.noreply.github.com>
Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>
Co-authored-by: JeffBeck-NOAA <55201531+JeffBeck-NOAA@users.noreply.github.com>
Co-authored-by: MatthewPyle-NOAA <48285220+MatthewPyle-NOAA@users.noreply.github.com>
Co-authored-by: Hannah C Barnes <38660891+hannahcbarnes@users.noreply.github.com>
Co-authored-by: Joseph Olson <joeolson42@users.noreply.github.com>
Co-authored-by: mdtoyNOAA <73618848+mdtoyNOAA@users.noreply.github.com>
Co-authored-by: haiqinli <38666296+haiqinli@users.noreply.github.com>
Co-authored-by: Ted Mansell <37668594+MicroTed@users.noreply.github.com>
Co-authored-by: Jessica Meixner <jessica.meixner@noaa.gov>
Co-authored-by: Jun Wang <jun.wang@noaa.gov>
Co-authored-by: joeolson42 <Joseph.B.Olson@noaa.gov>
Co-authored-by: HelinWei-NOAA <48133472+HelinWei-NOAA@users.noreply.github.com>
Co-authored-by: Brian Curtis <64433609+BrianCurtis-NOAA@users.noreply.github.com>
Co-authored-by: dustinswales <dustin.swales@noaa.gov>
Co-authored-by: Bin Liu <bin.liu@noaa.gov>
Co-authored-by: Dom Heinzeller <climbfuji@ymail.com>
Co-authored-by: Grant Firl <grantf@ucar.edu>
Co-authored-by: Daniel Rosen <daniel.rosen@noaa.gov>
Co-authored-by: man.zhang <Man.Zhang@noaa.gov>
Co-authored-by: William Ramstrom <William.Ramstrom@noaa.gov>
Co-authored-by: Ufuk Turuncoglu <ufuk.turuncoglu@noaa.gov>
Co-authored-by: Dusan Jovic <dusan.jovic@noaa.gov>
Co-authored-by: A-Kyle <kyle.ahern@noaa.gov>
Co-authored-by: ClaraDraper-NOAA <33430543+ClaraDraper-NOAA@users.noreply.github.com>
Co-authored-by: BinLi-NOAA <bin.li@noaa.gov>
SamuelTrahanNOAA added a commit to NOAA-GSL/ufs-weather-model that referenced this pull request Jun 27, 2022
* Add ufs driver. Remove NEMS submodule. Update CMEPS. Update CDEPS. Feature/ort addition.  (#931)

* Add UFS driver, a copy of two fortran files from the NEMS repository. Remove NEMS submodule.

* Update CMEPS for current escomp/master. Implement the ability to write mediator history files for ATM on tiles rather than mesh

* Update CDEPS component. The updated CDEPS includes code updates for GFS v16 data mode.

* Operation requirement test (ORT) for:
  - cpld_control_p7 thread and decomposition tests
  - cpld_bmark_p7 thread and decomposition tests
  - Due to the wave component in these tests, restart and debug do not work

Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com>
Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>
Co-authored-by: BinLi-NOAA <bin.li@noaa.gov>

* fv3atm cap cleanup (#954)

FV3 nuopc cap code cleanup. Remove used code/variables. Fix minor inconsistencies.

* CCPP cloud cover change for Thompson MP associated with Xu Randall, several bug fixes for regression testing scripts (#962)

This PR contains code changes 1) to remove old comments in GFS_typedefs.F90, increase cloud cover for Thompson MP by passing convective cloud water mixing ratio to progcld6. 2) Updates for auto fixes

Co-authored-by: Dusan Jovic <dusan.jovic@noaa.gov>

* ccpp-physics #806, #807, #813, and ort modifications (#970)

- CCPP Physics changes: see Wrapper for ccpp-physics #806, #807, #813, NOAA-EMC/fv3atm#447.
- UFS weather model changes for ORT updates following changes in Fix Auto-RT on Gaea, Jet and Orion #965

* Wrapper for ccpp-physics #808 and 816 (roughness length over ice and NoahMP tsurf bugfix) (#971)

* update submodule pointer for regression testing for ccpp-physics #808,816: roughness length over ice and NoahMP tsurf bugfix
* Reduce time step from 600s to 450s for tests control_csawmg and control_csawmg_debug

Co-authored-by: Dom Heinzeller <climbfuji@ymail.com>
Co-authored-by: climbfuji <dom.heinzeller@icloud.com>

* Use 'model set run clock' routine in FV3 NUOPC cap. (#967)

* Update FV3 cap by adding 'model set run clock' specialization routine. Because FV3 cap does not modify 'Earth clock' anymore in case of a restarted run, that adjustment is now done in the UFS driver.
* Fix model_configure file for two tests to define `fhrot` parameter.

* Revert iovr=4 (exponential cloud overlap method) in RRTMG (#963)

Revert/cleanup iovr=4 in RRTMG per developer's request. The current exponential cloud overlapping method used in RRTMG represents the method used in the pre-2018 operational HWRF model.

* Add fhzero ort test (#977)

* Fhzero ORT applies to the two tests `control` and `control_p7`
* Test can be run by invoking, for example, `./opnReqTest -n control -c fhz`

* Add a new CDEPS test to use GSFv16 data mode in CDEPS ,update rt_weekly.conf and update MOM6 to main branch (20211220) (#976)

* add a new cdeps test using GFS data mode
* update rt_weekly.conf to include cpld_bmark_p7_35d tests
* update MOM6

Co-authored-by: jiandewang <jiande.wang@noaa.gov>

* Bug fixes for WAM physics and regression tests, remove duplicate modules from ccpp-physics CMakeLists.txt (#947)

*adds a new regression test control_wam_debug and fixes problems with the existing control_wam test (e.g. IAER was not set as intended).

* Update global inline post for outputting foundation temperature (#968)

Update upp submodule in fv3atm with latest UPP version for outputting foundation temperature and fixing land-sea mask issue in global inline post.

* Regression test bug fixes for running on Jet or with Rocoto (#981)

* Bug fixes for running on jet or with rocoto
1. Correctly detect Jet when deciding build job runtimes
2. Do not delete out and err files in compile & test jobs
3. Correct dependencies for WW3 in Rocoto
4. Do not hard-code ACCNR to h-nems
This PR also contains fv3/ccpp physics (CCPP PR#812) updates that implements a vertically-integrated formula to properly compute tracer column mass and ensure mass conservation for all tracers except TKE. See issue NCAR/ccpp-physics#811 for details

* MYNN sfclay (RAP suite) restart reproducibility (#984)

*This PR turns on the rap_restart tests. Bit-for-bit reproducibility is achieved with the bug fixes for MYNN sfclay in the associated fv3atm and ccpp-physics PRs.

* fixes on initializing snow over fractional ice and GFS_surface_composite and z0ice value change (#996)

* Radar-derived microphysics temperature tendencies similar to operational HRRR (#986)

This PR implements a feature of the operational HRRR, radar-derived microphysics temperature tendencies applied in the first N minutes of the forecast to improve clouds in the first few hours. Two regression tests are added.

* Turn on CICE albedo use by ATM for coupled model; update Prototype test suite to initial P8 configuration;  (#944)

* add updates for suites, tests for initial P8 configuration
* add IOPT_SFC=3 as default; set IOPT_SFC=1 in p7 tests
* change min_seaice to 1e-6; set to 1e-11 in p7 tests
* set dddmp=0.1 in both P7 and P8. This matches the final
P7 configuration. It will change P7 baselines.
* set cdmbwd in both P7 and P8 to the final P7 configuration.
(1.0,2.2,1.0,1.0) for all resolutions. It will change P7 baselines.
* add dusan's ecflow fix for wcoss2
* update FV3 with Moorthis' snod fix
* update rt_weekly to use p8 tests
* change integration length to 5 days

* Thompson MP cloud tuning (#1005)

* improves cloud fraction when using Thompson MP. See NCAR/ccpp-physics#809 for more details.

* Add a new HAFS regression test with Thompson MP and GFDL surface layer schemes and Corrected S4 template regression testing job card (#995)

* Added a new regression test for HAFS. 
* Fixed the S4 RT template job card.

Co-authored-by: David Huber <david.huber@noaa.gov>
Co-authored-by: Bin Liu <Bin.Liu@noaa.gov>

* CCPP: Update consistency checks and effective radii calculation for Thompson MP (#1000)

This PR contains changes in ccpp-physics (Update consistency checks and effective radii calculation for Thompson MP) and adds two new regression tests for both Intel and GNU that use the new prognostic cloud scheme progcld_thompson with GFS v16 and with RAP. The new cloud scheme is activated by setting icloud == 3 in input.nml.

* Rremove nems_datm related code, update CDEPS, and remove run directories that are no longer needed by other tests (#1008)

* Add fhzero ort to coupled tests (#1009)

* Add fhz ort to cpld tests
* Set ACCNR in rt.sh, not detect_machine.sh. Add some comments explaining where the variable should not be set or used.

Co-authored-by: Samuel Trahan <Samuel.Trahan@noaa.gov>

* Unified write_netcdf, add single file 'cubed_sphere_grid' output (#1011)

* Unify write_netcdf and write_netcdf_parallel modules.
* Add support for writing 'cubed_sphere_grid' history files in a single netcdf file.

* Change standard name and unit of CCPP error flag variable in CCPP framework and physics (#1013)

* updates the submodule pointers for fv3atm, ccpp-framework and ccpp-physics for:  Change standard name and unit of CCPP error flag variable in CCPP framework and physics.

* ESMF managed threading (#1018)

Add code updates to enable the ESMF managed threading in UFS.


Co-authored-by: Gerhard Theurich <gerhard.j.theurich@noaa.gov >

* sync-ww3 (#1026)

* sync with ww3 develop branch

* Revisions to repair iovr=5 cloud overlap option (#1025)

* This PR addresses part 2 of issue #748 to activate the exponential-random cloud overlap method (iovr=5) in RRTMG. RT tests were added for  iovr=4 and iovr=5 options
* change iovr to use the environment variable in control.nml.IN; modify all tests that use control.nml.IN to set env. var. iovr=3 (except for new tests); remove restart files from new tests

* 2D decomposition on write grid, bug fix in FV3GFS_IO.F90, and quad precision in stochastic physics (#1028)

- The 2D decomposition is enabled on write grid component.
- FV3atm PR NOAA-EMC/fv3atm#468 from @MicroTed 
- stochastic physics PR noaa-psd/stochastic_physics#53 from Dom.

Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Dom Heinzeller <heinzell@ucar.edu>

* Update CMEPS for wave coupling (#1034)


* add run phases to allow waves to run in the slow loop
* re-order hafs wave test
* update aliases in fd_nems.yaml
* update CMEPS to prevent advertise conflicts when s2sw is
still using connectors
* add correct masking variables to hafs nems.configure files

* 4DIAU bug fix when iau_filter_increments=T (#990)

* Add lsm_cold_start variable for RUC LSM SCM support and gwdps bugfix (combined) (#1037)

Co-authored-by: @SMoorthi-emc

* Update esmf 821b04 (#1041)

* Update the version of ESMF library to 8.2.1b04
* Add the write group lon info for upp 2d decomposition test.

* Multiple output (nested) grids from fv3atm (#1044)

Update fv3atm to support writing multiple history files for parent and nested domains.
Nested HAFS regression tests were updated to write out history files from all nested domains.

Co-authored-by: Bin Liu <bin.liu@noaa.gov>

* GPU-enabled version of Grell-Freitas convection in ccpp-physics (#1043)

* Enable Thompson microphysics with UFS-Aerosols and update GOCART (#1049)

This update allows the Thompson microphysics scheme to be used with prognostic aerosols (UFS-Aerosols) and updates GOCART to version 2.0.2.

* Add support for Stochastically Perturbed Parameterizations (SPP) in FV3 (#982)

* Update CICE; add support for restart test when running single RT (was PR 1058); update MOM6 (was PR 1060) (#1045)


* update cice
* (1) pointing MOM6 to testing branch (MOM6 main repo PR: https://github.com/mom-ocean/MOM6/pull/1556#issuecomment-1043374317)
    (2) add three new files in mom_cmake file
            core/MOM_porous_barriers.F90
            external/drifters/MOM_particles.F90
            external/drifters/MOM_particles_types.F90
    (3) explicitly set in MOM_input for all resolutions to reflect the default value changes in code to retain answer
        EPBL_MLD_BISECTION = True (default changed from T to F in this MOM6 PR)
        INTERNAL_WAVE_SPEED_BETTER_EST = False (default changed from F to T in this MOM6 PR)
        note BBL_USE_EOS now set to use the value of USE_EOS in this MOM6 PR,
        but we set BBL_USE_EOS explicit so it has no impact

* remove duplicated CHL input setting in MOM_input_template_050
* add support for restart tests while running single RT

* lateral boundary fix for regional runs (#1050)

This PR fixes a transient increase in noise caused by incorrect processing of winds along the lateral boundary on the final acoustic time step before fresh boundary information is ingested into the model.

* Feature: NSSL Microphysics support (#1032)

* add new RT that uses NSSL MP; add new NSSL vars to default_vars.sh and rap.nml.IN; add new field table for using NSSL MP with hail on

* Merge ufs-community develop into gsl/develop (#119)

* Add ufs driver. Remove NEMS submodule. Update CMEPS. Update CDEPS. Feature/ort addition.  (#931)

* Add UFS driver, a copy of two fortran files from the NEMS repository. Remove NEMS submodule.

* Update CMEPS for current escomp/master. Implement the ability to write mediator history files for ATM on tiles rather than mesh

* Update CDEPS component. The updated CDEPS includes code updates for GFS v16 data mode.

* Operation requirement test (ORT) for:
  - cpld_control_p7 thread and decomposition tests
  - cpld_bmark_p7 thread and decomposition tests
  - Due to the wave component in these tests, restart and debug do not work

Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com>
Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>
Co-authored-by: BinLi-NOAA <bin.li@noaa.gov>

* fv3atm cap cleanup (#954)

FV3 nuopc cap code cleanup. Remove used code/variables. Fix minor inconsistencies.

* CCPP cloud cover change for Thompson MP associated with Xu Randall, several bug fixes for regression testing scripts (#962)

This PR contains code changes 1) to remove old comments in GFS_typedefs.F90, increase cloud cover for Thompson MP by passing convective cloud water mixing ratio to progcld6. 2) Updates for auto fixes

Co-authored-by: Dusan Jovic <dusan.jovic@noaa.gov>

* ccpp-physics #806, #807, #813, and ort modifications (#970)

- CCPP Physics changes: see Wrapper for ccpp-physics #806, #807, #813, NOAA-EMC/fv3atm#447.
- UFS weather model changes for ORT updates following changes in Fix Auto-RT on Gaea, Jet and Orion #965

* Wrapper for ccpp-physics #808 and 816 (roughness length over ice and NoahMP tsurf bugfix) (#971)

* update submodule pointer for regression testing for ccpp-physics #808,816: roughness length over ice and NoahMP tsurf bugfix
* Reduce time step from 600s to 450s for tests control_csawmg and control_csawmg_debug

Co-authored-by: Dom Heinzeller <climbfuji@ymail.com>
Co-authored-by: climbfuji <dom.heinzeller@icloud.com>

* Use 'model set run clock' routine in FV3 NUOPC cap. (#967)

* Update FV3 cap by adding 'model set run clock' specialization routine. Because FV3 cap does not modify 'Earth clock' anymore in case of a restarted run, that adjustment is now done in the UFS driver.
* Fix model_configure file for two tests to define `fhrot` parameter.

* Revert iovr=4 (exponential cloud overlap method) in RRTMG (#963)

Revert/cleanup iovr=4 in RRTMG per developer's request. The current exponential cloud overlapping method used in RRTMG represents the method used in the pre-2018 operational HWRF model.

* Add fhzero ort test (#977)

* Fhzero ORT applies to the two tests `control` and `control_p7`
* Test can be run by invoking, for example, `./opnReqTest -n control -c fhz`

* Add a new CDEPS test to use GSFv16 data mode in CDEPS ,update rt_weekly.conf and update MOM6 to main branch (20211220) (#976)

* add a new cdeps test using GFS data mode
* update rt_weekly.conf to include cpld_bmark_p7_35d tests
* update MOM6

Co-authored-by: jiandewang <jiande.wang@noaa.gov>

* Bug fixes for WAM physics and regression tests, remove duplicate modules from ccpp-physics CMakeLists.txt (#947)

*adds a new regression test control_wam_debug and fixes problems with the existing control_wam test (e.g. IAER was not set as intended).

* Update global inline post for outputting foundation temperature (#968)

Update upp submodule in fv3atm with latest UPP version for outputting foundation temperature and fixing land-sea mask issue in global inline post.

* Regression test bug fixes for running on Jet or with Rocoto (#981)

* Bug fixes for running on jet or with rocoto
1. Correctly detect Jet when deciding build job runtimes
2. Do not delete out and err files in compile & test jobs
3. Correct dependencies for WW3 in Rocoto
4. Do not hard-code ACCNR to h-nems
This PR also contains fv3/ccpp physics (CCPP PR#812) updates that implements a vertically-integrated formula to properly compute tracer column mass and ensure mass conservation for all tracers except TKE. See issue NCAR/ccpp-physics#811 for details

* MYNN sfclay (RAP suite) restart reproducibility (#984)

*This PR turns on the rap_restart tests. Bit-for-bit reproducibility is achieved with the bug fixes for MYNN sfclay in the associated fv3atm and ccpp-physics PRs.

* fixes on initializing snow over fractional ice and GFS_surface_composite and z0ice value change (#996)

* Radar-derived microphysics temperature tendencies similar to operational HRRR (#986)

This PR implements a feature of the operational HRRR, radar-derived microphysics temperature tendencies applied in the first N minutes of the forecast to improve clouds in the first few hours. Two regression tests are added.

* Turn on CICE albedo use by ATM for coupled model; update Prototype test suite to initial P8 configuration;  (#944)

* add updates for suites, tests for initial P8 configuration
* add IOPT_SFC=3 as default; set IOPT_SFC=1 in p7 tests
* change min_seaice to 1e-6; set to 1e-11 in p7 tests
* set dddmp=0.1 in both P7 and P8. This matches the final
P7 configuration. It will change P7 baselines.
* set cdmbwd in both P7 and P8 to the final P7 configuration.
(1.0,2.2,1.0,1.0) for all resolutions. It will change P7 baselines.
* add dusan's ecflow fix for wcoss2
* update FV3 with Moorthis' snod fix
* update rt_weekly to use p8 tests
* change integration length to 5 days

* Thompson MP cloud tuning (#1005)

* improves cloud fraction when using Thompson MP. See NCAR/ccpp-physics#809 for more details.

* Add a new HAFS regression test with Thompson MP and GFDL surface layer schemes and Corrected S4 template regression testing job card (#995)

* Added a new regression test for HAFS. 
* Fixed the S4 RT template job card.

Co-authored-by: David Huber <david.huber@noaa.gov>
Co-authored-by: Bin Liu <Bin.Liu@noaa.gov>

* CCPP: Update consistency checks and effective radii calculation for Thompson MP (#1000)

This PR contains changes in ccpp-physics (Update consistency checks and effective radii calculation for Thompson MP) and adds two new regression tests for both Intel and GNU that use the new prognostic cloud scheme progcld_thompson with GFS v16 and with RAP. The new cloud scheme is activated by setting icloud == 3 in input.nml.

* Rremove nems_datm related code, update CDEPS, and remove run directories that are no longer needed by other tests (#1008)

* Add fhzero ort to coupled tests (#1009)

* Add fhz ort to cpld tests
* Set ACCNR in rt.sh, not detect_machine.sh. Add some comments explaining where the variable should not be set or used.

Co-authored-by: Samuel Trahan <Samuel.Trahan@noaa.gov>

* Unified write_netcdf, add single file 'cubed_sphere_grid' output (#1011)

* Unify write_netcdf and write_netcdf_parallel modules.
* Add support for writing 'cubed_sphere_grid' history files in a single netcdf file.

* Change standard name and unit of CCPP error flag variable in CCPP framework and physics (#1013)

* updates the submodule pointers for fv3atm, ccpp-framework and ccpp-physics for:  Change standard name and unit of CCPP error flag variable in CCPP framework and physics.

* ESMF managed threading (#1018)

Add code updates to enable the ESMF managed threading in UFS.


Co-authored-by: Gerhard Theurich <gerhard.j.theurich@noaa.gov >

* sync-ww3 (#1026)

* sync with ww3 develop branch

* Revisions to repair iovr=5 cloud overlap option (#1025)

* This PR addresses part 2 of issue #748 to activate the exponential-random cloud overlap method (iovr=5) in RRTMG. RT tests were added for  iovr=4 and iovr=5 options
* change iovr to use the environment variable in control.nml.IN; modify all tests that use control.nml.IN to set env. var. iovr=3 (except for new tests); remove restart files from new tests

* 2D decomposition on write grid, bug fix in FV3GFS_IO.F90, and quad precision in stochastic physics (#1028)

- The 2D decomposition is enabled on write grid component.
- FV3atm PR NOAA-EMC/fv3atm#468 from @MicroTed 
- stochastic physics PR noaa-psd/stochastic_physics#53 from Dom.

Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Dom Heinzeller <heinzell@ucar.edu>

* Update CMEPS for wave coupling (#1034)


* add run phases to allow waves to run in the slow loop
* re-order hafs wave test
* update aliases in fd_nems.yaml
* update CMEPS to prevent advertise conflicts when s2sw is
still using connectors
* add correct masking variables to hafs nems.configure files

* 4DIAU bug fix when iau_filter_increments=T (#990)

* Add lsm_cold_start variable for RUC LSM SCM support and gwdps bugfix (combined) (#1037)

Co-authored-by: @SMoorthi-emc

* Update esmf 821b04 (#1041)

* Update the version of ESMF library to 8.2.1b04
* Add the write group lon info for upp 2d decomposition test.

* Multiple output (nested) grids from fv3atm (#1044)

Update fv3atm to support writing multiple history files for parent and nested domains.
Nested HAFS regression tests were updated to write out history files from all nested domains.

Co-authored-by: Bin Liu <bin.liu@noaa.gov>

* GPU-enabled version of Grell-Freitas convection in ccpp-physics (#1043)

* Enable Thompson microphysics with UFS-Aerosols and update GOCART (#1049)

This update allows the Thompson microphysics scheme to be used with prognostic aerosols (UFS-Aerosols) and updates GOCART to version 2.0.2.

* Add support for Stochastically Perturbed Parameterizations (SPP) in FV3 (#982)

* Update CICE; add support for restart test when running single RT (was PR 1058); update MOM6 (was PR 1060) (#1045)


* update cice
* (1) pointing MOM6 to testing branch (MOM6 main repo PR: https://github.com/mom-ocean/MOM6/pull/1556#issuecomment-1043374317)
    (2) add three new files in mom_cmake file
            core/MOM_porous_barriers.F90
            external/drifters/MOM_particles.F90
            external/drifters/MOM_particles_types.F90
    (3) explicitly set in MOM_input for all resolutions to reflect the default value changes in code to retain answer
        EPBL_MLD_BISECTION = True (default changed from T to F in this MOM6 PR)
        INTERNAL_WAVE_SPEED_BETTER_EST = False (default changed from F to T in this MOM6 PR)
        note BBL_USE_EOS now set to use the value of USE_EOS in this MOM6 PR,
        but we set BBL_USE_EOS explicit so it has no impact

* remove duplicated CHL input setting in MOM_input_template_050
* add support for restart tests while running single RT

* lateral boundary fix for regional runs (#1050)

This PR fixes a transient increase in noise caused by incorrect processing of winds along the lateral boundary on the final acoustic time step before fresh boundary information is ingested into the model.

* Update pointer to FV3

* Point to Christina's branches.

* Update RT log for Hera Intel.

* Adding RT log for Hera GNU tests.

* Updating RT log for Jet Intel tests.

* Latest regression tests on Jet.

* Adding regression logs.

* Updating regression test date stamp.

* Point to GSL repositories for FV3 and FV3/ccpp/physics

Co-authored-by: Dusan Jovic <48258889+DusanJovic-NOAA@users.noreply.github.com>
Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com>
Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>
Co-authored-by: BinLi-NOAA <bin.li@noaa.gov>
Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>
Co-authored-by: Dusan Jovic <dusan.jovic@noaa.gov>
Co-authored-by: Grant Firl <grant.firl@noaa.gov>
Co-authored-by: Dom Heinzeller <climbfuji@ymail.com>
Co-authored-by: mzhangw <mzhangw@gmail.com>
Co-authored-by: jiandewang <jiande.wang@noaa.gov>
Co-authored-by: WenMeng-NOAA <48260754+WenMeng-NOAA@users.noreply.github.com>
Co-authored-by: Samuel Trahan (NOAA contractor) <39415369+SamuelTrahanNOAA@users.noreply.github.com>
Co-authored-by: SMoorthi-emc <47667426+SMoorthi-emc@users.noreply.github.com>
Co-authored-by: ChunxiZhang-NOAA <49283036+ChunxiZhang-NOAA@users.noreply.github.com>
Co-authored-by: David Huber <david.huber@noaa.gov>
Co-authored-by: Bin Liu <Bin.Liu@noaa.gov>
Co-authored-by: Samuel Trahan <Samuel.Trahan@noaa.gov>
Co-authored-by: Jun Wang <37633869+junwang-noaa@users.noreply.github.com>
Co-authored-by: Gerhard Theurich <gerhard.j.theurich@noaa.gov >
Co-authored-by: Ali.Abdolali <37336972+aliabdolali@users.noreply.github.com>
Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Dom Heinzeller <heinzell@ucar.edu>
Co-authored-by: Jeff Whitaker <jswhit@fastmail.fm>
Co-authored-by: DomHeinzeller <58610420+DomHeinzeller@users.noreply.github.com>
Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>
Co-authored-by: JeffBeck-NOAA <55201531+JeffBeck-NOAA@users.noreply.github.com>
Co-authored-by: MatthewPyle-NOAA <48285220+MatthewPyle-NOAA@users.noreply.github.com>

* Remove GF Consistency Check & Add SDFs - V2 (#120)

* - Removes consistency check in GF, allows GF to run with different or no shallow scheme
- Add additional RAP based SDF

* - Add regression test logs

* jet intel test

* point to gsl/develop for FV3

Co-authored-by: Samuel Trahan <Samuel.Trahan@noaa.gov>

* p8b (with aerosols)  (#1071)


updates all p8 tests to the p8b settings, which includes GOCART

Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>

* Update to top of FV3 to get latest ccpp-physics codeowners (#123)

* Split CCPP finalize into physics_finalize and (framework) finalize, fix CCPP's metadata2html.py (#1061)

* add a cdeps test using GFSv16 input data (#1070)

* CMAKE for WW3  (#1089)


* Use WW3 CMake build

Co-authored-by: kgerheiser <kgerheiser@icloud.com>

* Add 'valid time' variable using ISO string format (#1073)

* Require cmake version 3.19
* Update FV3:
   - Add 'valid time' variable using ISO string format to netcdf history files.
   - Use double precision variable to set value of 'time' attribute in wrt comp import state
   - Update ccpp/physics (setting surface-related interstitial variables for SCM prescribed-surface-flux mode)
   - Update inline_post_stub.F90 subroutine interfaces to match inline_post.F90

* Add hi-res regional test on small domain (#1084)


* Add hi-res regional test on small domain

* Fix Lambert Conformal Conic mapping in the write component on the southern hemisphere (#1087)

* Subroutine `lambert` in the FV3 write component has been fixed to do the mapping on the southern hemisphere correctly.
* CI fix
  - Docker container library updated. CI input data updated. Add chm_petlist_bounds.
  - Change gocart output file name for debug test
  - Changes for opnReqTest to also scan rt_gnu.conf
  - Change CI tests to noaero

Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com>

* Rectify total number of Thompson MP species and add UFS-Aerosols to S2S and S2SW (#1118)

* Rectify total number of Thompson MP species
* Implement S2SWA/S2SA build app as the only one enabling prognostic aerosols in fully coupled model.
* Update output_fh documentation.

* Add a cdeps test and update mom6, fv3, and weekly tests (#1136)

Co-authored-by: jiandewang <jiande.wang@noaa.gov>
Co-authored-by: WenMeng-NOAA <48260754+WenMeng-NOAA@users.noreply.github.com>
Co-authored-by: DusanJovic-NOAA <dusan.jovic@noaa.gov>

* Updates to the MYNN-EDMF (#129)

* update submodule pointer for fv3atm

* Fault tolerance in rt.sh usage of rocoto.

* Update baseline date.

* Bug fixes to rocoto changes.

* Hera gnu tests passed.

* Sleep 60 seconds betwen calls to rocotorun to avoid seeing submitted jobs in an inconsistent state when the batch scheduler is overwhelmed.

* Hera intel tests passed.

* Jet intel tests passed.

* Corrections to Rocoto reliability changes

* Point to gsl/develop for FV3

Co-authored-by: samuel.trahan <Samuel.Trahan@noaa.gov>

* GWD, LSM and MYNN physics updates from RRFS_dev branch (#130)

* GWD, LSM and MYNN physics updates from RRFS_dev branch

* Temporary push to fork for transfer to Jet.

* Added regression test logs required for PR

* Reverting back to original rt.sh

* hera.gnu tests passed

* Jet intel tests passed.

* Hera intel tests passed

* Point to gsl/develop for FV3

Co-authored-by: samuel.trahan <Samuel.Trahan@noaa.gov>

* update submodule pointer for fv3atm

* Update fms to 2021.04 (#1135)

* Update fms to 2021.04
* Add RT for ocean IAU
* Update documentation

Co-authored-by: Phil Pegion <38869668+pjpegion@users.noreply.github.com>
Co-authored-by: Jun Wang <37633869+junwang-noaa@users.noreply.github.com>

* Point to top of gsl/develop for ccpp-physics

* land surface upgrades for p8 (#1095)

* reset the Noah-MP default options for those well tested

* P8c: Added capability for cdmbgwd(1) to scale GSL blocking drag (#1110)

* Update ESMF to version 8.3.0bs09 (#1146)

* Update ESMF to version 8.3.0b09
* Increase wall clock limit for tests/regional_3km to 15 min
* Update AutoRT to use tfe01 ecflow node on Jet
* Updated Cheyenne baseline epicufsrt path
* Updated Cheyenne epicufsrt hpc-stack build

Co-authored-by: JONG KIM <jong.kim@noaa.gov>
Co-authored-by: Brian Curtis <brian.curtis@noaa.gov>

* Update rrfs conus regression tests to compute fluxes.

* Remove nst_anl and a comment from the rrfs_conus13km_hrrr.nml.IN namelist template

* release/p8c: updated the TKE-EDMF PBL and saSAS cumulus scheme plus bug fix for the Thompson scheme (issue #881) (#1120)

* release/p8c: it is related to ccpp-physics PBL and cumulus updates plus a bug fix to the Thompson scheme

* Bring AQM model into UFS & Update MOM6 mixed FMS (#1137)

* Add AQM submodule to integrate NOAA's Air Quality Model embedding
the U.S. EPA Community Multiscale Air Quality Model (CMAQ).

* MOM6: An alternate fix to class(*) issues with FMS 2022-01

Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>
Co-authored-by: jiandewang <jiande.wang@noaa.gov>

* Radiation changes for prototype 8 (#1090)

* New feature in RRTMGP enabled ccpp-physics.
* Updated physics. New FV3 interstitials.
* Cleanup of RRTMGP regression tests.
* Replaced cld_mynn_ naming convention with cld_pbl_
* Some more cleanup of cloud-fraction...
* Pulled in Qingfu's changes to ccpp-physics
* New RT's for RRTMGP p8. Revert change in ccpp-physics
* Changed arguments to implied shape. in ccpp-physics
* Updated RRTMGP regression tests
* Changes to FV3/ccpp-physics.
* RRTMGP RT cleanup, again. Bug-fix in FV3/ccpp-physics
* Revert changes to names in FV3/ccpp-physics
* Cleanup p8 tests.
* More RT cleanup for p8.
* Add GP setting to non-coupled p8 tests.
* Added bounding to temperature at layer-interface used by RRTMGP.
* Reorder loop
* Combined gp sw and lw aerosol routines. Modest speedup (~4%). Add GP setting to test setup.
* Add GP settings to cpld_control_c96_noaero_p8 test.
* update test resources default_vars.sh

* HAFS moving-nesting related developments (#1104)

* Add the storm-following or specified moving-nesting capability for both the regional and global configuration.
* Expand the UFS/HAFS FV3ATM-HYCOM ocean coupling to support the static and moving nesting configuration.
* Expand the write grid component to support outputting the moving nest domain properly.
* Five HAFS related moving-nesting regression tests are added to test these newly added features. The regional_hafs RT was removed.

Co-authored-by: Daniel Rosen <daniel.rosen@noaa.gov>
Co-authored-by: Ufuk Turuncoglu <ufuk.turuncoglu@noaa.gov>
Co-authored-by: uturuncoglu <turuncu@ucar.edu>
Co-authored-by: Dusan Jovic <dusan.jovic@noaa.gov>
Co-authored-by: William Ramstrom <William.Ramstrom@noaa.gov>
Co-authored-by: A-Kyle <kyle.ahern@noaa.gov>

* fv3atm: Split GFS_typedefs.F90, rename CCPP_interstitial, import ccpp_t from ccpp_types; contains bug fix for WW3 debug (#1130)


Co-authored-by: Jessica Meixner <Jessica.Meixner@noaa.gov>

* Close MOM6 1deg at j=1; Update input-data for P8c (#909)


* update all tests using 1deg MOM6/CICE6
* pre-pend ufs to existing topo edits filename when
creating the new file
* update input-data directory

* Fix uninitialized min_rand variable in Thompson MP when using SPP (#1152)

* Updates to stochastic_physics and fv3 and CI

* Add RRFS-Smoke and regression tests.

* "update .gitmodules to point to the gsl/develop-smoke"

* "push tests/tests/rrfs_smoke_conus13km_hrrr_warm"

* Point to gsl/develop-smoke for ccpp-physics

* "update .gitmodules"

* "point to gsl/develop-smoke branch"

* "point to gsl/develop-smoke with active flag updates"

* "upload the hera.intel and hera.gnu regression test log files"

* Add the rrfs_smoke_conus13km_hrrr_warm regression test.

* Remove an extra read that went out of bounds

* hera.gnu tests passed

* hera.intel tests passed

* jet.intel tests passed

* Point to gsl/develop for FV3

* Point .gitmodules to NOAA-GSL

Co-authored-by: samuel.trahan <Samuel.Trahan@noaa.gov>

* A bug fix for wet scavenging of aerosols and a minor modification for moisture property calculation (#1176)

* point to one_scheme_to_one_file PR branch (#1175)

* merge ufs-community develop

* Point to gsl/merge-develop for FV3

* Changes missing from prior commit

* use community version of hafs_regional_atm

* merge rt.sh changes

* Point to gsl/merge-develop-to-community branch for FV3

* Revert gsl-specific changes to rt.sh

* Corrections to prior merge

* corrections to FV3_HRRR tests

* Add rrfs_smoke_conus13km_hrrr_warm test

* Add FV3_HRRR_smoke to rt builds

* Correct nst and domains_stack_size in rrfs_conus13km_hrrr.nml.IN

* Set SFCLAY_COMPUTE_FLUX=.true. for rrfs_smoke tests

* hera.gnu tests passed

* Hera BUILD_WALLTIME of 1 hour due to occasional extreme contention for intel licenses

* hera intel tests passed

* Jet tests passed

* Revert gsl-specific change to rt.sh

* Point FV3/.gitmodules to Sam's fork

* Correct issues found by reviewers in ccpp-physics and fv3atm

* Rename mynnpbl to mynnedmf in fv3atm and ccpp-physics

* FV3: Remove some empty _init and _finalize routines and update a comment in sgscloud_radpre

* Feature/lndp noahmp2 : add land perturbation scheme for Noah-MP fractional veg (#1143)

Updates to allow land perturbation scheme to be applied for Noah-MP. Currently, for Noah-MP model parameters can perturb only vegetation fraction.

Main code changes:
-added lsm_noahmp model option to lndp_type==2 land perturbation scheme (fv3-atm, stochastic_physics)
-updated comments
-cleaned up the namelists to make the different options clearer. Removed misleading lndp_each_step variable from gfs_physics_nml, and replaced it with "lndp_model_type" in the nam_sfcperts namelist for the different forecast types (cycling DA, short forecasts, perturbing only initial conditions) (fv3-atm, stochastic_physics, ufs-weather-model)
NOTE: see note below, re: specification of smc perturbation for RAP/HRRR/etc (lndp_model_type==2)
-revised code in stochastic_physics_wrapper to only allocate arrays that will be used when passed into lndp_apply_perts (fv3-atm)
-deleted unused albedo arrays in stochastic_physics_wrapper and lndp_apply_perts (fv3-atm, stochastic_physics)
-fixed bug causing precision errors when calculating the soil moisture ice content (often ~0.0; stochastic_physics)
-fixed bug in which lndp_apply_perts wasn't called for nscyc = 0. (fv3-atm)
-increased max_nvar_lndp at Jeff Ator's request (fv3-atm)
-added a new test for the Noah-MP and lndp==2 combination (ufs-weather-model)

* Add \file to smoke files

* Update CDEPS component, add hailcast code in FV3/atmos_cubed_sphere and reduce wall clock time used for the cpld_control_c192_p8 test on Jet (#1156)

Co-authored-by: Yunheng Wang

* Update CMEPS; Add gocart userguide (was #1182) (#1189)


* Update CMEPS submodule
* Add GOCART documentation

Co-authored-by: Li Pan <Li.Pan@noaa.gov>

* Combine PROD and REPRO build options into 'Release' build type (#1171)

Combined PROD and REPRO build modes into 'Release' build type. Now only 'Debug' and 'Release' build type are supported.
Both build types must produce bit-for-bit reproducible outputs using different number of threads, mpi tasks, different domain decomposition, have reproducible restarts etc.

* Unify inline post interfaces (#1181)

* replace pwatclm to pwat in the all diag tables under tests/parm/diag_table.

* adding bug fix from jili dong and removing U* averaging

* adding bug fix from jili dong and removing U* averaging

* Hera tests passed.

* Remove some commented-out code from cu_gf_driver in ccpp-physics.

* In FV3, turn rrtmg smoke band 10 into a model namelist variable

* Switch ccpp-physics smoke arguments to implied shape

* Remove some stops and gotos from ccpp-physics rrfs-smoke

* In FV3, remove some hard-coded constants and rename some meta entries

* In FV3, rename some standard_names, change a unit, and remove a "use physcons"

* Updated settings for NoahMP, Thompson and RRTMGP in RTs (#1196)

* add NoahMP opt_trs to namelist
* Updated namelist settings related to NoahMP scheme, Thompson scheme and RRTMGP (issue #1173 and #1185)
* Updated some namelist settings and add/remove some RTs related to RRTMGP

* Compute petlist bounds for each subcomponent from number of tasks. Update CICE (#1200)

* Instead of specifying petlist bounds (lower and upper bound), compute them using the number of tasks for each component. The function that computes petlist bounds assumes that the mediator always runs on the same tasks as ATM component, and that the Chemistry component also always runs on the same tasks as ATM.
* Update CICE submodule (@DeniseWorthen)

Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com>

* NSSL ccpp-physics bugfixes and new ccpp-framework debugging feature (#1202)

* Merge develop

* hera.gnu tests passed

* hera.intel tests passed

* jet intel tests passed

* hera.gnu tests passed.

* fv3atm: Only add cnv_3d_ud_mf to the restart files when needed

* More warnings not to change the conus13km tests

* adjust BL_DATE in rt.sh to 20220516

* WCOSS Cray tests passed

* fv3atm: pass some chemistry varibles to mynn_bl_driver correctly

* [AutoRT] hera.gnu Job Completed.

on-behalf-of @ufs-community <brian.curtis@noaa.gov>

* [AutoRT] hera.intel Job Completed.

on-behalf-of @ufs-community <brian.curtis@noaa.gov>

* [AutoRT] cheyenne.intel Job Completed.

on-behalf-of @ufs-community <brian.curtis@noaa.gov>

* WCOSS Dell P3 tests passed

* Correct argument passing within mynn edmf

* [AutoRT] gaea.intel Job Completed.

on-behalf-of @ufs-community <brian.curtis@noaa.gov>

* [AutoRT] cheyenne.gnu Job Completed.

on-behalf-of @ufs-community <brian.curtis@noaa.gov>

* Orion RT Passed

* [AutoRT] jet.intel Job Completed.

on-behalf-of @ufs-community <brian.curtis@noaa.gov>

* Point to NOAA-EMC develop for FV3

* merge gsl institutional fork (#1195)

Merge in GSL institutional fork.

* update FV3 to top of gsl/merge-community-to-develop

* FV3: point to sam's repo

* Point to GSL baseline areas

* FV3/ccpp/physics: Change to GSL CODEOWNERS

* hera.gnu tests passed

* hera.intel passed

* jet.intel tests passed

* Set regression test path to GSL areas on Hera

* FV3: Returned the comments on soil resistance back.

* point to gsl/develop for fv3atm

* Merge May 16 ufs-community develop to gsl/develop (#138)

* Add ufs driver. Remove NEMS submodule. Update CMEPS. Update CDEPS. Feature/ort addition.  (#931)

* Add UFS driver, a copy of two fortran files from the NEMS repository. Remove NEMS submodule.

* Update CMEPS for current escomp/master. Implement the ability to write mediator history files for ATM on tiles rather than mesh

* Update CDEPS component. The updated CDEPS includes code updates for GFS v16 data mode.

* Operation requirement test (ORT) for:
  - cpld_control_p7 thread and decomposition tests
  - cpld_bmark_p7 thread and decomposition tests
  - Due to the wave component in these tests, restart and debug do not work

Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com>
Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>
Co-authored-by: BinLi-NOAA <bin.li@noaa.gov>

* fv3atm cap cleanup (#954)

FV3 nuopc cap code cleanup. Remove used code/variables. Fix minor inconsistencies.

* CCPP cloud cover change for Thompson MP associated with Xu Randall, several bug fixes for regression testing scripts (#962)

This PR contains code changes 1) to remove old comments in GFS_typedefs.F90, increase cloud cover for Thompson MP by passing convective cloud water mixing ratio to progcld6. 2) Updates for auto fixes

Co-authored-by: Dusan Jovic <dusan.jovic@noaa.gov>

* ccpp-physics #806, #807, #813, and ort modifications (#970)

- CCPP Physics changes: see Wrapper for ccpp-physics #806, #807, #813, NOAA-EMC/fv3atm#447.
- UFS weather model changes for ORT updates following changes in Fix Auto-RT on Gaea, Jet and Orion #965

* Wrapper for ccpp-physics #808 and 816 (roughness length over ice and NoahMP tsurf bugfix) (#971)

* update submodule pointer for regression testing for ccpp-physics #808,816: roughness length over ice and NoahMP tsurf bugfix
* Reduce time step from 600s to 450s for tests control_csawmg and control_csawmg_debug

Co-authored-by: Dom Heinzeller <climbfuji@ymail.com>
Co-authored-by: climbfuji <dom.heinzeller@icloud.com>

* Use 'model set run clock' routine in FV3 NUOPC cap. (#967)

* Update FV3 cap by adding 'model set run clock' specialization routine. Because FV3 cap does not modify 'Earth clock' anymore in case of a restarted run, that adjustment is now done in the UFS driver.
* Fix model_configure file for two tests to define `fhrot` parameter.

* Revert iovr=4 (exponential cloud overlap method) in RRTMG (#963)

Revert/cleanup iovr=4 in RRTMG per developer's request. The current exponential cloud overlapping method used in RRTMG represents the method used in the pre-2018 operational HWRF model.

* Add fhzero ort test (#977)

* Fhzero ORT applies to the two tests `control` and `control_p7`
* Test can be run by invoking, for example, `./opnReqTest -n control -c fhz`

* Add a new CDEPS test to use GSFv16 data mode in CDEPS ,update rt_weekly.conf and update MOM6 to main branch (20211220) (#976)

* add a new cdeps test using GFS data mode
* update rt_weekly.conf to include cpld_bmark_p7_35d tests
* update MOM6

Co-authored-by: jiandewang <jiande.wang@noaa.gov>

* Bug fixes for WAM physics and regression tests, remove duplicate modules from ccpp-physics CMakeLists.txt (#947)

*adds a new regression test control_wam_debug and fixes problems with the existing control_wam test (e.g. IAER was not set as intended).

* Update global inline post for outputting foundation temperature (#968)

Update upp submodule in fv3atm with latest UPP version for outputting foundation temperature and fixing land-sea mask issue in global inline post.

* Regression test bug fixes for running on Jet or with Rocoto (#981)

* Bug fixes for running on jet or with rocoto
1. Correctly detect Jet when deciding build job runtimes
2. Do not delete out and err files in compile & test jobs
3. Correct dependencies for WW3 in Rocoto
4. Do not hard-code ACCNR to h-nems
This PR also contains fv3/ccpp physics (CCPP PR#812) updates that implements a vertically-integrated formula to properly compute tracer column mass and ensure mass conservation for all tracers except TKE. See issue NCAR/ccpp-physics#811 for details

* MYNN sfclay (RAP suite) restart reproducibility (#984)

*This PR turns on the rap_restart tests. Bit-for-bit reproducibility is achieved with the bug fixes for MYNN sfclay in the associated fv3atm and ccpp-physics PRs.

* fixes on initializing snow over fractional ice and GFS_surface_composite and z0ice value change (#996)

* Radar-derived microphysics temperature tendencies similar to operational HRRR (#986)

This PR implements a feature of the operational HRRR, radar-derived microphysics temperature tendencies applied in the first N minutes of the forecast to improve clouds in the first few hours. Two regression tests are added.

* Turn on CICE albedo use by ATM for coupled model; update Prototype test suite to initial P8 configuration;  (#944)

* add updates for suites, tests for initial P8 configuration
* add IOPT_SFC=3 as default; set IOPT_SFC=1 in p7 tests
* change min_seaice to 1e-6; set to 1e-11 in p7 tests
* set dddmp=0.1 in both P7 and P8. This matches the final
P7 configuration. It will change P7 baselines.
* set cdmbwd in both P7 and P8 to the final P7 configuration.
(1.0,2.2,1.0,1.0) for all resolutions. It will change P7 baselines.
* add dusan's ecflow fix for wcoss2
* update FV3 with Moorthis' snod fix
* update rt_weekly to use p8 tests
* change integration length to 5 days

* Thompson MP cloud tuning (#1005)

* improves cloud fraction when using Thompson MP. See NCAR/ccpp-physics#809 for more details.

* Add a new HAFS regression test with Thompson MP and GFDL surface layer schemes and Corrected S4 template regression testing job card (#995)

* Added a new regression test for HAFS. 
* Fixed the S4 RT template job card.

Co-authored-by: David Huber <david.huber@noaa.gov>
Co-authored-by: Bin Liu <Bin.Liu@noaa.gov>

* CCPP: Update consistency checks and effective radii calculation for Thompson MP (#1000)

This PR contains changes in ccpp-physics (Update consistency checks and effective radii calculation for Thompson MP) and adds two new regression tests for both Intel and GNU that use the new prognostic cloud scheme progcld_thompson with GFS v16 and with RAP. The new cloud scheme is activated by setting icloud == 3 in input.nml.

* Rremove nems_datm related code, update CDEPS, and remove run directories that are no longer needed by other tests (#1008)

* Add fhzero ort to coupled tests (#1009)

* Add fhz ort to cpld tests
* Set ACCNR in rt.sh, not detect_machine.sh. Add some comments explaining where the variable should not be set or used.

Co-authored-by: Samuel Trahan <Samuel.Trahan@noaa.gov>

* Unified write_netcdf, add single file 'cubed_sphere_grid' output (#1011)

* Unify write_netcdf and write_netcdf_parallel modules.
* Add support for writing 'cubed_sphere_grid' history files in a single netcdf file.

* Change standard name and unit of CCPP error flag variable in CCPP framework and physics (#1013)

* updates the submodule pointers for fv3atm, ccpp-framework and ccpp-physics for:  Change standard name and unit of CCPP error flag variable in CCPP framework and physics.

* ESMF managed threading (#1018)

Add code updates to enable the ESMF managed threading in UFS.


Co-authored-by: Gerhard Theurich <gerhard.j.theurich@noaa.gov >

* sync-ww3 (#1026)

* sync with ww3 develop branch

* Revisions to repair iovr=5 cloud overlap option (#1025)

* This PR addresses part 2 of issue #748 to activate the exponential-random cloud overlap method (iovr=5) in RRTMG. RT tests were added for  iovr=4 and iovr=5 options
* change iovr to use the environment variable in control.nml.IN; modify all tests that use control.nml.IN to set env. var. iovr=3 (except for new tests); remove restart files from new tests

* 2D decomposition on write grid, bug fix in FV3GFS_IO.F90, and quad precision in stochastic physics (#1028)

- The 2D decomposition is enabled on write grid component.
- FV3atm PR NOAA-EMC/fv3atm#468 from @MicroTed 
- stochastic physics PR noaa-psd/stochastic_physics#53 from Dom.

Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Dom Heinzeller <heinzell@ucar.edu>

* Update CMEPS for wave coupling (#1034)


* add run phases to allow waves to run in the slow loop
* re-order hafs wave test
* update aliases in fd_nems.yaml
* update CMEPS to prevent advertise conflicts when s2sw is
still using connectors
* add correct masking variables to hafs nems.configure files

* 4DIAU bug fix when iau_filter_increments=T (#990)

* Add lsm_cold_start variable for RUC LSM SCM support and gwdps bugfix (combined) (#1037)

Co-authored-by: @SMoorthi-emc

* Update esmf 821b04 (#1041)

* Update the version of ESMF library to 8.2.1b04
* Add the write group lon info for upp 2d decomposition test.

* Multiple output (nested) grids from fv3atm (#1044)

Update fv3atm to support writing multiple history files for parent and nested domains.
Nested HAFS regression tests were updated to write out history files from all nested domains.

Co-authored-by: Bin Liu <bin.liu@noaa.gov>

* GPU-enabled version of Grell-Freitas convection in ccpp-physics (#1043)

* Enable Thompson microphysics with UFS-Aerosols and update GOCART (#1049)

This update allows the Thompson microphysics scheme to be used with prognostic aerosols (UFS-Aerosols) and updates GOCART to version 2.0.2.

* Add support for Stochastically Perturbed Parameterizations (SPP) in FV3 (#982)

* Update CICE; add support for restart test when running single RT (was PR 1058); update MOM6 (was PR 1060) (#1045)


* update cice
* (1) pointing MOM6 to testing branch (MOM6 main repo PR: https://github.com/mom-ocean/MOM6/pull/1556#issuecomment-1043374317)
    (2) add three new files in mom_cmake file
            core/MOM_porous_barriers.F90
            external/drifters/MOM_particles.F90
            external/drifters/MOM_particles_types.F90
    (3) explicitly set in MOM_input for all resolutions to reflect the default value changes in code to retain answer
        EPBL_MLD_BISECTION = True (default changed from T to F in this MOM6 PR)
        INTERNAL_WAVE_SPEED_BETTER_EST = False (default changed from F to T in this MOM6 PR)
        note BBL_USE_EOS now set to use the value of USE_EOS in this MOM6 PR,
        but we set BBL_USE_EOS explicit so it has no impact

* remove duplicated CHL input setting in MOM_input_template_050
* add support for restart tests while running single RT

* lateral boundary fix for regional runs (#1050)

This PR fixes a transient increase in noise caused by incorrect processing of winds along the lateral boundary on the final acoustic time step before fresh boundary information is ingested into the model.

* Feature: NSSL Microphysics support (#1032)

* add new RT that uses NSSL MP; add new NSSL vars to default_vars.sh and rap.nml.IN; add new field table for using NSSL MP with hail on

* p8b (with aerosols)  (#1071)


updates all p8 tests to the p8b settings, which includes GOCART

Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>

* Split CCPP finalize into physics_finalize and (framework) finalize, fix CCPP's metadata2html.py (#1061)

* add a cdeps test using GFSv16 input data (#1070)

* CMAKE for WW3  (#1089)


* Use WW3 CMake build

Co-authored-by: kgerheiser <kgerheiser@icloud.com>

* Add 'valid time' variable using ISO string format (#1073)

* Require cmake version 3.19
* Update FV3:
   - Add 'valid time' variable using ISO string format to netcdf history files.
   - Use double precision variable to set value of 'time' attribute in wrt comp import state
   - Update ccpp/physics (setting surface-related interstitial variables for SCM prescribed-surface-flux mode)
   - Update inline_post_stub.F90 subroutine interfaces to match inline_post.F90

* Add hi-res regional test on small domain (#1084)


* Add hi-res regional test on small domain

* Fix Lambert Conformal Conic mapping in the write component on the southern hemisphere (#1087)

* Subroutine `lambert` in the FV3 write component has been fixed to do the mapping on the southern hemisphere correctly.
* CI fix
  - Docker container library updated. CI input data updated. Add chm_petlist_bounds.
  - Change gocart output file name for debug test
  - Changes for opnReqTest to also scan rt_gnu.conf
  - Change CI tests to noaero

Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com>

* Rectify total number of Thompson MP species and add UFS-Aerosols to S2S and S2SW (#1118)

* Rectify total number of Thompson MP species
* Implement S2SWA/S2SA build app as the only one enabling prognostic aerosols in fully coupled model.
* Update output_fh documentation.

* Add a cdeps test and update mom6, fv3, and weekly tests (#1136)

Co-authored-by: jiandewang <jiande.wang@noaa.gov>
Co-authored-by: WenMeng-NOAA <48260754+WenMeng-NOAA@users.noreply.github.com>
Co-authored-by: DusanJovic-NOAA <dusan.jovic@noaa.gov>

* update submodule pointer for fv3atm

* Update fms to 2021.04 (#1135)

* Update fms to 2021.04
* Add RT for ocean IAU
* Update documentation

Co-authored-by: Phil Pegion <38869668+pjpegion@users.noreply.github.com>
Co-authored-by: Jun Wang <37633869+junwang-noaa@users.noreply.github.com>

* land surface upgrades for p8 (#1095)

* reset the Noah-MP default options for those well tested

* P8c: Added capability for cdmbgwd(1) to scale GSL blocking drag (#1110)

* Update ESMF to version 8.3.0bs09 (#1146)

* Update ESMF to version 8.3.0b09
* Increase wall clock limit for tests/regional_3km to 15 min
* Update AutoRT to use tfe01 ecflow node on Jet
* Updated Cheyenne baseline epicufsrt path
* Updated Cheyenne epicufsrt hpc-stack build

Co-authored-by: JONG KIM <jong.kim@noaa.gov>
Co-authored-by: Brian Curtis <brian.curtis@noaa.gov>

* Update rrfs conus regression tests to compute fluxes.

* Remove nst_anl and a comment from the rrfs_conus13km_hrrr.nml.IN namelist template

* release/p8c: updated the TKE-EDMF PBL and saSAS cumulus scheme plus bug fix for the Thompson scheme (issue #881) (#1120)

* release/p8c: it is related to ccpp-physics PBL and cumulus updates plus a bug fix to the Thompson scheme

* Bring AQM model into UFS & Update MOM6 mixed FMS (#1137)

* Add AQM submodule to integrate NOAA's Air Quality Model embedding
the U.S. EPA Community Multiscale Air Quality Model (CMAQ).

* MOM6: An alternate fix to class(*) issues with FMS 2022-01

Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>
Co-authored-by: jiandewang <jiande.wang@noaa.gov>

* Radiation changes for prototype 8 (#1090)

* New feature in RRTMGP enabled ccpp-physics.
* Updated physics. New FV3 interstitials.
* Cleanup of RRTMGP regression tests.
* Replaced cld_mynn_ naming convention with cld_pbl_
* Some more cleanup of cloud-fraction...
* Pulled in Qingfu's changes to ccpp-physics
* New RT's for RRTMGP p8. Revert change in ccpp-physics
* Changed arguments to implied shape. in ccpp-physics
* Updated RRTMGP regression tests
* Changes to FV3/ccpp-physics.
* RRTMGP RT cleanup, again. Bug-fix in FV3/ccpp-physics
* Revert changes to names in FV3/ccpp-physics
* Cleanup p8 tests.
* More RT cleanup for p8.
* Add GP setting to non-coupled p8 tests.
* Added bounding to temperature at layer-interface used by RRTMGP.
* Reorder loop
* Combined gp sw and lw aerosol routines. Modest speedup (~4%). Add GP setting to test setup.
* Add GP settings to cpld_control_c96_noaero_p8 test.
* update test resources default_vars.sh

* HAFS moving-nesting related developments (#1104)

* Add the storm-following or specified moving-nesting capability for both the regional and global configuration.
* Expand the UFS/HAFS FV3ATM-HYCOM ocean coupling to support the static and moving nesting configuration.
* Expand the write grid component to support outputting the moving nest domain properly.
* Five HAFS related moving-nesting regression tests are added to test these newly added features. The regional_hafs RT was removed.

Co-authored-by: Daniel Rosen <daniel.rosen@noaa.gov>
Co-authored-by: Ufuk Turuncoglu <ufuk.turuncoglu@noaa.gov>
Co-authored-by: uturuncoglu <turuncu@ucar.edu>
Co-authored-by: Dusan Jovic <dusan.jovic@noaa.gov>
Co-authored-by: William Ramstrom <William.Ramstrom@noaa.gov>
Co-authored-by: A-Kyle <kyle.ahern@noaa.gov>

* fv3atm: Split GFS_typedefs.F90, rename CCPP_interstitial, import ccpp_t from ccpp_types; contains bug fix for WW3 debug (#1130)


Co-authored-by: Jessica Meixner <Jessica.Meixner@noaa.gov>

* Close MOM6 1deg at j=1; Update input-data for P8c (#909)


* update all tests using 1deg MOM6/CICE6
* pre-pend ufs to existing topo edits filename when
creating the new file
* update input-data directory

* Fix uninitialized min_rand variable in Thompson MP when using SPP (#1152)

* Updates to stochastic_physics and fv3 and CI

* A bug fix for wet scavenging of aerosols and a minor modification for moisture property calculation (#1176)

* point to one_scheme_to_one_file PR branch (#1175)

* merge ufs-community develop

* Point to gsl/merge-develop for FV3

* Changes missing from prior commit

* use community version of hafs_regional_atm

* merge rt.sh changes

* Point to gsl/merge-develop-to-community branch for FV3

* Revert gsl-specific changes to rt.sh

* Corrections to prior merge

* corrections to FV3_HRRR tests

* Add rrfs_smoke_conus13km_hrrr_warm test

* Add FV3_HRRR_smoke to rt builds

* Correct nst and domains_stack_size in rrfs_conus13km_hrrr.nml.IN

* Set SFCLAY_COMPUTE_FLUX=.true. for rrfs_smoke tests

* hera.gnu tests passed

* Hera BUILD_WALLTIME of 1 hour due to occasional extreme contention for intel licenses

* hera intel tests passed

* Jet tests passed

* Revert gsl-specific change to rt.sh

* Point FV3/.gitmodules to Sam's fork

* Correct issues found by reviewers in ccpp-physics and fv3atm

* Rename mynnpbl to mynnedmf in fv3atm and ccpp-physics

* FV3: Remove some empty _init and _finalize routines and update a comment in sgscloud_radpre

* Feature/lndp noahmp2 : add land perturbation scheme for Noah-MP fractional veg (#1143)

Updates to allow land perturbation scheme to be applied for Noah-MP. Currently, for Noah-MP model parameters can perturb only vegetation fraction.

Main code changes:
-added lsm_noahmp model option to lndp_type==2 land perturbation scheme (fv3-atm, stochastic_physics)
-updated comments
-cleaned up the namelists to make the different options clearer. Removed misleading lndp_each_step variable from gfs_physics_nml, and replaced it with "lndp_model_type" in the nam_sfcperts namelist for the different forecast types (cycling DA, short forecasts, perturbing only initial conditions) (fv3-atm, stochastic_physics, ufs-weather-model)
NOTE: see note below, re: specification of smc perturbation for RAP/HRRR/etc (lndp_model_type==2)
-revised code in stochastic_physics_wrapper to only allocate arrays that will be used when passed into lndp_apply_perts (fv3-atm)
-deleted unused albedo arrays in stochastic_physics_wrapper and lndp_apply_perts (fv3-atm, stochastic_physics)
-fixed bug causing precision errors when calculating the soil moisture ice content (often ~0.0; stochastic_physics)
-fixed bug in which lndp_apply_perts wasn't called for nscyc = 0. (fv3-atm)
-increased max_nvar_lndp at Jeff Ator's request (fv3-atm)
-added a new test for the Noah-MP and lndp==2 combination (ufs-weather-model)

* Add \file to smoke files

* Update CDEPS component, add hailcast code in FV3/atmos_cubed_sphere and reduce wall clock time used for the cpld_control_c192_p8 test on Jet (#1156)

Co-authored-by: Yunheng Wang

* Update CMEPS; Add gocart userguide (was #1182) (#1189)


* Update CMEPS submodule
* Add GOCART documentation

Co-authored-by: Li Pan <Li.Pan@noaa.gov>

* Combine PROD and REPRO build options into 'Release' build type (#1171)

Combined PROD and REPRO build modes into 'Release' build type. Now only 'Debug' and 'Release' build type are supported.
Both build types must produce bit-for-bit reproducible outputs using different number of threads, mpi tasks, different domain decomposition, have reproducible restarts etc.

* Unify inline post interfaces (#1181)

* replace pwatclm to pwat in the all diag tables under tests/parm/diag_table.

* adding bug fix from jili dong and removing U* averaging

* adding bug fix from jili dong and removing U* averaging

* Hera tests passed.

* Remove some commented-out code from cu_gf_driver in ccpp-physics.

* In FV3, turn rrtmg smoke band 10 into a model namelist variable

* Switch ccpp-physics smoke arguments to implied shape

* Remove some stops and gotos from ccpp-physics rrfs-smoke

* In FV3, remove some hard-coded constants and rename some meta entries

* In FV3, rename some standard_names, change a unit, and remove a "use physcons"

* Updated settings for NoahMP, Thompson and RRTMGP in RTs (#1196)

* add NoahMP opt_trs to namelist
* Updated namelist settings related to NoahMP scheme, Thompson scheme and RRTMGP (issue #1173 and #1185)
* Updated some namelist settings and add/remove some RTs related to RRTMGP

* Compute petlist bounds for each subcomponent from number of tasks. Update CICE (#1200)

* Instead of specifying petlist bounds (lower and upper bound), compute them using the number of tasks for each component. The function that computes petlist bounds assumes that the mediator always runs on the same tasks as ATM component, and that the Chemistry component also always runs on the same tasks as ATM.
* Update CICE submodule (@DeniseWorthen)

Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com>

* NSSL ccpp-physics bugfixes and new ccpp-framework debugging feature (#1202)

* Merge develop

* hera.gnu tests passed

* hera.intel tests passed

* jet intel tests passed

* hera.gnu tests passed.

* fv3atm: Only add cnv_3d_ud_mf to the restart files when needed

* More warnings not to change the conus13km tests

* adjust BL_DATE in rt.sh to 20220516

* WCOSS Cray tests passed

* fv3atm: pass some chemistry varibles to mynn_bl_driver correctly

* [AutoRT] hera.gnu Job Completed.

on-behalf-of @ufs-community <brian.curtis@noaa.gov>

* [AutoRT] hera.intel Job Completed.

on-behalf-of @ufs-community <brian.curtis@noaa.gov>

* [AutoRT] cheyenne.intel Job Completed.

on-behalf-of @ufs-community <brian.curtis@noaa.gov>

* WCOSS Dell P3 tests passed

* Correct argument passing within mynn edmf

* [AutoRT] gaea.intel Job Completed.

on-behalf-of @ufs-community <brian.curtis@noaa.gov>

* [AutoRT] cheyenne.gnu Job Completed.

on-behalf-of @ufs-community <brian.curtis@noaa.gov>

* Orion RT Passed

* [AutoRT] jet.intel Job Completed.

on-behalf-of @ufs-community <brian.curtis@noaa.gov>

* Point to NOAA-EMC develop for FV3

* merge gsl institutional fork (#1195)

Merge in GSL institutional fork.

* update FV3 to top of gsl/merge-community-to-develop

* FV3: point to sam's repo

* Point to GSL baseline areas

* FV3/ccpp/physics: Change to GSL CODEOWNERS

* hera.gnu tests passed

* hera.intel passed

* jet.intel tests passed

* Set regression test path to GSL areas on Hera

* FV3: Returned the comments on soil resistance back.

Co-authored-by: Dusan Jovic <48258889+DusanJovic-NOAA@users.noreply.github.com>
Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com>
Co-authored-by: Denise Worthen <denise.worthen@noaa.gov>
Co-authored-by: BinLi-NOAA <bin.li@noaa.gov>
Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>
Co-authored-by: Dusan Jovic <dusan.jovic@noaa.gov>
Co-authored-by: Grant Firl <grant.firl@noaa.gov>
Co-authored-by: Dom Heinzeller <climbfuji@ymail.com>
Co-authored-by: mzhangw <mzhangw@gmail.com>
Co-authored-by: jiandewang <jiande.wang@noaa.gov>
Co-authored-by: WenMeng-NOAA <48260754+WenMeng-NOAA@users.noreply.github.com>
Co-authored-by: SMoorthi-emc <47667426+SMoorthi-emc@users.noreply.github.com>
Co-authored-by: ChunxiZhang-NOAA <49283036+ChunxiZhang-NOAA@users.noreply.github.com>
Co-authored-by: David Huber <david.huber@noaa.gov>
Co-authored-by: Bin Liu <Bin.Liu@noaa.gov>
Co-authored-by: Jun Wang <37633869+junwang-noaa@users.noreply.github.com>
Co-authored-by: Gerhard Theurich <gerhard.j.theurich@noaa.gov >
Co-authored-by: Ali.Abdolali <37336972+aliabdolali@users.noreply.github.com>
Co-authored-by: Ted Mansell <ted.mansell@noaa.gov>
Co-authored-by: Dom Heinzeller <heinzell@ucar.edu>
Co-authored-by: Jeff Whitaker <jswhit@fastmail.fm>
Co-authored-by: DomHeinzeller <58610420+DomHeinzeller@users.noreply.github.com>
Co-authored-by: Raffaele Montuoro <raffaele.montuoro@noaa.gov>
Co-authored-by: JeffBeck-NOAA <55201531+JeffBeck-NOAA@users.noreply.github.com>
Co-authored-by: MatthewPyle-NOAA <48285220+MatthewPyle-NOAA@users.noreply.github.com>
Co-authored-by: Jessica Meixner <jessica.meixner@noaa.gov>
Co-authored-by: kgerheiser <kgerheiser@icloud.com>
Co-authored-by: RatkoVasic-NOAA <37597874+RatkoVasic-NOAA@users.noreply.github.com>
Co-authored-by: joeolson42 <Joseph.B.Olson@noaa.gov>
Co-authored-by: Phil Pegion <38869668+pjpegion@users.noreply.github.com>
Co-authored-by: HelinWei-NOAA <48133472+HelinWei-NOAA@users.noreply.github.com>
Co-authored-by: mdtoyNOAA <73618848+mdtoyNOAA@users.noreply.github.com>
Co-authored-by: JONG KIM <jong.kim@noaa.gov>
Co-authored-by: Brian Curtis <brian.curtis@noaa.gov>
Co-authored-by: Brian Curtis <64433609+BrianCurtis-NOAA@users.noreply.github.com>
Co-authored-by: dustinswales <dustin.swales@noaa.gov>
Co-authored-by: Daniel Rosen <daniel.rosen@noaa.gov>
Co-authored-by: Ufuk Turuncoglu <ufuk.turuncoglu@noaa.gov>
Co-authored-by: uturuncoglu <turuncu@ucar.edu>
Co-authored-by: William Ramstrom <William.Ramstrom@noaa.gov>
Co-authored-by: A-Kyle <kyle.ahern@noaa.gov>
Co-authored-by: ClaraDraper-NOAA <33430543+ClaraDraper-NOAA@users.noreply.github.com>
Co-authored-by: Li Pan <Li.Pan@noaa.gov>

* Correct .gitmodules

* Gsl/may16 followup pr (#139)

* new tests

* point to gsl/may16-followup-pr branch of fv3

* Change suite_FV3_GFS_v15_thompson_mynn to not use mynn surface layer scheme (from T…
vlakshmanan-scala pushed a commit to scala-computing/WRF that referenced this pull request Apr 4, 2024
…-MP and icloud3 (cloud fraction) scheme (wrf-model#1626)

TYPE: enhancement

KEYWORDS: microphysics, clouds, cloud fraction, Thompson microphysics

SOURCE:  Greg Thompson (UCAR/Joint Center for Satellite Data Assimilation)

DESCRIPTION OF CHANGES:
The purpose of these changes is to produce better cloud coverage and radiation amounts per tuning done with 
GFS global model and comparisons to observations.

Problem:
The Thompson-MP code (from ccpp-physics) has been used within UFS/GFS/FV-3 global model for multiple 
day simulations.  Anning Cheng and Ruiyu Sun (NCEP/EMC) have performed most of the simulations and 
analysis and iterated with me to make adjustments to the Thompson-MP.  At first, EMC found that high 
clouds were not present enough, particularly in the tropics.  Once switching to a larger upper limit of cloud 
ice before it becomes snow, the cloud amounts improved but some clouds and radiation balances were still 
not matching results from FV3's GFDL microphysics (which had already been tuned to observations).  
Therefore an effort to account for subgrid-scale clouds using the cloud fraction scheme (icloud=3) was 
adopted to GFS and tested with various parameter settings.

Solution:
The biggest change is in how cloud ice converts to snow at a threshold size of 300 microns (up from 200 
microns) and for rimed snow to convert to graupel (changed from 250 to 350 microns).  A change to allowed 
max size of ice means that the fall velocity constant for ice was changed to keep it aligned with snow at the 
same cut-over size (of 300 microns).  In addition, the cloud fraction scheme (icloud=3) can further improve 
the clouds and radiation together with Thompson-MP due to the overall under-prediction of clouds.  The 
icloud3 option was also making far too many clouds when compared to observations so its tuning knobs 
were adjusted until attaining the following overall improvements compared to cloud and radiation global 
climatologies that EMC uses:  cloud amounts of low, middle, high, and total cloud coverage, longwave 
radiation outgoing at top-of-atmosphere, and shortwave radiation reaching the ground.

ISSUE:  
There are corresponding issues or pull requests in the ccpp-physics repo, 
[778](NCAR/ccpp-physics#778)
[781](NCAR/ccpp-physics#781)
[809](NCAR/ccpp-physics#809)

LIST OF MODIFIED FILES: 
M  module_mp_thompson.F
M  module_radiation_driver.F
M  module_ra_rrtmg_lw.F
M  module_ra_rrtmg_lwf.F
M  module_ra_rrtmg_sw.F
M  module_ra_rrtmg_swf.F

TESTS CONDUCTED: 
1. A series of tests in GFS including 5, 7, 16, and 30-day long simulations compared to known cloud and 
radiation climatologies. The plots attached here in this comment were created by Anning Cheng. The numbers 
at the top of each panel represent global 3-day (days 3, 4, 5) average.

The first plot is the high/mid/low/total cloud amount.  The tunings in this PR reduced the high cloud amount 
from over 55% down to the low 40s, which matches observations of global cloud coverage pretty well.  And, 
another comparison is without the cloud fraction scheme, the mid-level clouds (and low/high) are simply less 
than observations, which is something well published in IPCC reports of global model clouds, especially the 
Southern Ocean and east sides of ocean basins (low stratus).
![gfnl_cf_20190611](https://user-images.githubusercontent.com/35609171/149175845-585914ce-84d4-4d72-a7e2-e96a5cb9d228.png)

The next plot's panel (a) shows the outgoing longwave radiation.  The target value global average is about 
240 W/m2.  Without the cloud fraction scheme and changes to ```D0s``` and ```D0g```, the result would be 
too much outgoing longwave since it will not contain enough clouds at lower temperatures.  Also, the older 
version of the cloud fraction scheme with its excessive amount of high clouds would make the outgoing 
radiation closer to 225 W/m2.  So the tuning of the scheme brings the results closer to observations.

![gfnl_prcp_20190611](https://user-images.githubusercontent.com/35609171/149176400-41f8b655-3d58-4cc4-adb0-6f6037beed52.png)

Lastly, the final plot shows the downward longwave reaching the ground (panel c) and downward shortwave 
reaching the ground (panel d). 

![gfnl_sfc_SWLW_20190611](https://user-images.githubusercontent.com/35609171/149177060-964ce8bd-02dc-4359-ad5d-37502b2d0f70.png)

2. Jenkins tests are passing.

RELEASE NOTE: Update of the Thompson microphysics scheme and cloud fraction scheme (icloud=3) to match the observations better. The modifications include updates to RRTMG LW and SW, and RRTMG fast LW and SW.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants