Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug fixes in MYNN surface layer scheme. Also, added lat/log to RUC LSM for debugging. #112

Merged
merged 7 commits into from
Nov 8, 2021

Conversation

tanyasmirnova
Copy link
Collaborator

  1. Fixed bugs in MYNN surface layer scheme in land/ice/water parts of the code
    to avoid using in computations "huge" numbers of qsfc_lnd, qsfc_ice and qsfc_wat.
  2. Fixed bugs in MYNN surface layer scheme for initialization and consistent
    use of surface QV from RUC LSM over land and ice.
  3. Passed into RUC LSM driver lat/lon for debug prints for a test point with known lat/lon.

…e code

to avoid using in computations "huge" numbers of qsfc_lnd, qsfc_ice and qsfc_wat.
Also, fixed bugs in MYNN surface layer scheme for initializattion and consistent
use of surface QV from RUC LSM over land and ice.
Copy link
Collaborator

@joeolson42 joeolson42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good, except for deleting the "pblh" in the print statement in module_sf_mynn.F90.

IF (debug_code >= 1) THEN
write(0,*)"ITIMESTEP=",ITIMESTEP," iter=",iter
DO I=its,ite
write(0,*)"=== important input to mynnsfclayer, i:", i
IF (dry(i)) THEN
write(0,*)"dry=",dry(i)," pblh=",pblh(i)," tsk=", tskin_lnd(i),&
write(0,*)"dry=",dry(i)," =",pblh(i)," tsk=", tskin_lnd(i),&
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets keep the "pblh"

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joeolson42 Sorry, Joe, I removed it by accident. Just committed the fix to that.

@@ -728,6 +796,7 @@ SUBROUTINE SFCLAY1D_mynn( &
GOVRTH(I)=G/TH1D(I)
ENDDO

!tgs - should QFX and HFX be separate for land, ice and water?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we go back to using the flux calculations, then yes they should be broken down into components.

@@ -1616,6 +1636,7 @@ SUBROUTINE SFCLAY1D_mynn( &

! Compute u* without vconv for use in HFX calc when isftcflx > 0
WSPDI(I)=MAX(SQRT(U1D(I)*U1D(I)+V1D(I)*V1D(I)), wmin)
!tgs - should USTM be separater for dry, icy, wet?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

USTM was a variable put in the scheme by the hurricane community a long time ago. I'm not sure if anyone uses it.

@@ -1662,6 +1686,7 @@ SUBROUTINE SFCLAY1D_mynn( &
!----AND COMPUTE THE MOISTURE SCALE (or q*)
!----------------------------------------------------

!tgs - should we have MOL and qstar separate for dry, icy and wet?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can probably make use out of both - especially for the negative qv problem. I think having a separate heat and moisture transfer coefficients may be the only way to fix that problem.

@@ -1750,15 +1775,16 @@ SUBROUTINE SFCLAY1D_mynn( &
! CALCULATE THE EXCHANGE COEFFICIENTS FOR HEAT (FLHC)
! AND MOISTURE (FLQC)
!------------------------------------------
!tgs - should FLQC, FLHC be separate for dry, icy and wet?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of the rest of the questions are up to us. If we go back to using the fluxes, which I think we want to do, then yes, we should break everything up into different types.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should go back to MYNN surface fluxes over water.

Copy link

@DomHeinzeller DomHeinzeller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great to the extent that I can tell. Let me run the regression tests with it (and create the fv3atm and ufs-weather-model PRs).

@DomHeinzeller DomHeinzeller changed the title Bug fixes in MYNN surface ayer scheme. Also, added lat/log to RUC LSM for debugging. Bug fixes in MYNN surface layer scheme. Also, added lat/log to RUC LSM for debugging. Nov 1, 2021
standard_name = longitude_in_degree
long_name = longitude in degree east
units = degree_east
dimensions = (horizontal_dimension)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tanyasmirnova all the debug runs in rt_ccpp_dev.conf that use RUC LSM crashed with

138: An error occurred in ccpp_physics_run for group physics, block    1 and thread    1 (ntX=   1)
138: Detected size mismatch for variable GFS_Data(cdata%blk_no)%Grid%xlon_d in group physics before lsm_ruc_run, expected      384 but got       32

I think this is because line 745 says (horizontal_dimension) instead of (horizontal_loop_extent).

Can you please fix this and run at least one manual test in DEBUG mode?

Thanks!

@climbfuji
Copy link

Thanks, Tanya! I kicked off the tests again.

@tanyasmirnova
Copy link
Collaborator Author

@climbfuji Thank you, Dom! I tried to run my usual C768 test in the debug mode, but it crashed with a different error:
....
652: WARNING from PE 652: diag_manager_mod::send_data_3d: A value for dynamics in field slp (Min: 1.01429E+003, Max: 1.20970E+003) is outside the range [ 8.00000E+002, 1.20000E+003], and not equal to the missing value.
652:
507: forrtl: error (65): floating invalid
507: Image PC Routine Line Source
507: fv3.exe 000000000B0B948E Unknown Unknown Unknown
507: libpthread-2.17.s 00002B9644110630 Unknown Unknown Unknown
507: fv3.exe 000000000611C59B Unknown Unknown Unknown
507: fv3.exe 0000000003E2B491 fv_mapz_mod_mp_la 556 fv_mapz.F90

Maybe it was caused by the FMS problem in the top of the gsl/develop branch that we saw before this PR?

@climbfuji
Copy link

@climbfuji Thank you, Dom! I tried to run my usual C768 test in the debug mode, but it crashed with a different error: .... 652: WARNING from PE 652: diag_manager_mod::send_data_3d: A value for dynamics in field slp (Min: 1.01429E+003, Max: 1.20970E+003) is outside the range [ 8.00000E+002, 1.20000E+003], and not equal to the missing value. 652: 507: forrtl: error (65): floating invalid 507: Image PC Routine Line Source 507: fv3.exe 000000000B0B948E Unknown Unknown Unknown 507: libpthread-2.17.s 00002B9644110630 Unknown Unknown Unknown 507: fv3.exe 000000000611C59B Unknown Unknown Unknown 507: fv3.exe 0000000003E2B491 fv_mapz_mod_mp_la 556 fv_mapz.F90

Maybe it was caused by the FMS problem in the top of the gsl/develop branch that we saw before this PR?

Probably.

Even with your bug fix, I still get crashes for all debug tests. I can't see why (hera is down today), so I will need to rerun the tests on Cheyenne. Stay tuned.

@DomHeinzeller
Copy link

There is still a problem with tsk_wat in MYNN SFC it seems. See attached screenshot.

Screen Shot 2021-11-02 at 11 03 52 AM

@DomHeinzeller
Copy link

@tanyasmirnova I think that in lines 664 to 746 in module_sf_mynn.F90 we should be using tskin_* instead of the local and yet uninitialized variables tsk_*. Can you confirm?

@tanyasmirnova
Copy link
Collaborator Author

@climbfuji Dom, I have a fix for this error. I will commit it soon. Please, stand by.

@joeolson42
Copy link
Collaborator

joeolson42 commented Nov 2, 2021 via email

@tanyasmirnova
Copy link
Collaborator Author

tanyasmirnova commented Nov 2, 2021

Joe, yes, this is the problem. I moved this piece of code to the beginning, and TSK_* are not defined yet. Should TSK_wat be 0.5 *(tskin_wat+tsurf_wat) even at ITIMESTEP=1?

@joeolson42
Copy link
Collaborator

joeolson42 commented Nov 2, 2021 via email

@DomHeinzeller
Copy link

Joe, yes, this is the problem. I moved this piece of code to the beginning, and TSK_* are not defined yet. Should TSK_wat be 0.5 *(tskin_wat+tsurf_wat) even at ITIMESTEP=1?

I just changed the first few instances of TSK_* to TSKIN_* but it still crashed in sfc_drv_ruc.F90: forrtl: severe (408): fort: (3): Subscript #1 of the array LEMITBL has value 0 which is less than the lower bound of 1 Image PC Routine Line Source fv3_gfs.x 0000000008B01750 Unknown Unknown Unknown fv3_gfs.x 00000000079D52CB lsm_ruc_mp_lsm_ru 170 sfc_drv_ruc.F90 fv3_gfs.x 0000000006D6F1B8 ccpp_fv3_hrrr_phy 617 ccpp_FV3_HRRR_physics_cap.F90 fv3_gfs.x 0000000006D57CF1 ccpp_fv3_hrrr_cap 157 ccpp_FV3_HRRR_cap.F90 fv3_gfs.x 00000000069064C5 ccpp_static_api_m 329 ccpp_static_api.F90 fv3_gfs.x 00000000068FC834 ccpp_driver_mp_cc 108 CCPP_driver.F90 fv3_gfs.x 00000000033B3D3C atmos_model_mod_m 722 atmos_model.F90 fv3_gfs.x 0000000002D453C9 module_fcst_grid_ 417 module_fcst_grid_comp.F90

On Tue, Nov 2, 2021 at 12:11 PM tanyasmirnova @.**> wrote: Joe, yes, this is the problem. I moved this piece of code to the beginning, and TSK_ are not defined yet. Should TSK_wat be 0.5 *(tskin_wat+tsurf_wat) even at ITIMESTE=1? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#112 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADLRR3WCF4XTK345RPPZRXTUKAW7VANCNFSM5HCUCOPA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
-- Joseph Olson Model Physics Branch Chief Environmental Prediction Advancement Division NOAA-Global Systems Laboratory Boulder, Colorado

The regression tests are still crashing in MYNN SFC. Looking at it.

@DomHeinzeller
Copy link

@tanyasmirnova @joeolson42 First of all, I had to make this change:

diff --git a/physics/module_sf_mynn.F90 b/physics/module_sf_mynn.F90
index 1ddee4bc..41e41991 100644
--- a/physics/module_sf_mynn.F90
+++ b/physics/module_sf_mynn.F90
@@ -661,6 +661,9 @@ CONTAINS
 !-------------------------------------------------------------------
       DO I=its,ite

+         ! PSFC ( in cmb) is used later in saturation checks
+         PSFC(I)=PSFCPA(I)/1000.
+
          IF (ITIMESTEP == 1) THEN
          !initialize surface specific humidity and mixing ratios for land, ice and water
             IF (wet(i)) THEN
@@ -749,8 +752,6 @@ CONTAINS
       ENDIF

       DO I=its,ite
-         ! PSFC ( in cmb) is used later in saturation checks
-         PSFC(I)=PSFCPA(I)/1000.
          QVSH(I)=QV1D(I)/(1.+QV1D(I))        !CONVERT TO SPEC HUM (kg/kg)
          THCON(I)=(100000./PSFCPA(I))**ROVCP
          ! DEFINE SKIN TEMPERATURES FOR LAND/WATER/ICE

Now the code is crashing in in module_sf_mynn.F90 in line 1181, because ZT_lnd(i) is zero:

       GZ1OZt_lnd(I)= LOG((ZA(I)+ZNTstoch_lnd(i))/ZT_lnd(i))

Can this be fixed?

@tanyasmirnova
Copy link
Collaborator Author

@joeolson42 @climbfuji Dom and Joe, I do not have any of the problems that you see in my test case. I run FV3_HRRR suite in the 3-km NA RRFS test case.
I put a print statement for ZT_lnd, and it never had a zero value in my test.

@climbfuji
Copy link

The fv3_gsd_noah_mynnsfc_debug regression test has the problem (and I think the other mynnsfc debug regression tests, too). They are using a binary landmask, you are using a fractional mask, correct? Maybe that makes the difference?

@tanyasmirnova
Copy link
Collaborator Author

Dom, we use frac_grid=.false. in RRFS. My guess is that MYNN surface layer might have problems with frac_grid=.true.

@joeolson42
Copy link
Collaborator

I don't see any values of 0 for zt*, but I am still getting a crash in sfc_drv_ruc.F90:

276: forrtl: severe (408): fort: (3): Subscript #1 of the array LEMITBL has value 0 which is less than the lower bound of 1
276:
276: Image PC Routine Line Source
276: fv3_gfs.x 0000000008B08B90 Unknown Unknown Unknown
276: fv3_gfs.x 00000000079D52D7 lsm_ruc_mp_lsm_ru 170 sfc_drv_ruc.F90
276: fv3_gfs.x 0000000006D6F1B8 ccpp_fv3_hrrr_phy 617 ccpp_FV3_HRRR_physics_cap.F90
276: fv3_gfs.x 0000000006D57CF1 ccpp_fv3_hrrr_cap 157 ccpp_FV3_HRRR_cap.F90
276: fv3_gfs.x 00000000069064C5 ccpp_static_api_m 329 ccpp_static_api.F90

@tanyasmirnova
Copy link
Collaborator Author

Joe,
The following line:
semisbase(i) = lemitbl(vtype(i))
just uses the table value of emissivity for the veg. type. Maybe you do not have Dom's changes to move vtype from real to integer?

@DomHeinzeller
Copy link

So far I tracked it back to ustar and restart having large values, which then cause the equations

          Zt = Z_0*EXP(-KARMAN*CZIL*SQRT(restar))
          Zt = MIN( Zt, 0.75*Z_0)

          Zq = Z_0*EXP(-KARMAN*CZIL*SQRT(restar))
          Zq = MIN( Zq, 0.75*Z_0)

to produce zeros: ustar and restar are something like 9113164117.8289948

@joeolson42
Copy link
Collaborator

joeolson42 commented Nov 2, 2021 via email

@DomHeinzeller
Copy link

DomHeinzeller commented Nov 2, 2021 via email

@tanyasmirnova
Copy link
Collaborator Author

I have changed the following lines in this PR:
DO I=its,ite
! CONVERT SKIN POTENTIAL TEMPERATURES TO VIRTUAL POTENTIAL TEMPERATURE:
THVSK_lnd(I) = THSK_lnd(I)(1.+EP1QVSH(I)) !(K)
THVSK_ice(I) = THSK_ice(I)(1.+EP1QVSH(I)) !(K)
THVSK_wat(I) = THSK_wat(I)(1.+EP1QVSH(I)) !(K)
ENDDO
Because Huge values were used in these lines. Iput THVSK computation for land into "dry" section, and so on. I also replaced QVSH (computed from the 1st level QV) with qsfc_lnd for land. I had problems with these lines, but got it work fine with RUC LSM. I added the check on THVSK:
if(THVSK_lnd(I) < 170. .or. THVSK_lnd(I) > 350.) &
print *,'THVSK_lnd(I)',i,THVSK_lnd(I),THSK_lnd(i),tsurf_lnd(i),tskin_lnd(i),qsfc_lnd(i)

Dom, do you see any prints for THVSK_lnd?

…ns in LSM, ice and

ocean. ! when iter = 1, flag_iter = .true. for all grids ! when iter =
2, flag_iter = .true. when wind < 2 for both land and ocean (when
nstf_name1 > 0)
2.Added flag_iter to all computations in MYNN surface layer scheme to
avoid inconsistencies with iterations in LSM, ice andocean models.
3.Removed averaging: 0.5*(tsurf + tskin) and used tskin instead, because
tsurf could be not defined.
4.Added checks on QSFC for non-defined values in LSM, ice and ocean models.
If there are not defined values, compute QSFC as at itimestep=1.
@tanyasmirnova
Copy link
Collaborator Author

I think I resolved the issues with the Noah LSM. I ran my C&^* test case in the debug mode, and it runs fine now.
Dom, could you, please, try again the regression tests? Thank you!

@climbfuji
Copy link

I think I resolved the issues with the Noah LSM. I ran my C&^* test case in the debug mode, and it runs fine now.
Dom, could you, please, try again the regression tests? Thank you!

Running this code now ...

@tanyasmirnova
Copy link
Collaborator Author

@climbfuji Dom, thank you very much for your help with this PR. Greatly appreciated!

@tanyasmirnova tanyasmirnova deleted the ruclsm_mynnsfclay branch November 8, 2021 17:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants