global_cycle fractional grid support

[pjpegion]

The coupled model is now using a fractional grid, but the data assimilation still needs to use global_cycle in order to bring in the land DA analysis increments. Global cycle needs to be updated to be consistent with the code in NCAR/ccpp-physics#695

[GeorgeGayno-NOAA]

Note, there is a related issue (#424) to ensure the CCPP physics and UFS_UTILS use the same version of sfcsub.F.

[edwardhartnett]

Let us consider how this will be tested...

[jswhit2]

@GeorgeGayno-NOAA @arunchawla-NOAA this is important for coupled DA testing, right now it's blocking us from upgrading to P7b.

[GeorgeGayno-NOAA]

@yangfanglin @jswhit2 @pjpegion Where can I get sample surface restart files for a fractional grid case?

[GeorgeGayno-NOAA]

Note to self: sfcsub.F now takes two land mask arguments - slmskl and slmskw:

https://github.com/NCAR/ccpp-physics/blob/59fd4c8debdd075a361ef8ee9864f05354cfd8d6/physics/gcycle.F90#L215

And these are defined in gcycle.F90 as:

https://github.com/NCAR/ccpp-physics/blob/59fd4c8debdd075a361ef8ee9864f05354cfd8d6/physics/gcycle.F90#L125

[yangfanglin]

George, please @bing.fu@noaa.gov Bing Fu for the restart files.

[GeorgeGayno-NOAA]

Updated the call to sfcsub.F for the new interface (see 6c98ecd). For now, just consider the non-fractional case. Ran a C768 test case on Hera: /scratch1/NCEPDEV/da/George.Gayno/ufs_utils.git/global_cycle_fraction/input_data. Only tile1 was run (S. America and Africa). The test failed in sfcsub.F in routine newice

 FATAL ERROR: inconsistency in slifcs or slianl.
  at lat=-34.9 lon=305.0 slifcs= 1.0 slimsk= 0.0 tsffcs=280.0 set to tsfanl=280.0
abort:

Note there is a mismatch in the two masks - slifcs and slimsk. The former is land and the latter is open water. Something is wrong.

[GeorgeGayno-NOAA]

Some diagnostics from the failed test:

Before the call to SFCCYCLE, the land mask variables for the problem point are all 'land':

 slmskl check    1.00000000000000
 slmskw check    1.00000000000000
 slifcs check    1.00000000000000
 slmask check    1.00000000000000

Then, this loop in sfcsub.F updates slianl from '1' to '0'

      do i=1,len
        if (sicanl(i) >= min_ice(i)) then
          slianl(i) = 2.0_kind_io8
        else
          slianl(i) = zero
          sicanl(i) = zero
        endif
      enddo

and this causes the crash in newice.

[GeorgeGayno-NOAA]

Asked @SMoorthi-emc for help. He thinks there may be a bug - the loop logic should only test points with slmskl /= 1:

@@ -2018,15 +2033,21 @@
 !     if(lprnt) print *,'tsfclm=',tsfclm(iprnt),' tsfcl2=',tsfcl2(iprnt)
 !    *,' tsffcs=',tsffcs(iprnt),' slianl=',slianl(iprnt)

+      print*,'before minice update slianl ',slianl(3073)
+
       do i=1,len
+       if(nint(slmskl(i)) /= 1) then
         if (sicanl(i) >= min_ice(i)) then
           slianl(i) = 2.0_kind_io8
         else
           slianl(i) = zero
           sicanl(i) = zero
         endif
+       endif
       enddo

When I made this change, my test ran to completion without any errors.

[GeorgeGayno-NOAA]

Sample restart files from @jiandewang are on Hera in the /scratch1/NCEPDEV/nems/emc.nemspara/RT/NEMSfv3gfs/develop-20220601/INTEL/cpld_bmark_p8/RESTART directory.

[GeorgeGayno-NOAA]

The 'feature/cycle_frac' branch is way out of date. I will create a new one for this issue.

[jswhit]

Thanks @GeorgeGayno-NOAA, we really need this to start cycling experiments with Prototype 8