Support for multiple output grids

[DusanJovic-NOAA]

Description

fv3atm needs to be updated to support multiple output grids (parent, either global or regional, and any number of nests) using the write grid component. In order to be able to identify which domain is currently running on a given set of PETs, dycore needs to provide total number of domains (grids) and current grid on a given PET.
This PR adds new function (get_nth_domain_info) and also returns ngrids (total number of grids/domains) and mygrid from atmosphere_domain routine.

Fixes # (issue)

How Has This Been Tested?

ufs-weather-model has been run on hera.

Checklist:

Please check all whether they apply or not

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• Any dependent changes have been merged and published in downstream modules

[bensonr]

Please use mpp_error (NOTE, ...) in place of direct write statements. The different levels of mpp_error notification are NOTE, WARNING, and FATAL. A NOTE is written by the root_pe (master in many cases), WARNING is a note written by all pes, and a FATAL is a WARNING that terminates execution the after the error message is written.

[DusanJovic-NOAA]

Please use mpp_error (NOTE, ...) in place of direct write statements. The different levels of mpp_error notification are NOTE, WARNING, and FATAL. A NOTE is written by the root_pe (master in many cases), WARNING is a note written by all pes, and a FATAL is a WARNING that terminates execution the after the error message is written.

Done. See 9d1fabc

But I still see a lot of debug prints not using mpp_error (NOTE, ...), for example:

 60:   Cubed-Sphere Grid Stats :          361 x         361 x           1
 60:    11500.7187500000        12835.7197265625        12322.9931390828     
 60:    11500.7187500000        12835.7197265625     
 60:       Grid Length               : min:    11500.72 max:    12835.72 avg:    12322.99 min/max:        0.90
 60:       Deviation from Orthogonal : min:  0.42794721366590E-06 max:  0.69068407717646E+01 avg:  0.18057407885902E+01
 60:       Aspect Ratio              : min:  0.10000000778672E+01 max:  0.10056774732283E+01 avg:  0.10012675765938E+01
 60:  
 60:  da_max/da_min=   1.25388118160101     
 60:  da_max_c/da_min_c=   1.26038510350106     
 60:   
 60:  Divergence damping Coefficients
 60:  For small dt=   4.500000    
 60:  External mode del-2 (m**2/s)=  0.000000000000000E+000
 60:  Internal mode del-2 SMAG dimensionless coeff=  0.1000000    
 60:  Internal mode del-2 background diff=  0.000000000000000E+000
 60:  Internal mode del-6 background diff=  0.1500000    
 60:  tracer del-2 diff=  0.0000000E+00
 60:  Vorticity del-4 (m**4/s)=   6137755.10792274     
 60:  beta=  0.0000000E+00
 60:   
 60:  in fv_restart ncnst=           0
 60:  FV_RESTART:            1 F F
  0: NOTE from PE     0: ==> External_ic::get_nggps_ic: using tiled data file INPUT/gfs_data.nc for NGGPS IC
  0: NOTE from PE     0: ==> External_ic::get_nggps_ic: using tiled data file INPUT/sfc_data.nc for NGGPS IC
  0: NOTE from PE     0: ==> External_ic::get_nggps_ic: using tiled data file INPUT/oro_data.nc for NGGPS IC
  0: Tracer sphum initialized with surface value of 0.100000E+31 and vertical multiplier of   1.000000
  0: Tracer liq_wat initialized with surface value of 0.100000E+31 and vertical multiplier of   1.000000
  0: Tracer rainwat initialized with surface value of 0.100000E+31 and vertical multiplier of   1.000000
  0: Tracer ice_wat initialized with surface value of 0.100000E+31 and vertical multiplier of   1.000000
  0: Tracer snowwat initialized with surface value of 0.100000E+31 and vertical multiplier of   1.000000
  0: Tracer graupel initialized with surface value of 0.100000E+31 and vertical multiplier of   1.000000
  0: Tracer o3mr initialized with surface value of 0.100000E+31 and vertical multiplier of   1.000000
  0: Tracer sgs_tke initialized with surface value of 0.000000E+00 and vertical multiplier of   1.000000
  0: Tracer cld_amt initialized with surface value of 0.000000E+00 and vertical multiplier of   1.000000
  0:   ptop & ks    20.00000              21
  0:  GFS ak(1)=  0.0000000E+00  ak(2)=   20.00000    
  0: NOTE from PE     0: INPUT gfs_data source string: FV3GFS GAUSSIAN NETCDF FILE
  0: NOTE from PE     0: Opened BC file: INPUT/gfs_bndy.tile7.000.nc
  0: NOTE from PE     0: Opened grid file: INPUT/grid.tile7.halo4.nc
  0: NOTE from PE     0: Opened topo file: INPUT/oro_data.tile7.halo4.nc
  0: NOTE from PE     0: Opened BC file: INPUT/gfs_bndy.tile7.000.nc
  1:  sphum =            1
  1:  clwmr =            2
  1:   o3mr =            7
  1:  ncnst =            9
  1:  ntracers =            9

All these lines not starting with NOTE from PE ...

[bensonr]

@DusanJovic-NOAA - In general messages we want to be easily found and are purely character data will go through FMS message handling. File manipulation during initialization, is one of the areas where we want to the NOTE prefix as it's easy to miss input files during the workdir creation. For messages outputting non-character data, direct write statements are okay. As for messages from other components, they will have their own standards.

[DusanJovic-NOAA]

Ready for merge. All UFS regression tests finished successfully on all supported platforms.

[laurenchilutti]

@junwang-noaa This PR is ready to merge. Do you want me to add this to the commit queue prior to merging?

[junwang-noaa]

@laurenchilutti Sorry, it may not be clear. The related ufs-weather-model PR has already been in the commit queue (https://github.com/ufs-community/ufs-weather-model/wiki/Commit-Queue). Currently all RT tests passes and code review is done for dycore, fv3atm and ufs-weather-model. Now we will commit the code starting from this PR. Would you please merge the PR? Thanks.

[laurenchilutti]

@junwang-noaa It has been merged. Thank you for explaining.