Introduction of new post phases for performance updates #146

mvertens · 2020-12-07T17:57:34Z

Description of changes

Introduction of new post phases for performance updates

Specific notes

This PR introduces new post phases for each component in order to remove duplicate mappings that were done in the prep phases.
Code Changes:

mapping has been moved from prep phases and put into the relevant post phases - EXCEPT for cases where updated ice fraction is needed in the mapping
med_merge_mod.F90 has been modified to zero out fields in the appropriate place (this should have been done this way in the initial implementation)
med_fraction_set is now a method that is called from med_phases_post_ice
med_phases_prep_ocn_map is no longer needed, med_phases_prep_ocn_merge and med_phases_prep_accum_fast have been unified into med_phases_prep_ocn_accum

Run Sequence Changes:

the auto generated runsequences in CIME have been modified to account for this
the following should be the new run sequence for UFS CMEPS warm run sequence

runSeq::
@3600
   MED med_phases_prep_ocn_avg
   MED -> OCN :remapMethod=redist
   OCN
   @900
     MED med_phases_prep_atm
     MED med_phases_prep_ice
     MED -> ATM :remapMethod=redist
     MED -> ICE :remapMethod=redist
     ATM
     ICE
     ATM -> MED :remapMethod=redist
     MED med_phases_post_atm
     ICE -> MED :remapMethod=redist
     MED med_phases_post_ice
     MED med_phases_prep_ocn_accum
   @
   OCN -> MED :remapMethod=redist
   MED med_phases_post_ocn
   MED med_phases_restart_write
   MED med_phases_profile
@
::

Contributors other than yourself, if any: None

CMEPS Issues Fixes: #145, #141

Are changes expected to change answers?

bit for bit
The results are bfb EXCEPT for the following mediator history differences due to the introduction of post phases:
AtmImp_ocn_xxx
lndExp_Sg_ice_covered_elevN and lndExp_Sg_topo_elevN in ERS_Vnuopc_Ld5.T31_g37_gl20.I1850Clm50SpG
different at roundoff level
more substantial

Any User Interface Changes (namelist or namelist defaults changes)?

Yes - the run sequence is different
No

Testing performed if application target is CESM:(either UFS-S2S or CESM testing is required):

(recommended) CIME_DRIVER=nuopc scripts_regression_tests.py
- machines:
- details (e.g. failed tests):
(required) CESM testlist_drv.xml
- machines and compilers: cheyenne/intel
- details (e.g. failed tests): compared to baseline nov28newrunseq/ and generated new baselines dec06
(optional) CESM prealpha test
- machines and compilers
- details (e.g. failed tests):

Testing performed if application target is UFS-S2S:

(required) UFS-S2S testing
- description:
- details (e.g. failed tests):

Hashes used for testing:

CESM:
- repository to check out: https://github.com/ESCOMP/CESM.git
- branch: nuopc_dev
- hash: dffd1e7
  This has contains branches for CDEPS and CIME (in addition to CMEPS) that need to have PRs to master.
UFS-S2S, then umbrella repostiory to check out and associated hash:
- repository to check out:
- branch:
- hash:

…lc->lnd for now - but also upcoming glc->ocn and glc->ice but only after a receive from glc

…dded post_rof to run sequence

…p' into mvertens/postphases

…tent with other phases and fixed bugs in run sequence for D and TG compsets

uturuncoglu · 2020-12-07T18:05:46Z

.github/pull_request_template.md


 Testing performed if application target is UFS-S2S:
- [ ] (required) UFS-S2S testing
+- [ ] (recommended) UFS-S2S testing
   - description:
   - details (e.g. failed tests):



Is it possible to add UFS-HAFS to the list?

@uturuncoglu - good point. I just added it.

uturuncoglu · 2020-12-07T18:10:48Z

mediator/med.F90

-    call ESMF_AttributeGet(gcomp, name="Profiling", value=cvalue, &
-         convention="NUOPC", purpose="Instance", rc=rc)
+    ! Obtain profiling level
+    call NUOPC_CompAttributeGet(gcomp, name="Profiling", value=cvalue, isPresent=isPresent, isSet=isSet, rc=rc)


We had a problem before related with setting verbosity and profiling. Does this fix it? Providing string such as low, high and max were not working and also there was some restriction in CIME side to set default value for the integer XML entiries.

What it does fix is that I can set the profiling to the maximum number (65535) and it will work. I am printing this out to the mediator log file now.

uturuncoglu · 2020-12-07T18:18:17Z

mediator/med_phases_prep_lnd_mod.F90

-
-  logical :: glc2lnd_coupling = .false.
-  logical :: cism_evolve = .false.
+  real(r8), pointer :: dataptr_scalar_lnd(:,:)


Do we need to assign them NULL?

good idea - thanks for pointing this out.

mediator/med_phases_post_lnd.F90

DeniseWorthen · 2020-12-07T20:11:21Z

In the current ufs run sequence, we have after ice->med, these 4 phases:

 MED med_fraction_set
 MED med_phases_prep_ocn_map
 MED med_phases_prep_ocn_merge
 MED med_phases_prep_ocn_accum_fast
 MED med_phases_profile

So med_phases_post_ice and prep_ocn_accum take care of the fraction_set, the ocn_map and the ocn_merge ?

mvertens · 2020-12-07T20:13:30Z

Yes - that is correct. Ultimately, this is a much simpler run sequence and that is beneficial to performance as well.

…

On Mon, Dec 7, 2020 at 1:11 PM Denise Worthen ***@***.***> wrote: In the current ufs run sequence, we have after ice->med, these 4 phases: MED med_fraction_set MED med_phases_prep_ocn_map MED med_phases_prep_ocn_merge MED med_phases_prep_ocn_accum_fast MED med_phases_profile So med_phases_post_ice and prep_ocn_accum take care of the fraction_set, the ocn_map and the ocn_merge ? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#146 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB4XCE5U5XHKRAJFOCV3QE3STUZHPANCNFSM4UQZZTCA> .

-- Mariana Vertenstein CESM Software Engineering Group Head National Center for Atmospheric Research Boulder, Colorado Office 303-497-1349 Email: mvertens@ucar.edu

DeniseWorthen · 2020-12-07T21:28:49Z

mediator/med_phases_post_ocn_mod.F90

+
+    ! local variables
+    type(InternalState)        :: is_local
+    character(len=*),parameter :: subname='(med_phases_prep_ice)'


This should be med_phases_post_ocn

DeniseWorthen · 2020-12-07T21:30:56Z

mediator/med_phases_prep_ocn_mod.F90

@@ -260,7 +131,7 @@ subroutine med_phases_prep_ocn_accum_avg(gcomp, rc)
    ! local variables
    type(InternalState)        :: is_local
    integer                    :: ncnt
-    character(len=*),parameter :: subname='(med_phases_prep_ocn_accum_avg)'
+    character(len=*),parameter :: subname='(med_phases_prep_ocn)'


here: med_phases_prep_ocn_avg

mvertens · 2020-12-07T21:31:05Z

Good point. I'll change it.

…

On Mon, Dec 7, 2020 at 2:29 PM Denise Worthen ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In mediator/med_phases_post_ocn_mod.F90 <#146 (comment)>: > + use ESMF , only : ESMF_GridComp + use ESMF , only : ESMF_LogWrite, ESMF_LOGMSG_INFO, ESMF_SUCCESS + use med_utils_mod , only : chkerr => med_utils_ChkErr + use med_constants_mod , only : dbug_flag => med_constants_dbug_flag + use med_map_mod , only : med_map_field_packed + use med_internalstate_mod , only : InternalState, logunit, mastertask + use esmFlds , only : compice, compocn + use perf_mod , only : t_startf, t_stopf + + ! input/output variables + type(ESMF_GridComp) :: gcomp + integer, intent(out) :: rc + + ! local variables + type(InternalState) :: is_local + character(len=*),parameter :: subname='(med_phases_prep_ice)' This should be med_phases_post_ocn — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#146 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB4XCE6FR25RSZOHPAVVWPDSTVCKDANCNFSM4UQZZTCA> .

-- Mariana Vertenstein CESM Software Engineering Group Head National Center for Atmospheric Research Boulder, Colorado Office 303-497-1349 Email: mvertens@ucar.edu

DeniseWorthen · 2020-12-08T12:55:43Z

I've tested this in my fork of ufs-weather (#4b1f313).

All cpld and datm RTs passed after making all the correct changes to the run sequences.

* adds multiple ice sheet functionality (ESCOMP#140) * adds post phases for performance enhancement (ESCOMP#146) * adds ocn->glc (land ice) coupling at multiple levels (ESCOMP#148)

Mariana Vertenstein and others added 22 commits November 28, 2020 12:57

added med_phases_post_glc_mod.F90 which carries out the mapping for g…

8c1a42f

…lc->lnd for now - but also upcoming glc->ocn and glc->ice but only after a receive from glc

added med_phases_post_glc to run sequence

b494a25

aded a post rof phase

5bc6ffa

merge to mvertens/icesheets

8f54ebe

fix avgdt variable and add nthreads value to gcomp

f520ad9

major performance update for I2000 compset

e045da7

reworked how scalar data was set in med_phases_prep_lnd

59ec4d1

more performance improvements

bfd5ca7

fixed bug introduced in prep_ocn_mod with skipping wave mapping and a…

4589a19

…dded post_rof to run sequence

updates for mapping glc->ocn

a722479

Merge remote-tracking branch 'jedwards/fix_avgdt_add_nthreads_to_gcom…

bcd340c

…p' into mvertens/postphases

removal of nems configuration - since nems driver is used in the future

de28293

fixed compile and restart problem for B compsets

7417a5a

Merge commit '7417a5a' into mvertens/postphases

f3e4d3a

added post phases for ocn and atm

7868fba

added med_phases_post_lnd

386e300

added post wave phase

9ea5b94

added post ice phase and fixed call to post atm phase

c310ff3

more cleanup of post run phases

f2edb0f

unified merge and accum ocn phase and renamed ocn phases to be consis…

435bb52

…tent with other phases and fixed bugs in run sequence for D and TG compsets

fixed problems encountered in PR testing

c570780

update to pull request template

4a82768

mvertens requested review from DeniseWorthen, jedwards4b and uturuncoglu December 7, 2020 17:57

jedwards4b approved these changes Dec 7, 2020

View reviewed changes

mvertens mentioned this pull request Dec 7, 2020

performance udpates ESCOMP/CDEPS#33

Merged

7 tasks

uturuncoglu approved these changes Dec 7, 2020

View reviewed changes

Mariana Vertenstein added 2 commits December 7, 2020 11:22

updated Makefile new post phase modules

bb8214c

updated pr template to add HAFS

94cde15

Mariana Vertenstein and others added 3 commits December 7, 2020 11:32

updated module arrays to be initialized to null

920bc97

update med.F90 dependancies

ab8b90c

update cmake file

70b5aac

DeniseWorthen reviewed Dec 7, 2020

View reviewed changes

mediator/med_phases_post_lnd.F90 Outdated Show resolved Hide resolved

Mariana Vertenstein and others added 2 commits December 7, 2020 12:51

corrected file name

c6b58ce

add post_ocn_mod

3a05967

DeniseWorthen reviewed Dec 7, 2020

View reviewed changes

DeniseWorthen approved these changes Dec 8, 2020

View reviewed changes

jedwards4b merged commit 0ccc872 into master Dec 8, 2020

mvertens deleted the mvertens/postphases branch December 8, 2020 16:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduction of new post phases for performance updates #146

Introduction of new post phases for performance updates #146

mvertens commented Dec 7, 2020 •

edited

Loading

uturuncoglu Dec 7, 2020

mvertens Dec 7, 2020

uturuncoglu Dec 7, 2020

mvertens Dec 7, 2020

uturuncoglu Dec 7, 2020

mvertens Dec 7, 2020

DeniseWorthen commented Dec 7, 2020

mvertens commented Dec 7, 2020 via email

DeniseWorthen Dec 7, 2020 •

edited

Loading

DeniseWorthen Dec 7, 2020

mvertens commented Dec 7, 2020 via email

DeniseWorthen commented Dec 8, 2020

Introduction of new post phases for performance updates #146

Introduction of new post phases for performance updates #146

Conversation

mvertens commented Dec 7, 2020 • edited Loading

Description of changes

Specific notes

uturuncoglu Dec 7, 2020

Choose a reason for hiding this comment

mvertens Dec 7, 2020

Choose a reason for hiding this comment

uturuncoglu Dec 7, 2020

Choose a reason for hiding this comment

mvertens Dec 7, 2020

Choose a reason for hiding this comment

uturuncoglu Dec 7, 2020

Choose a reason for hiding this comment

mvertens Dec 7, 2020

Choose a reason for hiding this comment

DeniseWorthen commented Dec 7, 2020

mvertens commented Dec 7, 2020 via email

DeniseWorthen Dec 7, 2020 • edited Loading

Choose a reason for hiding this comment

DeniseWorthen Dec 7, 2020

Choose a reason for hiding this comment

mvertens commented Dec 7, 2020 via email

DeniseWorthen commented Dec 8, 2020

mvertens commented Dec 7, 2020 •

edited

Loading

DeniseWorthen Dec 7, 2020 •

edited

Loading