Add GFSv17 IAU restart tests #2306
Comments
|
The RT test cpld_control_gfsv17_iau in ufs-weather-model generates consistent restart files for all the components. The test starts ar 2021032206 with fhrot=3 and total forecast length 30hr, the model produces forecast results from 2021032212 to 2021032312 (2021-03-23-43200). Below is the list of restart files: A test run directory can be seen on hera at: |
|
@aerorahul FYI. |
|
I checked the IAU test generated from global-workflow, the test case /scratch1/NCEPDEV/climate/Jessica.Meixner/IAUcheckpointrestarts/iau01/TMP/RUNDIRS/testiau/gfsfcst.2021032418/fcst.1642156 In the run directory we have the following in restart/FV3_RESTART: in restart/MOM6_RESTART: in restart/CICE_RESTART in restart/CMEPS_RESTART To make consistent restart files, we need to change the following line from: in model_configure: to: in model_configure: At this time, the atmosphere history files may not reproduce when the model restarts at fh=27, 57,87 and 117. New updates are required to let marine/CMEPS components to have flexible restart file output time, that will completely resolve the issue. |
|
@aerorahul FYI. |
|
@junwang-noaa Thanks. |
|
A GFSv17 IAU restart test cpld_restart_gfsv17_iau is added. Unfortunately the test does not reproduce the results from cpld_control_gfsv17_iau control test. Further debugging shows that the test reproduces in atmonly only mode and in s2s (atm-ocn-ice), but in coupled gfsv17 mode (atm-ocn-ice-wav). In the coupled gfsv17 mode, the coupling fields (e.g. wavImp_Sw_pstokes, wavImp_Sw_z0) that WW3 sends to ocn and atm do not reproduce in control run and restart run. for field wavImp_Sw_z0 that ww3 is sent to atm, it shows: |
|
The two run directory is at: The coupling fields in ufs.cpld.cpl.hi.wav.2021-03-22-68400.nc show that the import fields going to ww3 are identical in the control and restart run, but the coupled fields coming out of ww3 are different. @MatthewMasarik-NOAA would you please take a look? Thanks |
|
Hi @junwang-noaa, thank you for letting us know about this issue. We've added this to our task list and are in the process of prioritizing tasking for our group given our staffing for the next few months. I'll post back again shortly. |
|
Hi @junwang-noaa, I spoke with @sbanihash last week, who is now on leave through August. She advised to wait until she returned from leave for one of us to look into it. |
|
@MatthewMasarik-NOAA Thanks for letting me know. |
junwang-noaa
moved this from In Progress
to Blocked
in Model infrastructure development FY2024Q4
|
@junwang-noaa just wanted to let you know that we still haven't had time to dig into this but it is on our to-do list. Thank you for your patience. |
|
@junwang-noaa Wanted to let you know that I haven't had time to dig into this yet. I know you were also asking for any updates in the Monday meeting, so wanted to let you know that it is on my list, but haven't had time to get to it yet. Thanks for your patience. |
|
Trying to assess the issues here. I've tried using #2404 but am having trouble running the tests after trying to update. Is there something else I should be using for addressing this? i can use the workflow or something else if needed. |
|
@junwang-noaa, I've started looking at this and think I get to the same place with diffs first originating internal to WW3. I don't know if you were planning to continue looking at anything in particular. @JessicaMeixner-NOAA I'm not sure what was going wrong for you but this branch may be helpful: In short, cpld_control_gfsv17 restart reproduces control but IAU does not from FHROT=18. This is particularly confusing as I think IAUs are off before the IAU restart test even begins, right? If so, then this is just a normal restart test. Looking at the mediator history files written every coupling step, differences between cpld_control_gfsv17_iau_intel and cpld_restart_gfsv17_iau_intel first occur in: Note that ocn may not actually be last as it is in a 3600s slow coupling loop. Output is currently on hercules at /work2/noaa/stmp/nszapiro/stmp/nszapiro/FV3_RT/rt_2957580/ Via nccmp, we see that WAV receives the same variables b4b but sends different variables to CMEPS after the first hour of simulation: For maps, I'm plotting scatter points where |diff|>1.E-16 . These diffs occur globally, e.g., ncdiff of wavImp_Sw_pstokes_x1_gl_ufs.cpld.cpl.hi.wav.2021-03-23-03600.nc: I should say that I turned off Among the oddities, this restart test starts at fhrot=18 so I don't know why post is trying to make F15 Maybe it's helpful to share this update. I also wonder if 1) I understand the test correctly and 2) there are suggestions for where to dig in WW3 |
|
@NickSzapiro-NOAA Thanks for looking into this issue. I think you are getting the error that I had with the previous version. I don't have time to continue working on this issue. If you can share your branch that is updated to the latest ufs weather model develop branch with wave group, that will be great. Thanks |
|
Thanks @NickSzapiro-NOAA - I'll try to circle back to this next week from your branch. |
|
Just wanted to ping here and say I am starting to look at this now, no updates yet though. |
Description
GFSv17 is running with IAU turned on, the model should be able to restart with that configuration. A GFSv17 IAU restart test (cpld_restart_control_gfsv17_iau) based on cpld_control_gfsv17_iau needs to be added in RT.
Solution
Alternatives
Related to
The text was updated successfully, but these errors were encountered: