GFSv16.2.1 #899

- update GLDAS tag to gldas_gfsv16_release.v1.24.0 - update WAFS to gfs_wafs.v6.2.6 Refs: #399

- remove "-o" after compath.py in COMIN definitions - add "${envir}" and move closing ")" forward in line Refs: #399

Incorporate ecflow feedback from NCO - part 1

- update WAFS tag in sorc/checkout.sh and release notes Refs: #399

…NOAA/global-workflow into feature/ops-wcoss2 * 'feature/ops-wcoss2' of https://github.com/KateFriedman-NOAA/global-workflow: revert ecflow include files to NCO versions. Will adapt as necessary for proper use

Add PBS debug directive.

- update COMIN paths in GEMPAK JJOB scripts for COMINukmet, COMINecmwf, and COMINnam to add the respective systems to the end of the path definition Refs: #399

Refs: #399

Updates to ROTDIR/COMIN definitions related to compath.py and new GLDAS/WAFS tags

…e and intel. Ignore swp files

…nv-intel, craype and intel

Load compiler env. and modules in the ecf scripts.

…gnore in enkfgdas/post to ignore links

Request exclusive nodes for large memory jobs or jobs requesting all cores on WCOSS2 in ecf scripts.

…ster

…master

…he consistent grib_wafs ecf tasks

…recast hours

…gnore in enkfgdas/post to ignore links

…ster

…master

…he consistent grib_wafs ecf tasks

…recast hours

…m/NOAA-EMC/global-workflow into feature/nco-remove-duplicate-ecfs

Remove duplicate ecf scripts that are just copies of f000 in all cases.

Refs: #399

Update instructions in release notes for v16.2.0 to include final steps to run new ecf link script. Refs: #399

…_shared Update ecf scripts for memory and exclusivity; update release notes

- remove 500GB memory settings for jobs that are now running exclusive - add memory settings for jobs that run shared and didn't already have memory set - update GLDAS job to use 1 thread again Refs: #399

Refs: #399

- update workflow_utils.py to add "excl" PBS directive to WCOSS2 jobs when memory is not set (not running shared, running exclusive) Refs: #399

- change EMC tag in release notes instructions to be new hand-off tag - new tag: EMC-v16.2.0.1 Refs: #399

- update post job resources to use fewer nodes (2) - correct ecf script PBS resource line - update post jobs to run exclusive Refs: #399

- change post jobs to run on two nodes instead of ten Refs: #399

Series of small WCOSS2 updates - round 7

… of having a copy

…om %RUN%. This commit uses %RUN% to make it consistent and possibly will open the door for further consolidation between gdas and gfs families

…ources [Hotfix] remove `post/anl` ecf duplicate scripts and link to master instead.

Make job names consistent.

Bring NCO changes from 20220118

Refs: #399

- update GLDAS tag to new gldas_gfsv16_release.v1.25.0 Refs: #399

Update component tags based on NCO feedback 20220118

- remove COMINice and COMINwnd from JGLOBAL_WAVE_POST_SBS - wave developer confirmed they are no longer used/needed Refs: #399

…ng too many cores

… init jobs need modules for Intel loaded

…r overwrite section

… overwrite section

- update the memory setting in the get_resources block of workflow_utils.py to accept a separate gfs memory value - this change now allows gfs suite jobs to set different memory requirements compared to the gdas suite version of the job - this aligns the memory setting with how other resource values are differentiated between the suites Refs: #399

- add the missing get_awipsgroups function into the fcstonly version of the setup_workflow script (setup_workflow_fcstonly.py) - needed to add gfsawips jobs to xml in fcstonly mode Refs: #399

- remove unneeded MP_PULSE=0 setting; it's set in JJOB script - add if-block to set NTASKS=$NTASK_gfs if CDUMP=gfs; this allows waveprep to run with different # of tasks in gfs suite compared to gdas suite settings Refs: #399

- based on optimization testing the gdas and gfs analysis jobs will now use adjusted resource values in operations - update ecf PBS resource directive with new optimized settings Refs: #399

- based on optimization testing the gdasfcst job will use new resource settings at C768 - the updated resource settings run the gdasfcst with similar cores and timing compared to WCOSS1 ops Refs: #399

- folding in updates to resources based on NCO/EMC optimization testing - add some new gfs-specific variables so some jobs run differently in the gfs suite compared to the gdas suite - update to ppn value for the gdasfcst job (companion of config.fv3 updates) Refs: #399

- update the resources in the ecf script for the enkfgdas sfc job based on optimization testing Refs: #399

- add COMIN_OBS/COMIN_GES_OBS setting to config.base.emc.dyn for running analysis/enkf jobs in emc/dev mode; NCO mode uses compath.py in ops - COMIN_OBS/COMIN_GES_OBS needs to be set before use when in emc/dev mode - to support COMIN_GES_OBS setting in config.base values of GDATE, gPDY, gcyc, and GDUMP need to be set before config.base is read, therefore have added GDATE/gPDY/gcyc/GDUMP to generated rocoto xml envvars - CDUMP/PDY/cyc/CDUMP already in rocoto xml for similar job support - these changes should not impact NCO mode in ops, only emc/dev mode Refs: #399

Series of small WCOSS2 updates - round 8

- remove THEIA from machine if-block - remove extra whitespaces in a couple npe_wav* lines Refs: #399

…NOAA/global-workflow into feature/ops-wcoss2 * 'feature/ops-wcoss2' of https://github.com/KateFriedman-NOAA/global-workflow: Add COMIN_OBS/COMIN_GES_OBS and related xml support

- duplicate updates in config.resources.nco.static into the emc mode version (config.resources.emc.dyn) - use tested resources on WCOSS2 as starting point for emc mode Refs: #399

- fold in tested updates for C384 and C768 on WCOSS2 into emc mode version (config.fv3.emc.dyn) - use tested values on WCOSS2 as starting point for dev side of this port and support dev users in emc mode on WCOSS2 when running ops GFS Refs: #399

- update the gdasesfc, gdasanal, and gfsanal job resources in ecf scripts, based on optimization work between NCO/EMC Refs: #399

- based on optimization work between NCO/EMC, update the echgres job resources to be more appropriate for the larger node machine Refs: #399

Refs: #399

…o find a proper trigger for the remaining dump job

- fold in further resource optimization values from testing done by NCO/Wei and committed to ecf scripts in PR #603 Refs: #399

- adjust the analysis job walltimes to be 50mins for both the gdas and gfs suite jobs Refs: #399

Fixes requested by NCO on some of the resources used in operations.

@WeiWei-NCO

…re/ops-wcoss2 * upstream/feature/ops-wcoss2: Update analysis job walltime to 50mins bringing in changes from @WeiWei-NCO after his testing Update gdasechgres job resources Update esfc and analysis job resources update resources for more jobs in ecf scripts from NCO update resources for more jobs in ecf scripts from NCO update resources for jgfs_atmos_tropcy_qc_reloc.ecf. Remove developer overwrite section update resources for jgdas_atmos_tropcy_qc_reloc.ecf. Remove developer overwrite section update resources for more jobs to include memory in ecf scripts. wave init jobs need modules for Intel loaded update resources for wave jobs to include memory in ecf scripts update resources for atmos chgres for enkf in ecf scripts update resources for atmos pp wafs_gcip in ecf scripts update resources for atmos gempak_meta in ecf scripts update resources for atmos gempak in ecf scripts update resources for wave init, post and prep jobs in ecf scripts fix resource allocations for some jobs that NCO flagged were allocating too many cores

…/remove-obsproc-tasks * origin/feature/ops-wcoss2: Update analysis job walltime to 50mins Add COMIN_OBS/COMIN_GES_OBS and related xml support Update to WCOSS2 env file for waveprep job Add missing get_awipsgroups function to fcstonly Update memory setting in workflow_utils for gfs Remove unneeded COM paths from wavepostsbs JJOB

- remove unneeded WCOSS_C memory settings - remove unneeded THEIA memory reference - update awips job resources for atmosphere and wave Refs: #399

Remove obsproc tasks from ecflow suite.

WCOSS2 resource updates

- the usage of nth_max is not really needed on WCOSS2 with such large nodes compared to prior supported platforms - nth_max causes the incorrect thread value for the fcst job - removed all instances/use of nth_max from WCOSS2.env and retested jobs to confirm no negative impact from doing so (confirmed) Refs: #399

- per discussion with NCO/Wei the gdas analysis job walltime has been reduced closer to the actual runtime (back to 40mins) - NCO would rather not have to wait an extra ten minutes to learn of potential issues in the gdas analysis job while running in ops - left the gfs analysis job walltime at 50mins Refs: #399

- NCO/Wei reported that this specific job variable setting was causing issues with the DBNalerts and commented it out Refs: #399

- NCO/Wei caught an error in the wavepostbndpntbll config list in the associated JJOB script - updated the JJOB script to source the wavepostbndpntbll config instead of the wavepostbndpnt config at runtime Refs: #399

- NCO/Wei needed to change a few RUN entries to RUN2 in the JGFS_ATMOS_GEMPAK_PGRB2_SPEC JJOB script - updated tested and confirmed in NCO para testing on WCOSS2 Refs: #399

- similar to change made in JJOB script, NCO/Wei needed to change a few RUN variables to RUN2 for correct job execution - update tested and confirmed by NCO/Wei Refs: #399

- reduce gdas analysis job walltime back to 40 mins (closer to actual runtime) - increase npe_eobs back up to 480 tasks for correct job execution - updates tested and confirmed by both NCO and EMC Refs: #399

- based on optimization work between NCO/EMC the eobs and waveprep job resources are adjusted - eobs will use 7 nodes, similar to current WCOSS1 ops setting - gdaswaveprep will use 5 tasks and run shared with 100GB memory - gfswaveprep will use 65 tasks - gfspost walltime is set to 1hr (for dev users, shorter walltime in ecf scripts for ops) Refs: #399

Refs: #399

- turn on gdasgempak job (run in ops, was missing in rocoto) - add ROTDIR to gempak envars (now gempakenvars) Refs: #399

- move config and env file sourcing from gempak.sh into JJOBs to correct issue at runtime in ops - cleanup config.gempak accordingly Refs: #399

Refs: #399

- update the EMC hand-off tag to EMC-v16.2.0.2 Refs: #399

Refs: #399

Series of small WCOSS2 updates - round 9

- this tag fixes some non-WCOSS2 platform builds that were broken by the WCOSS2 changes Refs: #399

Refs: #399

…NOAA/global-workflow into feature/ops-wcoss2 * 'feature/ops-wcoss2' of https://github.com/KateFriedman-NOAA/global-workflow:

Update GLDAS tag to gldas_gfsv16_release.v1.28.0

- NCO/Wei found an infinite loop in the exgfs_atmos_gempak_meta.sh script during para testing - Provided and tested a fix, confirmed fix with Boi Refs: #399

- Russ Treadon provided resource adjustments to several GSI/EnKF jobs to get them closer to their WCOSS1 ops timings and use fewer cores - The enkfgdas_select_obs (gdaseobs), enkfgdas_update (gdaseupd), gdas_atmos_analysis_diag (gdasanaldiag), and gdas_atmos_analysis (gdasanal) jobs have been updated - The updated resources have been incorporated into the relevant ecf script PBS directives and resource configs Refs: #399

- not needed since setup script math results in 9ppn for eupd when using 14 threads Refs: #399

Refs: #399

- the new EMC hand-off tag is EMC-v16.2.0.3 Refs: #399

- clean up config.resources.nco.static and config.fv3.nco.static to remove non-WCOSS2 and non-ops refererences - both nco.static resource configs now only support operations on WCOSS2 and preserve the operational resource settings - the emc.dyn resource configs will be modified further to support other tier 1 platforms Refs: #399

Series of small WCOSS2 updates - round 10

Refs: #399

- increase gdas_atmos_analysis walltime to 50mins - reduced gfs_atmos_analysis walltime to 40mins - based on feedback from NCO/Wei Refs: #399

Analysis walltime adjustment and gempak script fix

jenkfgdas_sfc.ecf jgfs_wave_prdgen_bulls.ecf jgdas_wave_postsbs.ecf The following scripts changed to remove module load cfp: jenkfgdas_select_obs.ecf jenkfgdas_fcst.ecf jgfs_atmos_awips_master.ecf jgfs_atmos_wafs_grib2.ecf jgfs_atmos_awips_g2_master.ecf jgfs_atmos_fbwind.ecf jgfs_atmos_gempak_ncdc_upapgif.ecf jgfs_atmos_npoess_pgrb2_0p5deg.ecf jgfs_atmos_pgrb2_spec_gempak.ecf jgfs_wave_prdgen_bulls.ecf jgfs_forecast.ecf jgdas_forecast.ecf jgdas_atmos_gempak_meta_ncdc.ecf The following scripts changed to remove module load grib_util: jenkfgdas_fcst.ecf jgfs_atmos_postsnd.ecf jgfs_atmos_wafs_gcip.ecf jgfs_atmos_gempak.ecf jgfs_atmos_gempak_ncdc_upapgif.ecf jgfs_atmos_gempak_meta.ecf jgfs_atmos_npoess_pgrb2_0p5deg.ecf jgfs_atmos_pgrb2_spec_gempak.ecf jgfs_wave_prep.ecf jgfs_wave_gempak.ecf jgfs_wave_prdgen_bulls.ecf jgfs_forecast.ecf jgdas_forecast.ecf jgdas_atmos_gempak.ecf jgdas_atmos_gempak_meta_ncdc.ecf jgdas_wave_prep.ecf The following scripts changed to remove module load libjpeg: jenkfgdas_fcst.ecf jgfs_atmos_postsnd.ecf jgfs_atmos_wafs_gcip.ecf jgfs_atmos_gempak.ecf jgfs_atmos_gempak_ncdc_upapgif.ecf jgfs_atmos_gempak_meta.ecf jgfs_atmos_npoess_pgrb2_0p5deg.ecf jgfs_atmos_pgrb2_spec_gempak.ecf jgfs_wave_gempak.ecf jgfs_wave_prdgen_bulls.ecf jgfs_forecast.ecf jgdas_forecast.ecf jgdas_atmos_gempak.ecf jgdas_atmos_gempak_meta_ncdc.ecf The following scripts changed to remove module load gempak jgfs_wave_prdgen_bulls.ecf The following scripts changed to remove module load hdf5 jgfs_wave_prdgen_bulls.ecf The following scripts changed to remove module load netcdf jgfs_wave_prdgen_bulls.ecf The following scripts changed to remove module load bufr jgfs_wave_prdgen_bulls.ecf The following scripts changed to remove module load esmf jgfs_forecast.ecf jgdas_forecast.ecf Merge NCO resource change: jgfs_atmos_analysis.ecf jgfs_wave_post_bndpnt.ecf jgfs_wave_postpnt.ecf jgdas_atmos_analysis.ecf jgdas_wave_postpnt.ecf GitHub issue #587

- the gdas and gfs walltimes were flipped, gdas should be 50mins and gfs should be 40mins, have corrected to match ecf resource settings Refs: #399

A check on job/ush/script from HOMEgfs, I found the following reference to USE_CFP: gldas_forcing.sh exgdas_atmos_chgres_forenkf.sh exgdas_atmos_gldas.sh exgdas_enkf_update.sh exglobal_atmos_analysis.sh exglobal_diag.sh Therefore, remove the export USE_CFP=YES from the following ecflow scripts: jenkfgdas_ecen.ecf jenkfgdas_sfc.ecf jenkfgdas_select_obs.ecf jenkfgdas_post_master.ecf jenkfgdas_fcst.ecf jgdas_forecast.ecf jgdas_atmos_analysis_calc.ecf jgdas_atmos_emcsfc_sfc_prep.ecf jgdas_atmos_tropcy_qc_reloc.ecf jgdas_atmos_gempak_meta_ncdc.ecf jgdas_atmos_post_manager.ecf jgfs_atmos_awips_master.ecf jgfs_atmos_wafs_master.ecf jgfs_atmos_wafs_blending_0p25.ecf jgfs_atmos_wafs_blending.ecf jgfs_atmos_wafs_grib2.ecf jgfs_atmos_wafs_grib2_0p25.ecf jgfs_atmos_awips_g2_master.ecf jgfs_atmos_fbwind.ecf jgfs_atmos_analysis_calc.ecf jgfs_atmos_emcsfc_sfc_prep.ecf jgfs_atmos_tropcy_qc_reloc.ecf jgfs_atmos_gempak.ecf jgfs_atmos_gempak_ncdc_upapgif.ecf jgfs_atmos_npoess_pgrb2_0p5deg.ecf jgfs_atmos_pgrb2_spec_gempak.ecf jgfs_atmos_post_manager.ecf jgfs_forecast.ecf 2. Undo change related to resource updates 3. Remove esmf from efcs. Reference to GitHub #587

Fixed #534

Remove hardwired ImageMagick path for file util/ush/make_tif.sh

Correct analysis walltimes in config.resources for WCOSS2

jgdas_wave_prep.ecf jgfs_wave_prep.ecf

Consistency check of use of modules and variables in ecf job scripts

- update the NETCDF_LDFLAGS_F flags in gaussian_sfcanl makefile.sh - corrects the flags based on testing on R&D platforms Refs: #399

- use OMP_PLACES=cores setting based on optimization work for forecast job on Dogwood Refs: #399

- correct wrong w3emc_ver in run.ver to match version in build.ver - per feedback from NCO update the craype_ver to 2.7.10 and cray_pals_ver to 1.0.17 Refs: #399

- update the resource settings for the gfs_forecast job based on optimization work on Dogwood Refs: #399

- per request from NCO have moved the excl setting from it's separate line up into the main resource PBS directive line for jobs that run exclusively Refs: #399

Updates GSI tag from gfsda.v16.1.5 to gfsda.v16.1.6. Delivery Order 4 (DO-4) of NOAA's commercial GPS-RO data buy includes occultations from both Spire and GeoOptics. GeoOptics was previously assimilated in DO-2 and Spire is assimilated currently in DO-3. The update for DO-4 is a single line change in the global_convinfo.txt fix file to turn the GeoOptics data assimilation back on. Updates were also made to include a new historical fix file and update documentation. Refs: #656

- a few small adjustments to scripts for correctness on WCOSS2 Refs: #399

- ingest run.ver updates from NCO - create wcoss2.ver to set versions removed from run.ver but needed to support developers (e.g. prod_util) Refs: #399

- hand-off tag is now EMC-v16.2.0.4 Refs: #399

Refs: #399

- per NCO request created new transfer folder under /parm and moved all transfer list parm files into new folder Refs: #399

Refs: #399

- add sourcing of wcoss2.ver to load_fv3gfs_modules.sh - update wcoss2.ver to allow overrides from ecflow for prod_util_ver and prod_envir_ver when run in ops Refs: #399

- change matches update also done in config.resources.nco.static Refs: #399

Fixed typo in release notes. Updated the commented out block in config.anal which has the next needed historical convinfo file. Refs: #656

Refs: #399

- remove the commented out lines from the enkf transfer lists which are not needed - did not touch comment sections at top Refs: #399

- move all place=vscatter and place=excl settings in ecf scripts to their own line - now have one line for PBS select information and one line for PBS place information - changes decided on in collaboration between NCO and EMC Refs: #399

Series of small WCOSS2 updates - round 11

- increment the current operational GFS version in the release notes from v16.1.5 to v16.1.6 - the DO-4 v16.1.6 upgrade is happening before v16.2.0 and was also folded into the v16.2.0 package (GSI component) Refs: #399, #656

…ure/ops-wcoss2 * upstream/release/gfs.v16.1.6: GFS v16.1.6: Update release notes and comment in config.anal GFS v16.1.6: GSI update to add commercial GPSRO in DO-4

Refs: #399

- update the BASE_GIT and DMPDIR paths for WCOSS2 in the setup scripts to move them to the emc.global group account area and away from the initially established folders in a personal account space Refs: #399, #662

Fold GFSv16.1.6 updates into GFSv16.2.0 package

- new GLDAS tag fixes default path for CPCGAUGE to point to group account space for GDA on WCOSS2 instead of personal account Refs: #399, #662

Update GLDAS tag to gldas_gfsv16_release.v.2.0.0

A change for obsproc is being bundled with the DO-4 changes. Observation type 224 for uv needs to be set to monitor mode. The GSI tag name remains the same. The only changes in the workflow are to update the release notes and the comment in the config.anal file. Refs: #656

- update the GSI component submodule command to include "--init"; this corrects an issue during the libsrc submodule checkout - update release notes to include new EMC-v16.1.6.2 tag that will be cut after this is committed to release/gfs.v16.1.6 branch Refs: #656

- Add "/gempak" subfolder to several HPCUKMET, grid, and grid3 path variables used in gempak ush scripts for ukmet data - Boi Vuong and Wei Wei tested on WCOSS2 to confirm GDAS ukmet data match in terms of size between WCOSS1 and WCOSS2 Refs: #399

Add needed gempak subfolder to gempak ush scripts

- add obsproc_run_ver and prepobs_run_ver to wcoss2.ver for use when pointing to package installs during prep job Refs: #399

- add modulefile load from prepobs install - update JJOB name to JOBSPROC_GLOBAL_PREP and path to adjusted install location variable $HOMEobsproc Refs: #399

- change HOMEobsproc_prep to HOMEobsproc - remove HOMEobsproc_network and HOMEobsproc_global; replace with HOMEprepobs - point HOMEobsproc to group account BASE_GIT obsproc install using new obsproc_run_ver variable set in $target.ver files - point HOMEprepobs to group account BASE_GIT prepobs install using new prepobs_run_ver variable set in $target.ver files Refs: #399

- remove HOMEobsproc_prep, HOMEobsproc_network, and HOMEobsproc_global - add HOMEobsproc and point it to NCO ops install location Refs: #399

Update prep job functionality to use new WCOSS2-port obsproc/prepobs packages

- update to getjtbul.f for syndat_getjtbul.fd code - update provided by Qingfu Liu Refs: #698

Code update to syndat_getjtbul.fd for v16.1.7

add Release_Notes.gfs.v16.1.7.txt

…ure/ops-wcoss2 * upstream/release/gfs.v16.1.6: Update GSI submodule command and release notes GFS v16.1.6 update: Turn off uv 224 VADWND

Refs: #399, #702

Sync final GFSv16.1.6 updates into GFSv16.2.0 package

Refs: #399

Add WCOSS2 operational GFS defs files

- Based on optimization work done by EIB on WCOSS2. - Adjust the C768 gfs layout, thread #, I/O groups, and mpiprocs to a more optimized and consistent configuration. - This configuration has a faster initialization (~7mins) and more consistent output frequency to benefit downstream users. - This configuration runs in approximately 6260s (~104mins) and is under the max core # compared to WCOSS1. Refs: #399

- Job encountered significant runtime variability in ops - Investigation team recommended setting hyper=true to help Refs: #399

WCOSS2 GFS ops updates to address two issues (anal_calc & gfs_forecast)

- the nco module has a dependency on the gsl module being loaded - add gsl_ver=2.7 to versions/run.ver - add gsl module load to module_base.wcoss2.lua - add gsl module load to wave_prep ecf scripts Refs: #399

- Testing in WCOSS2 ops had an OOM kill with 40GB set. - Increase job memory request to 60GB. Refs: #399

- NCO requested to increase post job task number from 112 to 126 to more fully use the exclusive node on WCOSS2. Refs: #399

…ure/ops-wcoss2 * upstream/release/gfs.v16.1.7: update file Release_Notes.gfs.v16.1.7.txt add Release_Notes.gfs.v16.1.7.txt Code update to syndat_getjtbul.fd for v16.1.7

- Recent updates to the system result in a new tag for hand-off to NCO. - The new tag from EMC will be "EMC-v16.2.0.5". Refs: #399

- The v16.1.7 update is going into WCOSS1 operations before go-live on WCOSS2, so the release notes needed to be updated to reflect this new prior version and that certain things are not changing with regard to this operational versions. - Change instances of "v16.1.6" to "v16.1.7". Refs: #399, #698

- While adding the gsl module to module_base.wcoss2.lua and run.ver I examined the need to also add it to build.ver since the nco module is referenced in there via nco_ver. Examined the builds of the GFS components and found that the nco module isn't loaded/used in any of them. Therefore, nco_ver isn't needed in build.ver since nco isn't used at buildtime, only used at runtime in the wave_prep jobs. - Also, the gsl module isn't needed in build.ver because of this. The udunits module version already wasn't in build.ver (another dependency for the nco module) so this further shows that nco isn't needed at buildtime since the builds should have failed previously without that dependency. - Tested this by removing nco_ver from build.ver and running build_all.sh. No build issues were encountered. - Discussed this with NCO/Wei and we agreed to remove it from build.ver. Refs: #399

GFSv16.1.7 updates and changes from NCO for GFSv16.2.0

Wen Meng cut a new UPP tag after integrating a requested change from NCO for adjusting the icnt value from 1000 to 1080 in scripts/exglobal_atmos_pmgr.sh to resolve a bug. Refs: #399

Update UPP tag to upp_v8.1.2

- Based on conversations with NCO and EMC developers, changing the cpu-bind launcher flag argument from "core" to "depth" with threading. - Change "--cpu-bind core" to "--cpu-bind depth". Refs: #399

Update WCOSS2 env file cpu-bind flags: core to depth

- The new hand-off tag to NCO is EMC-v16.2.0.6 Refs: #399

- The HOMEobsproc setting in config.base.nco.static is not used in operations and thus not needed in this version of config.base. Refs: #399

- Update on Dogwood implemented cgroups, which means memory limits are now enforced. - Exclusive jobs must now use "place=exclhost" insted of "place=excl". - Associated exclusive ecf script PBS statements are updated to exclhost. Refs: #399

Need to set exclhost for exclusive jobs on WCOSS2 now after cgroups was implemented. Matches updates to exclusive job ecf script PBS statements. Refs: #399

Will consider removing at later date. Refs: #399

Update "excl" to "exclhost" for exclusive jobs on WCOSS2

- Update config.efcs to run EnKF forecast job with serial netcdf instead of parallel netcdf. Based on joint decision between NCO and EMC. - Update C384 config.fv3.nco.static block to set DELTIM=200 (NCO request). - Update C384 config.fv3.nco.static block to set WRITE_GROUP=2 to speed up serial EnKF forecast jobs to fit inside needed window in ops. Refs: #399

- Reviewed the various global-workflow utility code builds and added "-g" and/or "-traceback" flags if missing. - Added to only WCOSS2 makefiles/modulefiles for now for ops. Refs: #399, #791

Bo Cui updated gfs_bufr.sh to improve error handling Refs: #399, #790

- remove hyper=true in jgdas_atmos_analysis_calc.ecf - add export nth_echgres=$nth_echgres_gfs when CDUMP=gfs in config.analcalc; for correct thread setting at runtime - add export nth_echgres=4 to analcalc block in config.resources - add export nth_echgres_gfs=12 to analcalc block in config.resources Refs: #399

Hand-off tag to NCO is now EMC-v16.2.0.7 Refs: #399

Final pre-production freeze updates for GFSv16.2.0 package on WCOSS2

- NCO updated the default path for HOMENHC and tested it in prod on WCOSS2 during NHC test Refs: #399

- Based on testing on Dogwood after some WCOSS2 updates some memory and resource adjustments were made by NCO. - Memory updates to the gempak, awips, and fbwnd job ecf scripts. - Resource adjustments to remedy oversubscription errors in the post and postsnd jobs. Refs: #399

The gfspostsnd job was oversubscribing CPUs on WCOSS2 after updates on Dogwood. Updating resources settings to get them matching and working. Refs: #399

Refs: #665, #798

- Add updated memory values for awips and gempak jobs into resource configs to match similar updates in ecf scripts Refs: #399

WCOSS2 GFSv16.2.0 resource updates and NHC change

Co-authored-by: Rahul Mahajan <aerorahul@users.noreply.github.com>

for ops: Update calculation of starting time of rerun based on if wave restarts exist

Refs: #399

- Resource adjustments for the postsnd bufr_sounding job (based on GDIT and Bo Cui testing). - Fix to gfs_bufr code (Bo Cui). - gfs_bufr build update to no longer build with "-qopenmp". Refs: #399

Refs: #399

Fix to gfs_bufr code and job

@WeiWei-NCO

Merge remote-tracking branch 'origin/feature/ops-wcoss2' into operations_v16.2.1 * origin/feature/ops-wcoss2: (422 commits) Add release notes for GFSv16.2.1 Update to transfer_rdhpcs_gfs_nawips.list by NCO Bug fix for gfs_bufr (postsnd) job Adjust memory value for fbwind job update logic for linking wave restart files for rerun Update no rerun wave linking logic statement add error message and exit if wave model does not have restart on rerun add checking for wave restarts for RERUN date calculation Matching memory updates for awips/gempak in config Update prior GFS version in v16.2.0 release notes Update gfspostsnd job resources - oversubscribing Memory and resource adjustments to some jobs (NCO) Update to HOMENHC default path in JGLOBAL_ATMOS_TROPCY_QC_RELOC Update EMC tag name in v16.2.0 release notes Resource updates for analysis_calc job on WCOSS2 Updated error handling in gfs_bufr script Add -g and -traceback flags to utility builds if missing EnKf forecast serial netcdf updates and DELTIM=200 Add HOMEobsproc back to config.base.nco.static Update "excl" to "exclhost" in workflow_utils.py Update ecf PBS excl to exclhost Remove reference to HOMEobsproc in NCO config.base Update GFSv16.2.0 release notes for new hand-off tag Update WCOSS2 env file cpu-bind flags for threading Update UPP tag to upp_v8.1.2 Remove nco_ver from build.ver - not needed Update release notes to update prior version Update GFSv16.2.0 release notes to reflect new tag Increase post_master job to 126 tasks Update enkfgdas_sfc job to use 60GB Add gsl module load needed by nco module Set hyper=true for gdas_atmos_analysis_calc job Optimized gfs_forecast job resource configuration Add WCOSS2 operations gfs defs files Add missing --init flag to GSI checkout submodule update file Release_Notes.gfs.v16.1.7.txt add Release_Notes.gfs.v16.1.7.txt Code update to syndat_getjtbul.fd for v16.1.7 Update HOMEobsproc paths in config.base.nco.static Update obsproc package settings in dev config.base Update prep.sh to use new WCOSS2 obsproc packages Add obsproc/prepobs run versions to wcoss2.ver Add needed gempak subfolder to gempak ush scripts Update GSI submodule command and release notes GFS v16.1.6 update: Turn off uv 224 VADWND Update GLDAS tag to gldas_gfsv16_release.v.2.0.0 Update DMPDIR and BASE_GIT paths for WCOSS2 Update Externals.cfg with GFSv16.2.0 component versions Update release notes for current ops version Move all PBS place settings to separate line Remove commented out lines from transfer lists Update WAFS tag to gfs_wafs.v6.2.8 GFS v16.1.6: Update release notes and comment in config.anal Update npe_node_fcst_gfs in config.resources.emc.dyn Updates to support wcoss2.ver Fold in transfer parm list updates from NCO Move transfer lists into new transfer folder Update wave job resources with NCO feedback Update EMC tag name in v16.2.0 release notes Updates to run.ver and create wcoss2.ver Script updates from NCO GFS v16.1.6: GSI update to add commercial GPSRO in DO-4 Move excl setting into resource line in ecf scripts Update gfs_forecast job resources Update several versions in run.ver Add OMP_PLACES=cores for fcst block in WCOSS2.env Update compilation flags for gaussian_sfcanl build Add the following scripts changed to remove module load libjpeg: jgdas_wave_prep.ecf jgfs_wave_prep.ecf Remove hardwired DELL path util/ush/make_tif.sh Remove esmf from enkf fcst 1. A check on job/ush/script from HOMEgfs, I found the following reference to USE_CFP: gldas_forcing.sh exgdas_atmos_chgres_forenkf.sh exgdas_atmos_gldas.sh exgdas_enkf_update.sh exglobal_atmos_analysis.sh exglobal_diag.sh Correct analysis job walltimes in config.resources The following scripts changed to remove module load wgrib2: jenkfgdas_sfc.ecf jgfs_wave_prdgen_bulls.ecf jgdas_wave_postsbs.ecf Adjust analysis job walltimes for ops Add missing EXPDIR setting to JGDAS_ATMOS_GEMPAK Remove non-WCOSS2 references in nco.static configs Update EMC tag name in release notes Change npe_analdiag to 96 Remove npe_node_eupd=9 setting on WCOSS2 Update several GSI/EnKF job resources Update to correct infinite loop in gempak script Add missing character to GLDAS tag in release notes Update GLDAS tag to gldas_gfsv16_release.v.1.28.0 Remove excl for gfswaveprep job PBS directive Update GFSv16.2.0 release notes GEMPAK_META script updates from Boi Update GEMPAK scripts Update gempak job in setup_workflow scripts Back out comment of job variable in awips scripts Resource adjustments for eobs, waveprep, gfspost Resource updates for analysis and eobs Change RUN to RUN2 in awips scripts Change RUN to RUN2 in gempak pgrb2 spec script Correct config list for wavepostbndpntbll job Comment out job variable in awips ecf scripts Reduce gdas analysis job walltime back to 40mins Remove nth_max usage in WCOSS2.env A few resource updates from NCO and WCOSS_C removal Update analysis job walltime to 50mins Optimization resource updates from NCO remove obsproc ecfs and there references from suite.def. work needs to find a proper trigger for the remaining dump job bringing in changes from @WeiWei-NCO after his testing Update analysis job walltime to 50 mins Update gdasechgres job resources Update esfc and analysis job resources Update C384 and C768 values in config.fv3.emc.dyn Update config.resource.emc.dyn with tested values Cleanup of config.fv3.nco.static Add COMIN_OBS/COMIN_GES_OBS and related xml support Add COMIN_OBS/COMIN_GES_OBS and related xml support Update resources for gdasesfc job in ecf script Numerous resource updates based on optimization Update for C768 gdasfcst job resource settings Update analysis job ecf resource settings Update to WCOSS2 env file for waveprep job Add missing get_awipsgroups function to fcstonly Update memory setting in workflow_utils for gfs update resources for more jobs in ecf scripts from NCO update resources for more jobs in ecf scripts from NCO update resources for jgfs_atmos_tropcy_qc_reloc.ecf. Remove developer overwrite section update resources for jgdas_atmos_tropcy_qc_reloc.ecf. Remove developer overwrite section update resources for more jobs to include memory in ecf scripts. wave init jobs need modules for Intel loaded update resources for wave jobs to include memory in ecf scripts update resources for atmos chgres for enkf in ecf scripts update resources for atmos pp wafs_gcip in ecf scripts update resources for atmos gempak_meta in ecf scripts update resources for atmos gempak in ecf scripts update resources for wave init, post and prep jobs in ecf scripts fix resource allocations for some jobs that NCO flagged were allocating too many cores Remove unneeded COM paths from wavepostsbs JJOB Update GLDAS tag in release notes Update GLDAS tag to gldas_gfsv16_release.v1.25.0 wave init jobs just need cray-pals per NCO. remove rest put NCO identified changes from the global-workflow in a branch remove gdas remnant from enkfgdas jobnames some PBS jobnames were hardwired gdas or gfs, while some inherited from %RUN%. This commit uses %RUN% to make it consistent and possibly will open the door for further consolidation between gdas and gfs families post/anl job is the same as the forecast hour. create a link, instead of having a copy Update post job resources in config.resources.nco.static Update post jobs ecf script resources Update release notes for new EMC tag Update workflow_utils.py to support exclusive Add imagemagick_ver=7.0.8-7 to run.ver Update NCO resource config for memory Update v16.2.0 release notes for ecf script linking Add memory setting to jgfs_atmos_wafs_master.ecf Add memory setting to jgfs_atmos_wafs_grib2_0p25.ecf Add memory setting to jgfs_atmos_wafs_grib2.ecf Add memory setting to jgfs_atmos_wafs_blending_0p25.ecf Add memory setting to jgfs_atmos_wafs_blending.ecf Add memory setting to jgfs_atmos_awips_g2_master.ecf Add memory setting to jgfs_atmos_awips_master.ecf Add excl tag to jgfs_atmos_gempak.ecf Add excl tag to jgfs_forecast.ecf add pesky blank lines at the end of script. reviewers are brutal add script that sets up the links to the master.ecf that loop over forecast hours add gitignore in appropriate places to ignore links. update defs to the consistent grib_wafs ecf tasks remove duplicate jgfs_atmos_wafs_f*.ecf files and rename f00 as master remove duplicate jgfs_atmos_awips_g2_f*.ecf files and rename f000 as master remove duplicate jgfs_atmos_awips_f*.ecf files and rename f000 as master remove duplicate gfs_atmos_post_fxxx.ecf files and rename f000 as master fix typo that causes the opposite effect ignore gdas/atmos/post/ forecast hour ecf links remove duplicate gdas_atmos_post_fxxx.ecf files and rename f000 as master remove remnant from PR555 that copied gdas/enkf to enkfgdas. add gitignore in enkfgdas/post to ignore links remove duplicate enkfgdas_post_fxxx.ecf files and rename f003 as master add script that sets up the links to the master.ecf that loop over forecast hours add gitignore in appropriate places to ignore links. update defs to the consistent grib_wafs ecf tasks remove duplicate jgfs_atmos_wafs_f*.ecf files and rename f00 as master remove duplicate jgfs_atmos_awips_g2_f*.ecf files and rename f000 as master remove duplicate jgfs_atmos_awips_f*.ecf files and rename f000 as master remove duplicate gfs_atmos_post_fxxx.ecf files and rename f000 as master fix typo that causes the opposite effect ignore gdas/atmos/post/ forecast hour ecf links remove duplicate gdas_atmos_post_fxxx.ecf files and rename f000 as master remove remnant from PR555 that copied gdas/enkf to enkfgdas. add gitignore in enkfgdas/post to ignore links remove duplicate enkfgdas_post_fxxx.ecf files and rename f003 as master request exclusive node where ncpus=128 remove memory requests of 500gb and request exclusive node instead every ecf script that loads compiler dependent module, now loads PrgEnv-intel, craype and intel every ecf script that loads compiler dependent module, now loads PrgEnv-intel, craype and intel every ecf script that loads compiler dependent module, now loads PrgEnv-intel, craype and intel every ecf script that loads cray-mpich, now loads PrgEnv-intel, craype and intel. Ignore swp files Correct COMIN paths in GEMPAK driver scripts Update COMIN paths for ukmet, ecmwf, and nam Update WAFS tag to gfs_wafs.v6.2.7 add #PBS -l debug=true to all .ecf files Correct COMIN definitions Update GLDAS and WAFS tags in release notes Update GLDAS tag to gldas_gfsv16_release.v1.24.0 revert ecflow include files to NCO versions. Will adapt as necessary for proper use Remove NCO if-block from JJOB scripts Update ROTDIR in config.base.nco.static Remove ecflow post assignment in envir-p1.h Remove remark from envir-p1.h and head.h Update analysis ecflow script to use 128 for wcoss2 Remove extra CDATE Reference to NCO version: - Move enkf out of gdas and rename it to enkfgdas. Include all ecflow definition files job name Include all ecflow scripts name and job/log name - Move "model=gfs" to the top on each job except all jobs under obsproc. obsproc will no longer be part of GFS. Therefore leave it without change for testing purpose. - Remove the source of model_ver from each ecflow script except all jobs under obsproc. obsproc will no longer be part of GFS. Therefore leave it without change for testing purpose. Update enkf structure changes in ecflow definition files Update WAFS tag to gfs_wafs.v6.2.6 Remove envvar from module-setup.*.inc scripts Remove envvar from WCOSS2 driver scripts Update machine-setup based on NCO feedback ...

GFSv16.2.1 #899

GFSv16.2.1 #899

Commits on Jan 12, 2022

Commits on Jan 13, 2022

Commits on Jan 15, 2022

Commits on Jan 16, 2022

Commits on Jan 18, 2022

Commits on Jan 19, 2022

Commits on Jan 20, 2022

Commits on Jan 21, 2022

Commits on Jan 25, 2022

Commits on Jan 26, 2022

Commits on Jan 28, 2022

Commits on Jan 31, 2022

Commits on Feb 1, 2022

Commits on Feb 3, 2022

Commits on Feb 4, 2022

Commits on Feb 7, 2022

Commits on Feb 8, 2022

Commits on Feb 9, 2022

Commits on Feb 18, 2022

Commits on Feb 22, 2022

Commits on Feb 23, 2022

Commits on Mar 15, 2022

Commits on Mar 16, 2022

Commits on Mar 22, 2022

Commits on Mar 24, 2022

Commits on Mar 28, 2022

Commits on Mar 29, 2022

Commits on Mar 30, 2022

Commits on Apr 7, 2022

Commits on Apr 11, 2022

Commits on Apr 12, 2022

Commits on Apr 13, 2022

Commits on Apr 25, 2022

Commits on Apr 28, 2022

Commits on May 13, 2022

Commits on May 16, 2022

Commits on May 19, 2022

Commits on May 20, 2022

Commits on Jun 24, 2022

Commits on Jun 25, 2022

Commits on Jun 27, 2022

Commits on Jun 28, 2022

Commits on Jul 7, 2022