Skip to content

Commit

Permalink
[production/AQM.v7] AQM ecflow workflow EE2 review (ufs-community#952)
Browse files Browse the repository at this point in the history
* Restore ecflow workflow in AQM package

* Management authorized to remove hardwired workflow generation solution from developer on 10/17/2023.
Remove unnecessary ecflow template directory.
Fix issue where it hardwired job in ecflow workflow.
Remove incorrect configuration in ecflow script.
Move configuration directory into acceptable NCO code delivery location.

* Fixed bug for bias_correction and post_stat jobs
Fixed DBNALERT bug
Fixed wrong DO_AQM_SAVE_AIRNOW_HIST setting for NCO operation job
Fixed issue found in var_defns.sh where some parameter failed to export
Remove wrong parm/ecflow directory

* AQM ecflow workflow related change

* The current AQM implementation package has the architecture design that some jobs access DATA (stmp) from previous jobs.
It was communicated to the management that this design may not be acceptable to NCO.
However, ecflow workflow development is based on the current implementation package. The following change is needed to run jobs.
Modify aqm_cycled.def to use NCO production job naming format.
Modify aqm_cycled.def to use time based trigger requested from the AQM project manager.
Configure log file name for job nexus_gfs_sfc.

EX-script for Job exregional_make_ics.sh exregional_make_lbcs.sh exregional_nexus_emission.sh exregional_run_post.sh are designed with following issue:
1. Access the DATA (stmp) location from the previous job by hardwire wrong DATA directory name in ex-script.
2. No exception handling if the DATA directory is not found in nco production mode.
These four files have been fixed here by updating DATA location with ecflow job name and exception handling to fail the job if DATA is not found.

* The current AQM implementation package is designed for some jobs to access input files from other jobs' DATA (stmp) location.
These DATA directories can not be removed after current job finish with normal KEEPDATA=YES setting in the NCO production system.
These DATA directory names are hardwired with names that are not unique.
This solution is likely unacceptable to NCO production standards. This fact is communicated to the management.
However, ecflow workflow needs to be based on the current implementation package.
The data_cleanup job is designed to cleanup the following hardwired DATA directories:

* EE2 review for ecflow workflow

* parm/config is removed

* Add package configuration template files in parm/config

* Document configuration setup script
  • Loading branch information
lgannoaa committed Oct 24, 2023
1 parent d6d287a commit 015106f
Show file tree
Hide file tree
Showing 48 changed files with 2,082 additions and 1,037 deletions.
6 changes: 4 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
bin/
ecf/
exec/
fix/
sorc/*/
fix
ecf-EMC/
tests/WE2E/WE2E_tests_*.yaml
tests/WE2E/*.txt
tests/WE2E/*.log
Expand All @@ -11,6 +12,7 @@ ush/__pycache__/
ush/config.yaml
ush/python_utils/__pycache__/
ush/*.swp

devbuild.log
*TOBEDELETED
*.swp
__pycache__
Loading

0 comments on commit 015106f

Please sign in to comment.