Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[develop] Add plotting scripts to the workflow. #482

Merged
merged 21 commits into from
Dec 2, 2022
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
4a9314f
Add level for data: section indicating ics_lbcs.
danielabdi-noaa Nov 16, 2022
08470e6
Add the NaturalEarth shape files.
danielabdi-noaa Nov 16, 2022
3e3a6ec
Add plotting to the workflow.
danielabdi-noaa Nov 16, 2022
77967c8
Fix some deprecation warnings.
danielabdi-noaa Nov 16, 2022
bbf56ad
Don't plot 500mb vars if it is not available.
danielabdi-noaa Nov 16, 2022
3cbb2ba
Add modulefiles for plot_allvars task.
danielabdi-noaa Nov 16, 2022
aee4cd2
Add a test case for plotting graphics.
danielabdi-noaa Nov 16, 2022
5b7a6a1
Add diff plotting.
danielabdi-noaa Nov 16, 2022
95503e9
Remove ush/Python folder.
danielabdi-noaa Nov 16, 2022
9aa6a66
Turn off plotting by default.
danielabdi-noaa Nov 16, 2022
817a257
Bug fix plot_diff call.
danielabdi-noaa Nov 16, 2022
ca4ec44
Add missing diff plot script.
danielabdi-noaa Nov 16, 2022
ec73c8c
Make plotting work in NCO mode.
danielabdi-noaa Nov 16, 2022
7af2cae
Avoid positional arguments in the plotting scripts.
danielabdi-noaa Nov 16, 2022
cc5a45c
Make COMOUT_REF a template to compare multiple dates and cycles corre…
danielabdi-noaa Nov 17, 2022
98a3445
Use logging to capture output from plot scripts in log files.
danielabdi-noaa Nov 17, 2022
17d9369
Turn off debug level, too much information.
danielabdi-noaa Nov 17, 2022
d4b3ab7
Ignore warnings in plotting.
danielabdi-noaa Nov 17, 2022
68e3cc9
Bug fix diff plotter: mismatch in number of ticks, and ticklabels.
danielabdi-noaa Nov 17, 2022
eaa3b30
Increase wallclock time, bugfix orion FIXshp.
danielabdi-noaa Nov 17, 2022
65f6bac
Add some default values to config.community/nco for convenience.
danielabdi-noaa Dec 2, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
112 changes: 112 additions & 0 deletions jobs/JREGIONAL_PLOT_ALLVARS
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
#!/bin/bash

#
#-----------------------------------------------------------------------
#
# Source the variable definitions file and the bash utility functions.
#
#-----------------------------------------------------------------------
#
. $USHdir/source_util_funcs.sh
source_config_for_task "task_plot_allvars|task_run_fcst" ${GLOBAL_VAR_DEFNS_FP}
. $USHdir/job_preamble.sh
#
#-----------------------------------------------------------------------
#
# Save current shell options (in a global array). Then set new options
# for this script/function.
#
#-----------------------------------------------------------------------
#
{ save_shell_opts; . $USHdir/preamble.sh; } > /dev/null 2>&1
#
#-----------------------------------------------------------------------
#
# Get the full path to the file in which this script/function is located
# (scrfunc_fp), the name of that file (scrfunc_fn), and the directory in
# which the file is located (scrfunc_dir).
#
#-----------------------------------------------------------------------
#
scrfunc_fp=$( $READLINK -f "${BASH_SOURCE[0]}" )
scrfunc_fn=$( basename "${scrfunc_fp}" )
scrfunc_dir=$( dirname "${scrfunc_fp}" )
#
#-----------------------------------------------------------------------
#
# Print message indicating entry into script.
#
#-----------------------------------------------------------------------
#
print_info_msg "
========================================================================
Entering script: \"${scrfunc_fn}\"
In directory: \"${scrfunc_dir}\"

This is the J-job script for the plotting task
========================================================================"
#
#-----------------------------------------------------------------------
#
# Set grid name and COMOUT locations.
#
#-----------------------------------------------------------------------
#
GRID_NAME=${PREDEF_GRID_NAME:-$EXPT_SUBDIR}
if [ "${RUN_ENVIR}" != "nco" ]; then
export COMOUT="${COMOUT}${SLASH_ENSMEM_SUBDIR}/postprd"
fi
# if COMOUT_REF is templatized with $PDY and $cyc, evaluate it
# This is necessary to compare multiple cycles correctly
COMOUT_REF=$(eval echo ${COMOUT_REF})
#
#-----------------------------------------------------------------------
#
# Call the ex-script for this J-job and pass to it the necessary variables.
#
#-----------------------------------------------------------------------
#

# plot all variables
$SCRIPTSdir/exregional_plot_allvars.py \
--cycle ${CDATE} \
--start ${PLOT_FCST_START} \
--end ${PLOT_FCST_END:-$FCST_LEN_HRS} \
--inc ${PLOT_FCST_INC:-1} \
--comout ${COMOUT} \
--cartopy-dir ${FIXshp} \
--domain ${GRID_NAME} || \
print_err_msg_exit "\
Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed."

# plot all variables diff with baseline
if [ ! -z $COMOUT_REF ]; then
$SCRIPTSdir/exregional_plot_allvars_diff.py \
--cycle ${CDATE} \
--start ${PLOT_FCST_START} \
--end ${PLOT_FCST_END:-$FCST_LEN_HRS} \
--inc ${PLOT_FCST_INC:-1} \
--comout-1 ${COMOUT} \
--comout-2 ${COMOUT_REF} \
--cartopy-dir ${FIXshp} \
--domain ${GRID_NAME} || \
print_err_msg_exit "\
Call to ex-script corresponding to J-job \"${scrfunc_fn}\" failed."
fi
#
#-----------------------------------------------------------------------
#
# Run job postamble.
#
#-----------------------------------------------------------------------
#
job_postamble
#
#-----------------------------------------------------------------------
#
# Restore the shell options saved at the beginning of this script/function.
#
#-----------------------------------------------------------------------
#
{ restore_shell_opts; } > /dev/null 2>&1

1 change: 1 addition & 0 deletions modulefiles/tasks/cheyenne/plot_allvars.local.lua
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
load("miniconda_regional_workflow")
1 change: 1 addition & 0 deletions modulefiles/tasks/gaea/plot_allvars.local.lua
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
load("miniconda_regional_workflow")
1 change: 1 addition & 0 deletions modulefiles/tasks/hera/plot_allvars.local.lua
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
load("miniconda_regional_workflow")
1 change: 1 addition & 0 deletions modulefiles/tasks/jet/plot_allvars.local.lua
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
load("miniconda_regional_workflow")
1 change: 1 addition & 0 deletions modulefiles/tasks/noaacloud/plot_allvars.local.lua
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
load("miniconda_regional_workflow")
1 change: 1 addition & 0 deletions modulefiles/tasks/orion/plot_allvars.local.lua
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
load("miniconda_regional_workflow")
1 change: 1 addition & 0 deletions modulefiles/tasks/wcoss2/plot_allvars.local.lua
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
load("python_regional_workflow")
39 changes: 39 additions & 0 deletions parm/FV3LAM_wflow.xml
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ Workflow task names.
<!ENTITY VX_ENSPOINT_TN "{{ vx_enspoint_tn }}">
<!ENTITY VX_ENSPOINT_MEAN_TN "{{ vx_enspoint_mean_tn }}">
<!ENTITY VX_ENSPOINT_PROB_TN "{{ vx_enspoint_prob_tn }}">
<!ENTITY PLOT_ALLVARS_TN "{{ plot_allvars_tn }}">

<!--
Flags that specify whether to run the preprocessing and/or verification tasks.
Expand Down Expand Up @@ -755,6 +756,44 @@ the <task> tag to be identical to the ones above for other output times.
{%- endif %}
{%- endif %}

{%- if run_task_plot_allvars %}
<!--
************************************************************************
************************************************************************
-->
<task name="&PLOT_ALLVARS_TN;" cycledefs="forecast" maxtries="{{ maxtries_plot_allvars }}">

&RSRV_DEFAULT;
<command>&LOAD_MODULES_RUN_TASK_FP; "&PLOT_ALLVARS_TN;" "&JOBSdir;/JREGIONAL_PLOT_ALLVARS"</command>
<nodes>{{ nnodes_plot_allvars }}:ppn={{ ppn_plot_allvars }}</nodes>
<walltime>{{ wtime_plot_allvars }}</walltime>
<nodesize>&NCORES_PER_NODE;</nodesize>
{%- if machine not in ["WCOSS2"] %}
<native>&SCHED_NATIVE_CMD;</native>
{%- endif %}
<jobname>&PLOT_ALLVARS_TN;</jobname>
<join>&LOGDIR;/&PLOT_ALLVARS_TN;<cyclestr>_@Y@m@d@H</cyclestr>&LOGEXT;</join>

<envar><name>GLOBAL_VAR_DEFNS_FP</name><value>&GLOBAL_VAR_DEFNS_FP;</value></envar>
<envar><name>USHdir</name><value>&USHdir;</value></envar>
<envar><name>PDY</name><value><cyclestr>@Y@m@d</cyclestr></value></envar>
<envar><name>cyc</name><value><cyclestr>@H</cyclestr></value></envar>
<envar><name>subcyc</name><value><cyclestr>@M</cyclestr></value></envar>
<envar><name>LOGDIR</name><value>&LOGDIR;</value></envar>
<envar><name>SLASH_ENSMEM_SUBDIR</name><value><cyclestr>{{ slash_ensmem_subdir }}</cyclestr></value></envar>
<envar><name>ENSMEM_INDX</name><value><cyclestr>#{{ ensmem_indx_name }}#</cyclestr></value></envar>

<dependency>
{%- if write_dopost %}
<taskdep task="&RUN_FCST_TN;{{ uscore_ensmem_name }}"/>
{%- else %}
<metataskdep metatask="&RUN_POST_TN;{{ uscore_ensmem_name }}"/>
{%- endif %}
</dependency>

</task>
{%- endif %}

{%- if run_task_get_obs_ccpa %}
<!--
************************************************************************
Expand Down
Loading