Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/updates for gdas app #1091

Merged
merged 18 commits into from
Oct 25, 2022
Merged

Feature/updates for gdas app #1091

Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
1a1004f
Various updates to restore ability to cycle GDASApp (see #1067)
RussTreadon-NOAA Oct 17, 2022
d71af26
Merge branch 'develop' into feature/updates_for_GDASApp
RussTreadon-NOAA Oct 19, 2022
bee4d87
Add missing argument, atmges_fv3, to INCPY (#1067)
RussTreadon-NOAA Oct 19, 2022
bb0cf88
Merge branch 'develop' into feature/updates_for_GDASApp
RussTreadon-NOAA Oct 20, 2022
ab6dec0
Update hash for UFS-based DA checkout, correct typo in load_ufsda_mod…
RussTreadon-NOAA Oct 20, 2022
8b155de
add FV3_GFS_v16_no_nsst to CCPP_SUITES for UFS-based DA (#1067)
RussTreadon-NOAA Oct 20, 2022
5a97bda
Use identity B for fv3jedi_var, set OMP_NUM_THREADS in fv3jedi analys…
RussTreadon-NOAA Oct 20, 2022
670602b
Revert R2D2_BC_DUMP in config.atmensanal back to original default (#1…
RussTreadon-NOAA Oct 21, 2022
b7127fd
Merge branch 'develop' into feature/updates_for_GDASApp (#1067)
RussTreadon-NOAA Oct 22, 2022
f2672be
Merge branch 'develop' into feature/updates_for_GDASApp (#1067)
RussTreadon-NOAA Oct 24, 2022
8764a0d
Merge branch 'develop' into feature/updates_for_GDASApp (#1067)
RussTreadon-NOAA Oct 24, 2022
a3d60d4
Update GDASApp hash, address linter warnings for load_ufsda_module.sh…
RussTreadon-NOAA Oct 24, 2022
1997c26
address linter errors in ufsda analysis run scripts (#1067)
RussTreadon-NOAA Oct 24, 2022
508bf32
Merge branch 'develop' into feature/updates_for_GDASApp (#1067)
RussTreadon-NOAA Oct 24, 2022
f82fab4
refactor correction to linter errors in ufsda analysis run scripts (#…
RussTreadon-NOAA Oct 24, 2022
ed2a98c
Update GDASAapp hash in checkout.sh (#1067)
RussTreadon-NOAA Oct 25, 2022
3c452dd
Update gsi-enkf and GDASApps hashes in Externals.cfg (#1067)
RussTreadon-NOAA Oct 25, 2022
5b576f0
Merge branch 'develop' into feature/updates_for_GDASApp
RussTreadon-NOAA Oct 25, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 2 additions & 9 deletions jobs/JGDAS_GLOBAL_ATMOS_ANALYSIS_POST
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,20 +79,13 @@ mkdir -p $COMOUT
export COMIN_GES="$ROTDIR/$GDUMP.$gPDY/$gcyc/$COMPONENT"
export COMIN_GES_ENS="$ROTDIR/enkf$GDUMP.$gPDY/$gcyc/$COMPONENT"

# NOTE BELOW IS A HACK FOR TESTING
# PLEASE FIX THIS LATER
# ASK @aerorahul
# HOW TO HANDLE DIFFERENT COMPILERS/ETC. FOR MODEL VS DA
# PROD_UTIL, ETC. DO NOT EXIST FOR JEDI MODULE VERSIONS
module purge
module use $HOMEgfs/sorc/gdas.cd/modulefiles
module load GDAS/orion
# Add UFSDA to PYTHONPATH
export PYTHONPATH=$HOMEgfs/sorc/gdas.cd/ush/:$PYTHONPATH

###############################################################
# Run relevant script

EXSCRIPT=${GDASPOSTPY:-$HOMEgfs/sorc/gdas.cd/scripts/exgdas_global_atmos_analysis_post.py}
EXSCRIPT=${GDASPOSTPY:-$HOMEgfs/scripts/exgdas_global_atmos_analysis_post.py}
$EXSCRIPT
status=$?
[[ $status -ne 0 ]] && exit $status
Expand Down
12 changes: 2 additions & 10 deletions jobs/JGDAS_GLOBAL_ATMOS_ANALYSIS_PREP
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,20 +79,12 @@ mkdir -p $COMOUT
export COMIN_GES="$ROTDIR/$GDUMP.$gPDY/$gcyc/$COMPONENT"
export COMIN_GES_ENS="$ROTDIR/enkf$GDUMP.$gPDY/$gcyc/$COMPONENT"

# NOTE BELOW IS A HACK FOR TESTING
# PLEASE FIX THIS LATER
# ASK @aerorahul
# HOW TO HANDLE DIFFERENT COMPILERS/ETC. FOR MODEL VS DA
# PROD_UTIL, ETC. DO NOT EXIST FOR JEDI MODULE VERSIONS
module purge
module use $HOMEgfs/sorc/gdas.cd/modulefiles
module load GDAS/orion
# Add UFSDA to PYTHONPATH
export PYTHONPATH=$HOMEgfs/sorc/gdas.cd/ush/:$PYTHONPATH

###############################################################
# Run relevant script

EXSCRIPT=${GDASPREPPY:-$HOMEgfs/sorc/gdas.cd/scripts/exgdas_global_atmos_analysis_prep.py}
EXSCRIPT=${GDASPREPPY:-$HOMEgfs/scripts/exgdas_global_atmos_analysis_prep.py}
$EXSCRIPT
status=$?
[[ $status -ne 0 ]] && exit $status
Expand Down
11 changes: 2 additions & 9 deletions jobs/JGDAS_GLOBAL_ATMOS_ANALYSIS_RUN
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,20 +79,13 @@ mkdir -p $COMOUT
export COMIN_GES="$ROTDIR/$GDUMP.$gPDY/$gcyc/$COMPONENT"
export COMIN_GES_ENS="$ROTDIR/enkf$GDUMP.$gPDY/$gcyc/$COMPONENT"

# NOTE BELOW IS A HACK FOR TESTING
# PLEASE FIX THIS LATER
# ASK @aerorahul
# HOW TO HANDLE DIFFERENT COMPILERS/ETC. FOR MODEL VS DA
# PROD_UTIL, ETC. DO NOT EXIST FOR JEDI MODULE VERSIONS
module purge
module use $HOMEgfs/sorc/gdas.cd/modulefiles
module load GDAS/orion
# Add UFSDA to PYTHONPATH
export PYTHONPATH=$HOMEgfs/sorc/gdas.cd/ush/:$PYTHONPATH

###############################################################
# Run relevant script

EXSCRIPT=${GDASRUNSH:-$HOMEgfs/sorc/gdas.cd/scripts/exgdas_global_atmos_analysis_run.sh}
EXSCRIPT=${GDASRUNSH:-$HOMEgfs/scripts/exgdas_global_atmos_analysis_run.sh}
$EXSCRIPT
status=$?
[[ $status -ne 0 ]] && exit $status
Expand Down
11 changes: 2 additions & 9 deletions jobs/JGDAS_GLOBAL_ATMOS_ENSANAL_POST
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,20 +82,13 @@ export COMIN="$ROTDIR/$CDUMP.$PDY/$cyc/$COMPONENT"
export COMIN_GES="$ROTDIR/$GDUMP.$gPDY/$gcyc/$COMPONENT"
export COMIN_GES_ENS="$ROTDIR/enkf$GDUMP.$gPDY/$gcyc/$COMPONENT"

# NOTE BELOW IS A HACK FOR TESTING
# PLEASE FIX THIS LATER
# ASK @aerorahul
# HOW TO HANDLE DIFFERENT COMPILERS/ETC. FOR MODEL VS DA
# PROD_UTIL, ETC. DO NOT EXIST FOR JEDI MODULE VERSIONS
module purge
module use $HOMEgfs/sorc/gdas.cd/modulefiles
module load GDAS/orion
# Add UFSDA to PYTHONPATH
export PYTHONPATH=$HOMEgfs/sorc/gdas.cd/ush/:$PYTHONPATH

###############################################################
# Run relevant script

EXSCRIPT=${GDASPOSTPY:-$HOMEgfs/sorc/gdas.cd/scripts/exgdas_global_atmos_ensanal_post.py}
EXSCRIPT=${GDASPOSTPY:-$HOMEgfs/scripts/exgdas_global_atmos_ensanal_post.py}
$EXSCRIPT
status=$?
[[ $status -ne 0 ]] && exit $status
Expand Down
11 changes: 2 additions & 9 deletions jobs/JGDAS_GLOBAL_ATMOS_ENSANAL_PREP
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,20 +82,13 @@ export COMIN="$ROTDIR/$CDUMP.$PDY/$cyc/$COMPONENT"
export COMIN_GES="$ROTDIR/$GDUMP.$gPDY/$gcyc/$COMPONENT"
export COMIN_GES_ENS="$ROTDIR/enkf$GDUMP.$gPDY/$gcyc/$COMPONENT"

# NOTE BELOW IS A HACK FOR TESTING
# PLEASE FIX THIS LATER
# ASK @aerorahul
# HOW TO HANDLE DIFFERENT COMPILERS/ETC. FOR MODEL VS DA
# PROD_UTIL, ETC. DO NOT EXIST FOR JEDI MODULE VERSIONS
module purge
module use $HOMEgfs/sorc/gdas.cd/modulefiles
module load GDAS/orion
# Add UFSDA to PYTHONPATH
export PYTHONPATH=$HOMEgfs/sorc/gdas.cd/ush/:$PYTHONPATH

###############################################################
# Run relevant script

EXSCRIPT=${GDASPREPPY:-$HOMEgfs/sorc/gdas.cd/scripts/exgdas_global_atmos_analysis_prep.py}
EXSCRIPT=${GDASPREPPY:-$HOMEgfs/scripts/exgdas_global_atmos_analysis_prep.py}
$EXSCRIPT
status=$?
[[ $status -ne 0 ]] && exit $status
Expand Down
11 changes: 2 additions & 9 deletions jobs/JGDAS_GLOBAL_ATMOS_ENSANAL_RUN
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,20 +82,13 @@ export COMIN="$ROTDIR/$CDUMP.$PDY/$cyc/$COMPONENT"
export COMIN_GES="$ROTDIR/$GDUMP.$gPDY/$gcyc/$COMPONENT"
export COMIN_GES_ENS="$ROTDIR/enkf$GDUMP.$gPDY/$gcyc/$COMPONENT"

# NOTE BELOW IS A HACK FOR TESTING
# PLEASE FIX THIS LATER
# ASK @aerorahul
# HOW TO HANDLE DIFFERENT COMPILERS/ETC. FOR MODEL VS DA
# PROD_UTIL, ETC. DO NOT EXIST FOR JEDI MODULE VERSIONS
module purge
module use $HOMEgfs/sorc/gdas.cd/modulefiles
module load GDAS/orion
# Add UFSDA to PYTHONPATH
export PYTHONPATH=$HOMEgfs/sorc/gdas.cd/ush/:$PYTHONPATH

###############################################################
# Run relevant script

EXSCRIPT=${GDASRUNSH:-$HOMEgfs/sorc/gdas.cd/scripts/exgdas_global_atmos_ensanal_run.sh}
EXSCRIPT=${GDASRUNSH:-$HOMEgfs/scripts/exgdas_global_atmos_ensanal_run.sh}
$EXSCRIPT
status=$?
[[ $status -ne 0 ]] && exit $status
Expand Down
6 changes: 4 additions & 2 deletions jobs/rocoto/atmanalpost.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
#! /usr/bin/env bash

export STRICT="NO"
WalterKolczynski-NOAA marked this conversation as resolved.
Show resolved Hide resolved
source "$HOMEgfs/ush/preamble.sh"
export STRICT="YES"

###############################################################
# Source FV3GFS workflow modules
. $HOMEgfs/ush/load_fv3gfs_modules.sh
# Source UFSDA workflow modules
. $HOMEgfs/ush/load_ufsda_modules.sh
status=$?
[[ $status -ne 0 ]] && exit $status

Expand Down
6 changes: 4 additions & 2 deletions jobs/rocoto/atmanalprep.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
#! /usr/bin/env bash

export STRICT="NO"
source "$HOMEgfs/ush/preamble.sh"
export STRICT="YES"

###############################################################
# Source FV3GFS workflow modules
. $HOMEgfs/ush/load_fv3gfs_modules.sh
# Source UFSDA workflow modules
. $HOMEgfs/ush/load_ufsda_modules.sh
status=$?
[[ $status -ne 0 ]] && exit $status

Expand Down
6 changes: 4 additions & 2 deletions jobs/rocoto/atmanalrun.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
#! /usr/bin/env bash

export STRICT="NO"
source "$HOMEgfs/ush/preamble.sh"
export STRICT="YES"

###############################################################
# Source FV3GFS workflow modules
. $HOMEgfs/ush/load_fv3gfs_modules.sh
# Source UFSDA workflow modules
. $HOMEgfs/ush/load_ufsda_modules.sh
status=$?
[[ $status -ne 0 ]] && exit $status

Expand Down
6 changes: 4 additions & 2 deletions jobs/rocoto/atmensanalpost.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
#! /usr/bin/env bash

export STRICT="NO"
source "$HOMEgfs/ush/preamble.sh"
export STRICT="YES"

###############################################################
# Source FV3GFS workflow modules
. $HOMEgfs/ush/load_fv3gfs_modules.sh
# Source UFSDA workflow modules
. $HOMEgfs/ush/load_ufsda_modules.sh
status=$?
[[ $status -ne 0 ]] && exit $status

Expand Down
6 changes: 4 additions & 2 deletions jobs/rocoto/atmensanalprep.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
#! /usr/bin/env bash

export STRICT="NO"
source "$HOMEgfs/ush/preamble.sh"
export STRICT="YES"

###############################################################
# Source FV3GFS workflow modules
. $HOMEgfs/ush/load_fv3gfs_modules.sh
# Source UFSDA workflow modules
. $HOMEgfs/ush/load_ufsda_modules.sh
status=$?
[[ $status -ne 0 ]] && exit $status

Expand Down
6 changes: 4 additions & 2 deletions jobs/rocoto/atmensanalrun.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
#! /usr/bin/env bash

export STRICT="NO"
source "$HOMEgfs/ush/preamble.sh"
export STRICT="YES"

###############################################################
# Source FV3GFS workflow modules
. $HOMEgfs/ush/load_fv3gfs_modules.sh
# Source UFSDA workflow modules
. $HOMEgfs/ush/load_ufsda_modules.sh
status=$?
[[ $status -ne 0 ]] && exit $status

Expand Down
2 changes: 1 addition & 1 deletion parm/config/config.atmanal
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ echo "BEGIN: config.atmanal"
export OBS_YAML_DIR=$HOMEgfs/sorc/gdas.cd/parm/atm/obs/config/
export OBS_LIST=$HOMEgfs/sorc/gdas.cd/parm/atm/obs/lists/gdas_prototype.yaml
export ATMVARYAML=$HOMEgfs/sorc/gdas.cd/parm/atm/variational/3dvar_dripcg.yaml
export BERROR_YAML=$HOMEgfs/sorc/gdas.cd/parm/atm/berror/hybvar_bump.yaml
export BERROR_YAML=$HOMEgfs/sorc/gdas.cd/parm/atm/berror/staticb_identity.yaml
export FV3JEDI_FIX=$HOMEgfs/fix/gdas
export R2D2_OBS_DB='ufsda_test'
export R2D2_OBS_DUMP='oper_gdas'
Expand Down
3 changes: 1 addition & 2 deletions parm/config/config.atmensanal
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,7 @@ export R2D2_OBS_DB='ufsda_test'
export R2D2_OBS_DUMP='oper_gdas'
export R2D2_OBS_SRC='ncdiag'
export R2D2_BC_SRC='gsi'
##export R2D2_BC_DUMP='oper_gdas'
export R2D2_BC_DUMP='prgdasens'
export R2D2_BC_DUMP='oper_gdas'
export R2D2_ARCH_DB='local'
export INTERP_METHOD='barycentric'

Expand Down
8 changes: 5 additions & 3 deletions scripts/exgdas_global_atmos_analysis_run.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,17 +131,19 @@ $NLN $JEDIVAREXE $DATA/fv3jedi_var.x

################################################################################
# run executable
export OMP_NUM_THREADS=$NTHREADS_ATMANAL
export pgm=$JEDIVAREXE
. prep_step
$APRUN_ATMANAL $DATA/fv3jedi_var.x $DATA/fv3jedi_var.yaml 1>&1 2>&2
export err=$?; err_chk

################################################################################
# translate FV3-JEDI increment to FV3 readable format
atmges_fv3=$COMIN_GES/${GPREFIX}atmf006.ensres.nc
atminc_jedi=$DATA/anl/atminc.${PDY}_${cyc}0000z.nc4
atminc_fv3=$COMOUT/${CDUMP}.${cycle}.atminc.nc
if [ -s $atminc_jedi ]; then
$INCPY $atminc_jedi $atminc_fv3
$INCPY $atmges_fv3 $atminc_jedi $atminc_fv3
export err=$?
else
echo "***WARNING*** missing $atminc_jedi ABORT"
Expand All @@ -164,11 +166,11 @@ cp -rf $DATA/bc $COMOUT/
# Deterministic abias used in enkf cycle
alist="abias abias_air abias_int abias_pc"
for abias in $alist; do
cp $COMIN_GES/${GPREFIX}${abias} $COMOUT/${APREFIX}${abias}
cp "${COMIN_GES}/${GPREFIX}${abias}" "${COMOUT}/${APREFIX}${abias}"
done

################################################################################

exit $err
exit ${err}

################################################################################
12 changes: 7 additions & 5 deletions scripts/exgdas_global_atmos_ensanal_run.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,8 @@ $NLN $JEDIENSEXE $DATA/fv3jedi_ens.x

################################################################################
# run executable
export pgm=$JEDIVAREXE
export OMP_NUM_THREADS=$NTHREADS_ATMENSANAL
export pgm=$JEDIENSEXE
. prep_step
$APRUN_ATMENSANAL $DATA/fv3jedi_ens.x $DATA/fv3jedi_ens.yaml 1>&1 2>&2
export err=$?; err_chk
Expand All @@ -135,11 +136,12 @@ export err=$?; err_chk
# translate FV3-JEDI increment to FV3 readable format
for imem in $(seq 1 $NMEM_ENKF); do
memchar="mem"$(printf %03i $imem)
atmges_fv3=$COMIN_GES_ENS/$memchar/${GPREFIX}atmf006.nc
atminc_jedi=$DATA/anl/$memchar/atminc.${PDY}_${cyc}0000z.nc4
atminc_fv3=$COMOUT_ENS/$memchar/${CDUMP}.${cycle}.atminc.nc
mkdir -p $COMOUT_ENS/$memchar
if [ -s $atminc_jedi ]; then
$INCPY $atminc_jedi $atminc_fv3
$INCPY $atmges_fv3 $atminc_jedi $atminc_fv3
export err=$?
else
echo "***WARNING*** missing $atminc_jedi ABORT"
Expand All @@ -154,12 +156,12 @@ echo "$CDUMP $CDATE atminc done at $(date)" > $COMOUT_ENS/${CDUMP}.${cycle}.logi

################################################################################
# Copy diags and YAML to $COMOUT
cp -r $DATA/fv3jedi_ens.yaml $COMOUT_ENS/${CDUMP}.${cycle}.fv3jedi_ens.yaml
cp -rf $DATA/diags $COMOUT_ENS/
cp -r ${DATA}/fv3jedi_ens.yaml ${COMOUT_ENS}/${CDUMP}.${cycle}.fv3jedi_ens.yaml
cp -rf "${DATA}/diags" "${COMOUT_ENS}/"


################################################################################

exit $err
exit ${err}

################################################################################
2 changes: 1 addition & 1 deletion sorc/build_ufs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ cwd=$(pwd)

# Default settings
APP="S2SWA"
CCPP_SUITES="FV3_GFS_v16,FV3_GFS_v16_ugwpv1,FV3_GFS_v17_p8,FV3_GFS_v16_coupled_nsstNoahmpUGWPv1,FV3_GFS_v17_coupled_p8"
CCPP_SUITES="FV3_GFS_v16,FV3_GFS_v16_no_nsst,FV3_GFS_v16_ugwpv1,FV3_GFS_v17_p8,FV3_GFS_v16_coupled_nsstNoahmpUGWPv1,FV3_GFS_v17_coupled_p8"

export RT_COMPILER="intel"
source $cwd/ufs_model.fd/tests/detect_machine.sh
Expand Down
4 changes: 2 additions & 2 deletions sorc/checkout.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,11 +157,11 @@ checkout "ufs_utils.fd" "https://github.com/ufs-community/UFS_UTILS.git"
checkout "verif-global.fd" "https://github.com/NOAA-EMC/EMC_verif-global.git" "c267780" ; errs=$((errs + $?))

if [[ ${checkout_gsi} == "YES" ]]; then
checkout "gsi_enkf.fd" "https://github.com/NOAA-EMC/GSI.git" "67f5ab4"; errs=$((errs + $?))
checkout "gsi_enkf.fd" "https://github.com/NOAA-EMC/GSI.git" "48d8676"; errs=$((errs + $?))
fi

if [[ ${checkout_gdas} == "YES" ]]; then
checkout "gdas.cd" "https://github.com/NOAA-EMC/GDASApp.git" "5952c9d"; errs=$((errs + $?))
checkout "gdas.cd" "https://github.com/NOAA-EMC/GDASApp.git" "9efabe5"; errs=$((errs + $?))
fi

if [[ ${checkout_gsi} == "YES" || ${checkout_gdas} == "YES" ]]; then
Expand Down
Loading