Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unify checkout and build systems #374

Closed
WalterKolczynski-NOAA opened this issue Jul 26, 2021 · 0 comments · Fixed by #375
Closed

Unify checkout and build systems #374

WalterKolczynski-NOAA opened this issue Jul 26, 2021 · 0 comments · Fixed by #375
Assignees
@WalterKolczynski-NOAA

Comments

@WalterKolczynski-NOAA
Copy link
Contributor

The checkout and build systems for the standard GFS, the coupled version, and the aerosol version need to be unified around a single set of scripts.

See also: #366, #368
@WalterKolczynski-NOAA WalterKolczynski-NOAA self-assigned this Jul 26, 2021
WalterKolczynski-NOAA added a commit to WalterKolczynski-NOAA/global-workflow that referenced this issue Jul 26, 2021
@WalterKolczynski-NOAA
Unify build system
b4ee33a 
Combines the checkout and build from the standard and coupled workflows
system that checks out the current UFS.

The model source directory is now ufs_model.fd rather than fv3gfs.fd or
ufs_coupled.fd. The build script is renamed build_ufs.sh and the link
script is renamed to simply link.sh.

The checkout script checks out the same UFS version (current as of
2021 Jul 20) for all applications, removing the coupled option from the
coupled branch. A new option, -m, allows the user to specify a different
model hash to check out. Checkout log files are now placed in the logs
directory like the build logs in order to keep the sorc directory clean.

Two new options are added to the build_all script: -c to build the
coupled (S2SW) app, and -a to build the aerosol (ATMAERO) app. Without
either argument, the traditional (ATMW) app is built. If either option
is provided, the cpl_build.cfg file controls which executables are built;
otherwise fv3gfs_build.cfg is still used. The build_ufs script corre-
spondingly accepts the same options.

The reg2grb2 source directory is copied over from the coupled model and
cmake files for workflow_utils are updated correspondingly to build.
Additionally, the build_workflow_utils modulefiles are updated to use the
ncio module in hpc-stack and the hack that cloned and built it as part of
the workflow_utils build is removed.

The build_ww3prepost script is updated to make the code more easily main-
tained and the corresponding modulefiles are updated to remove unnecessary
modules.

The link script is updated to the new fix location (ending in fix_NEW)
removes links with targets that no longer exist. A new third argument to
link, coupled, is added to additionally link files needed by the coupled
system.

Note that while the checkout, build, and link have all been tested and
complete without error, the workflow is otherwise untested and may not
work until additional updates are made to use the current UFS.

Refs: NOAA-EMC#366, NOAA-EMC#368, NOAA-EMC#374
Merged
@WalterKolczynski-NOAA WalterKolczynski-NOAA linked a pull request Jul 26, 2021 that will close this issue
Unify build system #375
Merged
WalterKolczynski-NOAA added a commit to WalterKolczynski-NOAA/global-workflow that referenced this issue Jul 26, 2021
@WalterKolczynski-NOAA
Rename link script, update usage
6929179 
The link script is updated to something indicating it is in the
workflow.

Link script usage statement is updated to reflect its new name.

Refs: NOAA-EMC#366, NOAA-EMC#368, NOAA-EMC#374
WalterKolczynski-NOAA added a commit to WalterKolczynski-NOAA/global-workflow that referenced this issue Jul 26, 2021
@WalterKolczynski-NOAA
Fix exists message for UFS model in checkout script
84ece56 
The directory exists message for the UFS model erroneously stated
the ufs.fd directory exists instead of the ufs_model.fd directory
exists.

Refs: NOAA-EMC#366, NOAA-EMC#368, NOAA-EMC#374
WalterKolczynski-NOAA added a commit to WalterKolczynski-NOAA/global-workflow that referenced this issue Jul 26, 2021
@WalterKolczynski-NOAA
Use cmake directly in UFS build
7618aba 
The UFS build script now calls cmake directly instead of using the
build script located in UFS.

In the future, the build directory for all components will be moved
out of the cloned repos and into workflow.

Refs: NOAA-EMC#366, NOAA-EMC#368, NOAA-EMC#374
WalterKolczynski-NOAA added a commit to WalterKolczynski-NOAA/global-workflow that referenced this issue Jul 26, 2021
@WalterKolczynski-NOAA
Fix linked executables for WAFS
44bcb49 
The executables for WAFS have a .x suffix. The link script now uses
the correct executable names.

Refs: NOAA-EMC#366, NOAA-EMC#368, NOAA-EMC#374
WalterKolczynski-NOAA added a commit to WalterKolczynski-NOAA/global-workflow that referenced this issue Jul 29, 2021
@WalterKolczynski-NOAA
Reorganize some module loads
adb561c 
The order of module loads is reordered so HDF and NetCDF are loaded
before modules that use them.

Refs: NOAA-EMC#366, NOAA-EMC#374
WalterKolczynski-NOAA added a commit to WalterKolczynski-NOAA/global-workflow that referenced this issue Jul 29, 2021
@WalterKolczynski-NOAA
Quiet UFS build module load
6bf34cd 
Module loading in the UFS build script now sets +x first to turn off
the command trace.

Refs: NOAA-EMC#366, NOAA-EMC#368, NOAA-EMC#374
WalterKolczynski-NOAA added a commit to WalterKolczynski-NOAA/global-workflow that referenced this issue Jul 29, 2021
@WalterKolczynski-NOAA
Reorder HDF/NetCDF load order
5498ed9 
Moves the HDF and NetCDF module loads before all the NCEP libs.

Refs: NOAA-EMC#366, NOAA-EMC#374
WalterKolczynski-NOAA added a commit to WalterKolczynski-NOAA/global-workflow that referenced this issue Jul 29, 2021
@WalterKolczynski-NOAA
Quiet UFS build module purge
c77da47 
Moved the UFS build module purge inside the set +x block.

Refs: NOAA-EMC#366, NOAA-EMC#374
WalterKolczynski-NOAA added a commit to WalterKolczynski-NOAA/global-workflow that referenced this issue Jul 29, 2021
@WalterKolczynski-NOAA
Update UPP to 10.0.8
e9be461 
Updates UPP to 10.0.8 to be consistent with UFS. This also adds
consistency between the moduleload and checked out version.

Refs: NOAA-EMC#366, NOAA-EMC#374
WalterKolczynski-NOAA added a commit that referenced this issue Jul 29, 2021
@WalterKolczynski-NOAA @aerorahul
Unify build system (#375)
471d67e 
* Unify build system

Combines the checkout and build from the standard and coupled workflows
system that checks out the current UFS.

The model source directory is now ufs_model.fd rather than fv3gfs.fd or
ufs_coupled.fd. The build script is renamed build_ufs.sh and the link
script is renamed to simply link.sh.

The checkout script checks out the same UFS version (current as of
2021 Jul 20) for all applications, removing the coupled option from the
coupled branch. A new option, -m, allows the user to specify a different
model hash to check out. Checkout log files are now placed in the logs
directory like the build logs in order to keep the sorc directory clean.

Two new options are added to the build_all script: -c to build the
coupled (S2SW) app, and -a to build the aerosol (ATMAERO) app. Without
either argument, the traditional (ATMW) app is built. If either option
is provided, the cpl_build.cfg file controls which executables are built;
otherwise fv3gfs_build.cfg is still used. The build_ufs script corre-
spondingly accepts the same options.

The reg2grb2 source directory is copied over from the coupled model and
cmake files for workflow_utils are updated correspondingly to build.
Additionally, the build_workflow_utils modulefiles are updated to use the
ncio module in hpc-stack and the hack that cloned and built it as part of
the workflow_utils build is removed.

The build_ww3prepost script is updated to make the code more easily main-
tained and the corresponding modulefiles are updated to remove unnecessary
modules.

The link script is updated to the new fix location (ending in fix_NEW)
removes links with targets that no longer exist. A new third argument to
link, coupled, is added to additionally link files needed by the coupled
system.

Note that while the checkout, build, and link have all been tested and
complete without error, the workflow is otherwise untested and may not
work until additional updates are made to use the current UFS.

Refs: #366, #368, #374

* Rename link script, update usage

The link script is updated to something indicating it is in the
workflow.

Link script usage statement is updated to reflect its new name.

Refs: #366, #368, #374

* Fix exists message for UFS model in checkout script

The directory exists message for the UFS model erroneously stated
the ufs.fd directory exists instead of the ufs_model.fd directory
exists.

Refs: #366, #368, #374

* Use cmake directly in UFS build

The UFS build script now calls cmake directly instead of using the
build script located in UFS.

In the future, the build directory for all components will be moved
out of the cloned repos and into workflow.

Refs: #366, #368, #374

* Fix linked executables for WAFS

The executables for WAFS have a .x suffix. The link script now uses
the correct executable names.

Refs: #366, #368, #374

* Reorganize some module loads

The order of module loads is reordered so HDF and NetCDF are loaded
before modules that use them.

Refs: #366, #374

* Quiet UFS build module load

Module loading in the UFS build script now sets +x first to turn off
the command trace.

Refs: #366, #368, #374

* Update sorc/build_ufs.sh

* Reorder HDF/NetCDF load order

Moves the HDF and NetCDF module loads before all the NCEP libs.

Refs: #366, #374

* Quiet UFS build module purge

Moved the UFS build module purge inside the set +x block.

Refs: #366, #374

* Update UPP to 10.0.8

Updates UPP to 10.0.8 to be consistent with UFS. This also adds
consistency between the moduleload and checked out version.

Refs: #366, #374

Co-authored-by: Rahul Mahajan <aerorahul@users.noreply.github.com>
kayeekayee pushed a commit to kayeekayee/global-workflow that referenced this issue May 30, 2024
@bensonr
Updates to use FMS 2021.03 (NOAA-EMC#374)
2dc03c1 
* io/FV3GFS_io.F90 updated to use fms2_io
* atmos_model.F90 and ccpp/data/GFS_typefs.F90 updates to input_nml_file handling
* atmos_cubed_sphere submodule updated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@WalterKolczynski-NOAA WalterKolczynski-NOAA

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Unify build system WalterKolczynski-NOAA/global-workflow
1 participant
@WalterKolczynski-NOAA