Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[release/public-v2.2.0] Update MacOS configuration and compatibility #953

Merged

Conversation

natalie-perlin
Copy link
Collaborator

@natalie-perlin natalie-perlin commented Oct 26, 2023

DESCRIPTION OF CHANGES:

Updated modulefiles, scripts and configurations for running SRW v2.2.0 on MacOS platforms, summarized below. The changes include adaptation of the code for two MacOS architecthures, Intel/x86_64 and M1/arm64 (could be aarch64 if using Rosetta)

  • modulefiles in ./modulefiles, ./modulefiles/tasks/macos/, srw_common.lua (successfully tested on all platforms but cloud)
  • machine file in ./ush/machine/macos.yaml
  • ./etc/lmod-setup.*
  • scripts in ./ush/wrappers/* ./jobs/ , ./scripts/*.sh - the header changed to #!/usr/bin/env bash; a small bug in one of the scripts
  • a patch file is applied for the ./sorc/CMakeLists.txt to build SRW on M1/arm64 machine; patch is located in ./patches/patch_macos_arm64_sorc_cmakelists.txt, and it is applied in ./devbuild.sh when needed
  • small fixes in ./ush/job_preamble.sh, ./ush/preamble.sh (use mac-specific commands for runtime estimates)
  • fix for ./ush/python_utils/filesys_cmds_vrfy.py: function vrfy_ln creates links with gnu-ln on MacOS (from coreutils)
  • updates to the documentation for the MacOS users and adding options to install workflow conda environments.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

TESTS CONDUCTED:

Tested and adapted for the following MacOS configurations:

  • MacBook2019 i9 Intel/x86_64, 8-core, Darwin 21.6.0, Monterey 12.6.3, 32 GB RAM
  • MacAir 2020 M1 /arm64, Darwin 22.1.0, Ventura 13.0.1, 16 GB RAM, running natively/no Rosetta

On both systems, SRW have been built and completed successfully 6-h test run for 20190615, RRFS_CONUS_25km grid, physics suite FV3_GFS_v16. Tests were run using wrappers scripts from ./ush/wrapper/run_*,sh

Building the SRW has been tested on all the RDHPCS except NOAA cloud:
Gaea, Orion, Hercules, Gaea-c5, Cheyenne, Derecho, Hera, Jet.

Same 6-h test run has been tested on Gaea, Gaea-c5, Hercules, Orion, and Derecho.
All fundamental tests successfully pass on Gaea, Gaea-c5, Hercules, Derecho. Orion has 2 tests out of 7 that died:
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v17_p8_plot
grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_HRRR_suite_HRRR
The reason or workflow task that failed has not been checked yet.

Attached are log files from building current version on all the platforms, and two zipped bundles of log files from the run on the two MacOS platform.

build.srw.v2.2.0.macos_macos_x86_64.log
build.srw-v2.2.0_macos_macos_arm64.log
build.srw-v2.2.0.macos_cheyenne.log
build.srw-v2.2.0.macos_derecho.log
build.srw-v2.2.0.macos_gaea-c5.log
build.srw-v2.2.0.macos_gaea.log
build.srw-v2.2.0.macos_orion.log
build.srw-v2.2.0.macos.hera.log
build.srw-v2.2.0.macos_jet.log
SRWtest_MacOS_arm64.tgz
SRWtest_MacOS_x86_64.tgz

DOCUMENTATION:

Added notes:

  • on MacOS platforms tested for the current release of the SRW v2.2.0
  • on ways to building conda environments (workflow_tools for the main workflow, regional_workflow for graphic generation (a legacy name), and regional_workflow_cmaq for running the Air Quality Model (AQM/CMAQ).

ISSUE:

Current SRW does not run on Mac as-is and all the configurations are outdated

CHECKLIST

  • My code follows the style guidelines in the Contributor's Guide
  • I have performed a self-review of my own code using the Code Reviewer's Guide
  • I have commented my code, particularly in hard-to-understand areas
  • My changes need updates to the documentation, in progress
  • My changes do not require updates to the documentation; all the documentation changes included.
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • Any dependent changes have been merged and published

LABELS (optional):

Add an X to check off a box. Delete any unnecessary labels. -->
A Code Manager needs to add the following labels to this PR:

  • Work In Progress
  • bug
  • enhancement
  • documentation
  • release
  • high priority
  • run_ci
  • run_we2e_fundamental_tests
  • run_we2e_comprehensive_tests
  • Needs Cheyenne test
  • Needs Jet test
  • Needs Hera test
  • Needs Orion test
  • help wanted

CONTRIBUTORS (optional):

Gillian Petro - @gspetro-NOAA - documentation editing and formatting

@natalie-perlin natalie-perlin added bug Something isn't working Work in Progress labels Oct 26, 2023
@MichaelLueken MichaelLueken changed the title Update MacOS configuration and compatibility [release/public-v2.2.0] Update MacOS configuration and compatibility Oct 26, 2023
Comment on lines 268 to 270
source ~/conda/etc/profile.d/conda.sh
conda activate
conda install -y conda-build conda-verify
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These will also need to be indented to line up with 265-267.

natalie-perlin and others added 5 commits October 31, 2023 12:25
Edits to text and formatting for RunSRW.rst in documentation section

Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com>
Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com>
@natalie-perlin
Copy link
Collaborator Author

@MichaelLueken - the final edits are in. Gillian could approve it after taking a final look at the documentation built locally.

Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com>
Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com>
@@ -216,6 +286,9 @@ In future shells, you can activate and use this environment with:

See the `workflow-tools repository <https://github.com/ufs-community/workflow-tools>`__ for additional documentation.
Copy link
Collaborator

@gspetro-NOAA gspetro-NOAA Oct 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@natalie-perlin Didn't see this before, but lines 280-287 need to be indented 3 spaces to fall within section 2).

Natalie Perlin and others added 7 commits October 31, 2023 13:31
Formatting changes

Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com>
Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com>
Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com>
Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com>
Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com>
@MichaelLueken MichaelLueken merged commit e0ea410 into ufs-community:release/public-v2.2.0 Oct 31, 2023
3 checks passed
@natalie-perlin natalie-perlin deleted the update_macos branch October 31, 2023 17:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Priority: HIGH release This PR/issue is related to a release branch run_we2e_coverage_tests Run the coverage set of SRW end-to-end tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants