METPlus Tutorial Environment on Cheyenne not working

[cohenjt]

Hi! I am trying to follow the tutorial here: https://dtcenter.org/metplus-practical-session-guide-version-5-0/session-1-metplus-setupgrid-grid/metplus-setup/metplus-initial-setup/setting-tutorial-environment and am running my code on Casper. I followed these directions, but when I ran source METplus-5.0.0_TutorialSetup.sh, a number of modules could not load as it seemed the version numbers were out of date:

Lmod has detected the following error: The following module(s) are unknown: "ncarenv/1.3" "netcdf/4.8.0"

I think this is just caused by the outdated version numbers in the config file, but I'm not sure how to fix this to get the environment running correctly. Thanks for your help!

[georgemccabe]

Hi @cohenjt,

The METplus 5.0 tutorial instructions were tested using Cheyenne, but they were not tested on Casper. If you are able to use Cheyenne instead of Casper, I would suggest doing that.

There are instructions to set up METplus on Casper on the METplus-5.0 Existing Builds page. However it appears that the modules on Casper have changed since the Casper-specific METplus module files were created, because the metplus/5.0.1 and met/11.0.1 module scripts were broken. I worked with another METplus developer just now to fix these module scripts so they do not produce errors (thanks @jprestop !), but we have not tested that everything in the tutorial will work as expected. Also, other METplus components like METcalcpy and METplotpy were not configured for casper, so tutorial instructions that use those components will not work.

If you would like to try using the tutorial on Casper, you will need to make a few modifications to the tutorial setup script after you copy it into your directory, described below. Again, please keep in mind that the tutorial has not been tested to run this way and some portions of the tutorial will likely not work as expected.

Change the module use command for METplus to (previously did not have casper directory):
module use /glade/p/ral/jntp/MET/METplus/casper/modulefiles

and remove the METcalcpy, METplotpy, and METdataIO module loads:

module use /glade/p/ral/jntp/MET/METcalcpy/modulefiles
module load metcalcpy/2.0.2
module use /glade/p/ral/jntp/MET/METplotpy/modulefiles
module load metplotpy/2.0.1
module use /glade/p/ral/jntp/MET/METdataio/modulefiles
module load metdataio/2.0.2

[ahijevyc]

Just in case someone else familiar with the old met versions has the same problem after running module use /glade/p/ral/jntp/MET/METplus/casper/modulefiles
you might need to reset your modules module reset to remove any wrong module locations (i.e. /glade/p/ral/jntp/MET/MET_releases/modulefiles). I still had the old one in my path from the previous attempts. And you have to load metplus not met.

[jprestop]

Hi @ahijevyc. I wanted to let you know that we have updated our installations on Derecho and Caper and we are now using modulefiles in new locations. You can find that information on the Existing Builds page for the coordinated 5.1 release under "NCAR machines". Please let us know if you have any questions or problems. If you do not have any further questions, we'd love to have you select "Mark as Answer" for the answer you feel best resolved the problem and then I'll lock this Discussion to prevent future posts. Our team has decided that once discussions have been answered, and the answer has been confirmed, we’ll lock them. We want to encourage users to ask new questions in new discussions rather than posting to old ones. Hopefully that’ll make the questions/answers easier for other users to follow. So if/when more issues/questions arise, please feel free to start a new discussion.

[cohenjt]

Hi @jprestop. Thank you for your comment. I have been successfully running MODE with python embedding. I used the new paths from the existing builds page to change the bash file I use to setup MODE to the following. I think I did something incorrectly because MODE is not running anymore. Please let me know if the mistake is easy to identify, otherwise I can upload my config files and the resulting log file. Thank you!

# METplus v5.0.0 Tutorial Setup Script on Cheyenne (NCAR) using bash

# module use /glade/p/ral/jntp/MET/METplus/casper/modulefiles
module use /glade/work/dtcrt/METplus/casper/components/METplus/installations/modulefiles
module load metplus/5.1.0

module use /glade/work/dtcrt/METplus/casper/components/MET/installations/modulefiles
module load met/11.1.0

# Directory to store tutorial files - configurations, output, notes, etc.
export METPLUS_TUTORIAL_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"

# Path to the METplus installation location
# was /glade/p/ral/jntp/MET/METplus/METplus-5.0.1
export METPLUS_BUILD_BASE=/glade/work/dtcrt/METplus/casper/components/METplus/installations/METplus-5.1.0

# Path to the MET installation location
# was /glade/p/ral/jntp/MET/MET_releases/11.0.1
export MET_BUILD_BASE=/glade/work/dtcrt/METplus/casper/components/MET/installations/11.1.0

# Path to input data required for running use cases
# was /glade/p/ral/jntp/MET/METplus/METplus-5.0_sample_data
export METPLUS_DATA=/glade/work/dtcrt/METplus/data/components/METplus/METplus-5.1_sample_data

[jprestop]

Hi @cohenjt. All of those module paths look correct to me. I loaded met/11.1.0 and metplus/5.1.0 and both seemed to have worked as expected. Please pass along your config files and log file. Hopefully, the log file will contain an error that we can take a look at and try to resolve.

[cohenjt]

Hi @jprestop. I just uploaded all my files to the server under cohen_data. Please let me know if there's anything else I can send over. The log file is ~15MB and seems to contain a lot of error messages that I am having trouble understanding. Thank you for your help!

[jprestop]

@cohenjt Thank you so much for uploading your log file. It was very helpful. I see the error:

ImportError: /glade/work/dtcrt/METplus/casper/miniconda/miniconda3/envs/metplus_v5.1_py3.10/lib/python3.10/site-packages/netCDF4/../../../libnetcdf.so.19: undefined symbol: H5Pset_fapl_ros3

which I have seen before. We have addressed this problem in our MET-12.0.0-beta3 release. I'm going to talk with my team about the best way to proceed. I will follow up once I have more information. In the meantime, please feel free to switch back to what you were using previously, but at some point soon, I know that access to /glade/p will be going away.