Multivariate MODE Improvements

[lindsayrblank]

Replace italics below with details for this issue.

Describe the Enhancement

Some possible improvements added by @davidalbo to build on what was completed in this issue:
#1283 are as follows:

1. Possibly change the merge_flag default to NONE, and allow it to not be set for MvMode fields. If not set right now the code uses a default merge_flag=THRESH and merge_thresh = ">=1.25". MvMode is now implemented so that the user MUST set all the merge_flag (and merge_thresh) values explicity. Would need to see if changing for both mvmode and single var mode causes problems for single var mode.
2. Possibly allow different number of obs and forecast inputs to create super objects, and to produce output when all the intensity flags are false. More logic would be needed to tell mode which obs input (with intensity=TRUE) should be compared to which fcst input (with intensity = TRUE). Is this useful?
3. Allow quilting. Might lead to an explosion in the naming.
4. Allow multiple convolution radii and thresholds without quilting. Might lead to an explosion in the naming.
5. Change so do not read data in twice in part 1 simple, then again in part 1 merge. Store in memory instead.
6. Can merge_flag ENGINE or BOTH be allowed? How would that work?

The following is a list of enhancements desired to multivariate MODE:

Version 10.1 beta 1

1. Resultant super object netCDF file should only have super object field in it.
2. The above super object field should be named something like "super_object".
3. Combine the fields and the super object config file into one config file.
4. Make it so you don't have to run the MODE executable twice if you don't want to.
5. Allow for matching/merging of super objects in same config file.
6. Not have the user create the output directories for the single fields.

Future work (as time and funding allow)

1. Make writing the output for the individual runs of MODE optional.

Time Estimate

Estimate the amount of work required here.
Issues should represent approximately 1 to 3 days of work.

Sub-Issues

Consider breaking the enhancement down into sub-issues.

• Add a checkbox for each sub-issue here.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

2700042

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Review projects and select relevant Repository and Organization ones or add "alert:NEED PROJECT ASSIGNMENT" label
• Select milestone to next major version milestone or "Future Versions"

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdatadb, METviewer, METexpress, METcalcpy, METplotpy

Enhancement Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s), Project(s), Milestone, and Linked issues
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.

[lindsayrblank]

Issues:

2. Make it so the user can customize the name of the resulting super object field. (The FCST and OBS prefix will be automatically added and not customizable). Would parse this in the fcst and obs dictionaries.

3. What is defined in the field dictionary applies to the individual MODE run, anything at the fcst or obs dictionaries (next level up) applies to super object.

4. Run in memory.

[davidalbo]

@hertneky @JohnHalleyGotway I'm going through the MODE tutorial online, getting through until a python problem once it gets to MetPlus, here:
https://dtcenter.org/metplus-practical-session-guide-version-5-0/session-5-mode-mtd/metplus-use-case-mode

The python version problem seen below seems familiar. Any thoughts?

seneca:/d1/personal/dave/MET/MET_development/METplus-5.0.0_Tutorial% run_metplus.py
? ${METPLUS_BUILD_BASE}/parm/use_cases/met_tool_wrapper/MODE/MODE.conf
? ${METPLUS_TUTORIAL_DIR}/tutorial.conf
? config.OUTPUT_BASE=${METPLUS_TUTORIAL_DIR}/output/MODE
WARNING: Python 3.8.6 or higher is recommended. You are using 3.7.3.
See the METplus documentation for more information.
Traceback (most recent call last):
File "/usr/local/METplus-5.0.0/ush/run_metplus.py", line 29, in
from metplus.util import metplus_check
File "/usr/local/METplus-5.0.0/metplus/init.py", line 26, in
from .util import *
File "/usr/local/METplus-5.0.0/metplus/util/init.py", line 5, in
from .time_util import *
File "/usr/local/METplus-5.0.0/metplus/util/time_util.py", line 13, in
from dateutil.relativedelta import relativedelta
ModuleNotFoundError: No module named 'dateutil'

[hertneky]

@davidalbo If you are working on seneca, you need to be using a python environment that has the necessary packages (e.g. dateutil).
@JohnHalleyGotway Does he just need to set the MET_PYTHON_EXE=/home/met_test/.conda/envs/pygrib.v5/bin/python3

[georgemccabe]

@davidalbo it looks like the version of python3 in your path is from /usr/bin. You should add /usr/local/python3/bin to your path so it uses that version of python3.

[davidalbo]

@georgemccabe That did it, thanks. Easy once you know where the correct python3 is.

[davidalbo]

Closing this issue as everything in it has been addressed by issue #2745