Update develop-ref after #1851. (#1856)

* Update install_met_env.kiowa Commented out items and added text to description * #1817 Added to_north argument at two_to_one() * #1817 Calls two_to_one() with false for the third argument, to_north, if latitude moves north to south * Feature 1819 automation doc warnings (#1836) * Removing "/lib" that should not be there. * #1508 checking stat() methoid returns no error before looking at FILE flag * #1838 Added log message for lat/lon values * #1838 Give warning if the first and lat lat/lon is same * #1838 Support the double data type variable on reading float type values * Feature 1833 develop discussions (#1849) * Per #1833, changed references to met_help to Discussions. * Per #1833, fixed typo and removed an unnecessary word * Feature 1746 wavelet stat (#1851) * For issue #1746 modified code to allow users to pass in an empty list (or NA) for forecast and observation thresholds in order to skip applying the threhsolds, but it will still compute stats with the raw fields. SL * For issue #1746, added new unit test that uses a config file that has empty lists for the forecast and observation thresholds. SL * For issue #1746 Added some content related to allowing users to set forecast and observation cat thresholds to an empty list in order to skip the binary masking (and consider all grid-points for stats). SL * Per #1746, cleaning up for consistent indentation. * Per #1746, cleaning up for consistent indentation. * Per #1746, add a revision history note, update the plotting range in the postscript output to be [-n,n] where n is the maximum value of the maximum absolute difference and 1.0, and also fix a bug. When the NA threshold comes AFTER a real threshold, the resulting data and difference values were not being updated. * Per #1746, change the Wavelet-Stat config file values in the the wvlt_plot dictionary by setting plot_min = plot_max = 0.0. That enables the default logic of the tool to take effect. Choose the plotting range of the wavelet plots as [-n,n], where n is the maximum of 1.0 and the maximum absolute difference. * Per #1746, used apply_fcst_thresh where it should have been apply_obs_thresh. * Per issue #1746, modified some content related to users being able to skip applying the categorical threhsolds by putting an empty list or NA in the configuration file. SL * Per issue #1746 Added some warnings if the forecast threshold is set to NA but the observation threshold is not NA (a numeric threshold) and vice versa. SL * Per #1746, fix a couple of typos and tweak wording in the wavelet-stat chapter. * Per #1746, loop over each pair of fcst/obs thresholds to check for inconsistent use of the NA threshold type. * Per #1746, a bit of code cleanup replacing calls to n_elements() with n() to make the code more concise. * Per #1746, need to reinitialize apply_fcst_thresh and apply_obs_thresh to true inside the loop since the NA threshold can appear anywhere in the list of thresholds. Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu> Co-authored-by: John Halley Gotway <johnhg@ucar.edu> Co-authored-by: jprestop <jpresto@ucar.edu> Co-authored-by: Howard Soh <hsoh@kiowa.rap.ucar.edu> Co-authored-by: hsoh-u <hsoh@ucar.edu> Co-authored-by: Seth Linden <linden@ucar.edu> Co-authored-by: Seth Linden <linden@kiowa.rap.ucar.edu>
dtcenter · Jul 15, 2021 · 1d7f37d · 1d7f37d
1 parent 7b3fa26
commit 1d7f37d
Show file tree

Hide file tree

Showing 19 changed files with 318 additions and 59 deletions.
diff --git a/met/configure.ac b/met/configure.ac
@@ -2,7 +2,7 @@
 # Process this file with autoconf to produce a configure script.
 
 AC_PREREQ([2.69])
-AC_INIT([MET], [m4_esyscmd_s(echo ${MET_BUILD_VERSION:-development})], [met_help@ucar.edu], [], [http://dtcenter.org/community-code/model-evaluation-tools-met])
+AC_INIT([MET], [m4_esyscmd_s(echo ${MET_BUILD_VERSION:-development})], [https://github.com/dtcenter/METplus/discussions], [], [http://dtcenter.org/community-code/model-evaluation-tools-met])
 AC_CONFIG_SRCDIR([src/tools/tc_utils/tc_dland/tc_dland.cc])
 AC_CONFIG_HEADERS([config.h])
 

diff --git a/met/data/config/WaveletStatConfig_default b/met/data/config/WaveletStatConfig_default
@@ -131,8 +131,8 @@ obs_raw_plot = {
 
 wvlt_plot = {
    color_table = "MET_BASE/colortables/NCL_colortables/BlWhRe.ctable";
-   plot_min = -1.0;
-   plot_max =  1.0;
+   plot_min = 0.0;
+   plot_max = 0.0;
 }
 
 ////////////////////////////////////////////////////////////////////////////////

diff --git a/met/data/wrappers/read_tmp_ascii.py b/met/data/wrappers/read_tmp_ascii.py
@@ -16,7 +16,7 @@
 
 __author__ = 'David Fillmore'
 __version__ = '1.0.0'
-__email__ = 'met_help@ucar.edu'
+
 
 import argparse
 

diff --git a/met/docs/Users_Guide/appendixA.rst b/met/docs/Users_Guide/appendixA.rst
@@ -36,7 +36,7 @@ A. No - it will not. In the future, we may add options to allow additional model
 
 **Q. How do I get help if my questions are not answered in the User's Guide?**
 
-A. First, look on our `MET User's Guide website <https://dtcenter.org/community-code/model-evaluation-tools-met>`_. If that doesn't answer your question, then email: met_help@ucar.edu.
+A. First, look on our `MET User's Guide website <https://dtcenter.org/community-code/model-evaluation-tools-met>`_. If that doesn't answer your question, create a post in the `METplus GitHub Discussions Forum <https://github.com/dtcenter/METplus/discussions>`_.
 
 **Q. Where are the graphics?**
 
@@ -94,9 +94,9 @@ The first place to look for help with individual commands is this user's guide o
 Where to get help
 _________________
 
-If none of the above suggestions have helped solve your problem, help is available through: met_help@ucar.edu
+If none of the above suggestions have helped solve your problem, help is available through the `METplus GitHub Discussions Forum <https://github.com/dtcenter/METplus/discussions>`_.
 
 How to contribute code
 ______________________
 
-If you have code you would like to contribute, we will gladly consider your contribution. Please send email to: met_help@ucar.edu
+If you have code you would like to contribute, we will gladly consider your contribution. Please create a post in the `METplus GitHub Discussions Forum <https://github.com/dtcenter/METplus/discussions>`_.
diff --git a/met/docs/Users_Guide/index.rst b/met/docs/Users_Guide/index.rst
@@ -6,7 +6,7 @@ User's Guide
 
 This User's guide is provided as an aid to users of the Model Evaluation Tools (MET). MET is a set of verification tools developed by the Developmental Testbed Center (DTC) for use by the numerical weather prediction community to help them assess and evaluate the performance of numerical weather predictions. It is also the core component of the unified METplus verification framework. More details about METplus can be found on the `METplus website <http://dtcenter.org/community-code/metplus>`_.
 
-It is important to note here that MET is an evolving software package. This documentation describes the |release| release dated |release_date|. Previous releases of MET have occurred each year since 2008. Intermediate releases may include bug fixes. MET is also able to accept new modules contributed by the community. If you have code you would like to contribute, we will gladly consider your contribution. Please send an email to: `met_help@ucar.edu <mailto:>`__. We will then determine the maturity of the new verification method and coordinate the inclusion of the new module in a future version.
+It is important to note here that MET is an evolving software package. This documentation describes the |release| release dated |release_date|. Previous releases of MET have occurred each year since 2008. Intermediate releases may include bug fixes. MET is also able to accept new modules contributed by the community. If you have code you would like to contribute, we will gladly consider your contribution. Please create a post in the `METplus GitHub Discussions Forum <https://github.com/dtcenter/METplus/discussions>`_. We will then determine the maturity of the new verification method and coordinate the inclusion of the new module in a future version.
 
 **Model Evaluation Tools (MET)  TERMS OF USE - IMPORTANT!**
 

diff --git a/met/docs/Users_Guide/overview.rst b/met/docs/Users_Guide/overview.rst
@@ -81,13 +81,13 @@ MET is an evolving verification software package. New capabilities are planned i
 Code support
 ____________
 
-MET support is provided through a MET-help email address: met_help@ucar.edu. We will endeavor to respond to requests for help in a timely fashion. In addition, information about MET and tools that can be used with MET are provided on the `MET web page <https://dtcenter.org/community-code/model-evaluation-tools-met>`_.
+MET support is provided through the `METplus GitHub Discussions Forum <https://github.com/dtcenter/METplus/discussions>`_. We will endeavor to respond to requests for help in a timely fashion. In addition, information about MET and tools that can be used with MET are provided on the `MET web page <https://dtcenter.org/community-code/model-evaluation-tools-met>`_.
 
 We welcome comments and suggestions for improvements to MET, especially information regarding errors. Comments may be submitted using the MET Feedback form available on the MET website. In addition, comments on this document would be greatly appreciated. While we cannot promise to incorporate all suggested changes, we will certainly take all suggestions into consideration.
 
 **-help** and **-version** command line options are available for all of the MET tools. Typing the name of the tool with no command line options also produces the usage statement.
 
-The MET package is a "living" set of tools. Our goal is to continually enhance it and add to its capabilities. Because our time, resources, and talents are limited, we welcome contributed code for future versions of MET. These contributions may represent new verification methodologies, new analysis tools, or new plotting functions. For more information on contributing code to MET, please contact met_help@ucar.edu.
+The MET package is a "living" set of tools. Our goal is to continually enhance it and add to its capabilities. Because our time, resources, and talents are limited, we welcome contributed code for future versions of MET. These contributions may represent new verification methodologies, new analysis tools, or new plotting functions. For more information on contributing code to MET, please create a post in the `METplus GitHub Discussions Forum <https://github.com/dtcenter/METplus/discussions>`_.
 
 Fortify
 _______

diff --git a/met/docs/Users_Guide/tc-pairs.rst b/met/docs/Users_Guide/tc-pairs.rst
@@ -222,7 +222,7 @@ ____________________
      time_offset = -14400;
   }
 
-The **watch_warn** field specifies the file name and time applied offset to the **watch_warn** flag. The **file_name** string specifies the path of the watch/warning file to be used to determine when a watch or warning is in effect during the forecast initialization and verification times. The default file is named **wwpts_us.txt**, which is found in the installed *share/met/tc_data/* directory within the MET build. The **time_offset** string is the time window (in seconds) assigned to the watch/warning. Due to the non-uniform time watches and warnings are issued, a time window is assigned for which watch/warnings are included in the verification for each valid time. The default watch/warn file is static, and therefore may not include warned storms beyond the current MET code release date; therefore users may wish to contact met_help@ucar.edu to obtain the most recent watch/warning file if the static file does not contain storms of interest.
+The **watch_warn** field specifies the file name and time applied offset to the **watch_warn** flag. The **file_name** string specifies the path of the watch/warning file to be used to determine when a watch or warning is in effect during the forecast initialization and verification times. The default file is named **wwpts_us.txt**, which is found in the installed *share/met/tc_data/* directory within the MET build. The **time_offset** string is the time window (in seconds) assigned to the watch/warning. Due to the non-uniform time watches and warnings are issued, a time window is assigned for which watch/warnings are included in the verification for each valid time. The default watch/warn file is static, and therefore may not include warned storms beyond the current MET code release date; therefore users may wish to create a post in the `METplus GitHub Discussions Forum <https://github.com/dtcenter/METplus/discussions>`_ in order to obtain the most recent watch/warning file if the static file does not contain storms of interest.
 
 .. code-block:: none
 

diff --git a/met/docs/Users_Guide/wavelet-stat.rst b/met/docs/Users_Guide/wavelet-stat.rst
@@ -30,7 +30,7 @@ The method
 
 The Intensity Scale approach can be summarized in the following 5 steps:
 
-1. For each threshold, the forecast and observation fields are transformed into binary fields: where the grid-point precipitation value meets the threshold criteria it is assigned 1, where the threshold criteria are not met it is assigned 0. :numref:`wavelet-stat_NIMROD_3h_fcst` illustrates an example of a forecast and observation fields, and their corresponding binary fields for a threshold of 1mm/h. This case shows an intense storm of the scale of 160 km displaced almost its entire length. The displacement error is clearly visible from the binary field difference and the contingency table image obtained for the same threshold :numref:`contingency_table_counts`.
+1. For each threshold, the forecast and observation fields are transformed into binary fields: where the grid-point precipitation value meets the threshold criteria it is assigned 1, where the threshold criteria are not met it is assigned 0. This can also be done with no thresholds indicated at all and in that case the grid-point values are not transformed to binary fields and instead the raw data is used as is for statistics. :numref:`wavelet-stat_NIMROD_3h_fcst` illustrates an example of a forecast and observation fields, and their corresponding binary fields for a threshold of 1mm/h. This case shows an intense storm of the scale of 160 km displaced almost its entire length. The displacement error is clearly visible from the binary field difference and the contingency table image obtained for the same threshold :numref:`contingency_table_counts`.
 
 2. The binary forecast and observation fields obtained from the thresholding are then decomposed into the sum of components on different scales, by using a 2D Haar wavelet filter (:numref:`wavelet-stat_NIMROD_diff`). Note that the scale components are fields, and their sum adds up to the original binary field. For a forecast defined over square domain of **2ⁿ x 2ⁿ** grid-points, the scale components are **n+1: n** mother wavelet components + the largest father wavelet (or scale-function) component. The **n** mother wavelet components have resolution equal to **1, 2, 4, ...** :math:`\mathbf{2^{n-1}}`  grid-points. The largest father wavelet component is a constant field over the **2ⁿ x 2ⁿ** grid-point domain with value equal to the field mean.
 
@@ -230,6 +230,23 @@ _______________________
 
 .. code-block:: none
 
+  // Empty list of thresholds
+  cat_thresh = [];
+
+  // Or explicitly set the NA threshold type
+  cat_thresh = [>0.0, >=5.0, NA];
+
+   
+The **cat_thresh** option defines an array of thresholds for each field defined in the fcst and obs dictionaries. The number of forecast and observation categorical thresholds must match. If set to an empty list, the thresholds will not be applied (no binary masking) and all the raw grid-point values will be used for downstream statistics.
+
+If the array of thresholds is an empty list, the application will set the threshold to NA internally and skip applying the thresholds. If the threshold is set to NA explicitly in the list, the application will also skip applying the threshold.
+
+Since the application has the ability to loop through multiple thresholds (for multiple fields), a user can include NA in the list of thresholds to produce statistics for the raw data values for the given field.
+
+_______________________
+
+.. code-block:: none
+		
   grid_decomp_flag = AUTO;
   
   tile = {

diff --git a/met/scripts/config/WaveletStatConfig_APCP_12 b/met/scripts/config/WaveletStatConfig_APCP_12
@@ -50,7 +50,7 @@ fcst = {
       {
         name       = "APCP";
         level      = [ "A12" ];
-        cat_thresh = [ >0.0, >=5.0 ];
+        cat_thresh = [ >0.0, >=5.0, NA ];
       }
    ];
 }
@@ -59,7 +59,7 @@ obs = {
       {
         name       = "APCP_12";
         level      = [ "(*,*)" ];
-        cat_thresh = [ >0.0, >=5.0 ];
+        cat_thresh = [ >0.0, >=5.0, NA ];
       }
    ];
 }
@@ -137,8 +137,8 @@ obs_raw_plot = {
 
 wvlt_plot = {
    color_table = "MET_BASE/colortables/NCL_colortables/BlWhRe.ctable";
-   plot_min = -1.0;
-   plot_max =  1.0;
+   plot_min = 0.0;
+   plot_max = 0.0;
 }
 
 ////////////////////////////////////////////////////////////////////////////////

diff --git a/met/src/libcode/vx_data2d_nccf/nccf_file.cc b/met/src/libcode/vx_data2d_nccf/nccf_file.cc
@@ -2909,6 +2909,11 @@ void NcCfFile::get_grid_from_lat_lon_vars(NcVar *lat_var, NcVar *lon_var,
 
   double dlat = lat_values[1] - lat_values[0];
   double dlon = rescale_lon(lon_values[1] - lon_values[0]);
+  mlog << Debug(7) << method_name << " -> lat[0]=" << lat_values[0]
+       << " lat[" << (lat_counts-1) << "]=" << lat_values[lat_counts-1]
+       << " dlat=" << dlat << " lon[0]=" << lon_values[0]
+       << " lon[" << (lon_counts-1) << "]=" << lon_values[lon_counts-1]
+       << " dlon=" << dlon << "\n";
 
   ConcatString point_nccf;
   bool skip_sanity_check = get_att_value_string(_ncFile, nc_att_met_point_nccf, point_nccf);

diff --git a/met/src/libcode/vx_nc_util/nc_utils.cc b/met/src/libcode/vx_nc_util/nc_utils.cc
@@ -1433,6 +1433,18 @@ bool get_nc_data(NcVar *var, float *data) {
       if (NcType::nc_FLOAT == type_id) {
          var->getVar(data);
       }
+      else if (NcType::nc_DOUBLE == type_id) {
+         int cell_count = 1;
+         for (int idx=0; idx<var->getDimCount();idx++) {
+            cell_count *= get_dim_size(var, idx);
+         }
+         double *double_data = new double[cell_count];
+         var->getVar(double_data);
+         for (int idx=0; idx<cell_count; idx++) {
+            data[idx] = (float)double_data[idx];
+         }
+         delete [] double_data;
+      }
       else {
          int cell_count = 1;
          for (int idx=0; idx<var->getDimCount();idx++) {