Skip to content

Commit

Permalink
Per #1019, initial setup for supporting -format uscrn. It compiles bu…
Browse files Browse the repository at this point in the history 
…t I still need to make it work for the variety of USCRN inputs.
  • Loading branch information
@JohnHalleyGotway
JohnHalleyGotway committed Jan 7, 2025
1 parent 54798e8 commit 30bf0b7
Show file tree
Hide file tree
Showing 6 changed files with 468 additions and 21 deletions.
6 changes: 4 additions & 2 deletions docs/Users_Guide/reformat_point.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -450,6 +450,8 @@ While initial versions of the ASCII2NC tool only supported a simple 11 column AS

• `AErosol RObotic NEtwork (AERONET) versions 2 and 3 format <http://aeronet.gsfc.nasa.gov/>`_

• `U.S. Climate Reference Network (USCRN) Quality Controlled Datasets <https://www.ncei.noaa.gov/access/crn/qcdatasets.html>`_

• Python embedding of point observations, as described in :numref:`pyembed-point-obs-data`. See example below in :numref:`ascii2nc-pyembed`.

The default ASCII point observation format consists of one row of data per observation value. Each row of data consists of 11 columns as shown in :numref:`table_reformat-point_ascii2nc_format`.
Expand Down Expand Up @@ -507,7 +509,7 @@ Once the ASCII point observations have been formatted as expected, the ASCII fil
Usage: ascii2nc
ascii_file1 [ascii_file2 ... ascii_filen]
netcdf_file
[-format ASCII_format]
[-format type]
[-config file]
[-valid_beg time]
[-valid_end time]
Expand All @@ -530,7 +532,7 @@ Required Arguments for ascii2nc
Optional Arguments for ascii2nc
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

3. The **-format ASCII_format** option may be set to "met_point", "little_r", "surfrad", "wwsis", "airnowhourlyaqobs", "airnowhourly", "airnowdaily_v2", "ndbc_standard", "ismn", "iabp", "aeronet", "aeronetv2", "aeronetv3", or "python". If passing in ISIS data, use the "surfrad" format flag.
3. The **-format type** option may be set to "met_point", "little_r", "surfrad", "wwsis", "airnowhourlyaqobs", "airnowhourly", "airnowdaily_v2", "ndbc_standard", "ismn", "iabp", "aeronet", "aeronetv2", "aeronetv3", "uscrn", or "python". If passing in ISIS data, use the "surfrad" format flag.

4. The **-config file** option is the configuration file for generating time summaries.

Expand Down
1 change: 1 addition & 0 deletions src/tools/other/ascii2nc/Makefile.am
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ ascii2nc_SOURCES = ascii2nc.cc \
aeronet_handler.cc aeronet_handler.h \
ismn_handler.cc ismn_handler.h \
iabp_handler.cc iabp_handler.h \
uscrn_handler.cc uscrn_handler.h \
$(OPT_PYTHON_SOURCES)

ascii2nc_CPPFLAGS = ${MET_CPPFLAGS} -I../../../basic/vx_log
Expand Down
25 changes: 23 additions & 2 deletions src/tools/other/ascii2nc/Makefile.in
View file
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,8 @@ am__ascii2nc_SOURCES_DIST = ascii2nc.cc ascii2nc_conf_info.cc \
ndbc_locations.cc ndbc_locations.h airnow_locations.cc \
airnow_locations.h aeronet_handler.cc aeronet_handler.h \
ismn_handler.cc ismn_handler.h iabp_handler.cc iabp_handler.h \
python_handler.h python_handler.cc
uscrn_handler.cc uscrn_handler.h python_handler.h \
python_handler.cc
@ENABLE_PYTHON_TRUE@am__objects_1 = ascii2nc-python_handler.$(OBJEXT)
am__objects_2 = $(am__objects_1)
am_ascii2nc_OBJECTS = ascii2nc-ascii2nc.$(OBJEXT) \
Expand All @@ -127,7 +128,8 @@ am_ascii2nc_OBJECTS = ascii2nc-ascii2nc.$(OBJEXT) \
ascii2nc-airnow_locations.$(OBJEXT) \
ascii2nc-aeronet_handler.$(OBJEXT) \
ascii2nc-ismn_handler.$(OBJEXT) \
ascii2nc-iabp_handler.$(OBJEXT) $(am__objects_2)
ascii2nc-iabp_handler.$(OBJEXT) \
ascii2nc-uscrn_handler.$(OBJEXT) $(am__objects_2)
ascii2nc_OBJECTS = $(am_ascii2nc_OBJECTS)
am__DEPENDENCIES_1 =
ascii2nc_DEPENDENCIES = $(am__DEPENDENCIES_1) $(am__DEPENDENCIES_1) \
Expand Down Expand Up @@ -164,6 +166,7 @@ am__depfiles_remade = ./$(DEPDIR)/ascii2nc-aeronet_handler.Po \
./$(DEPDIR)/ascii2nc-ndbc_locations.Po \
./$(DEPDIR)/ascii2nc-python_handler.Po \
./$(DEPDIR)/ascii2nc-surfrad_handler.Po \
./$(DEPDIR)/ascii2nc-uscrn_handler.Po \
./$(DEPDIR)/ascii2nc-wwsis_handler.Po
am__mv = mv -f
AM_V_lt = $(am__v_lt_@AM_V@)
Expand Down Expand Up @@ -396,6 +399,7 @@ ascii2nc_SOURCES = ascii2nc.cc \
aeronet_handler.cc aeronet_handler.h \
ismn_handler.cc ismn_handler.h \
iabp_handler.cc iabp_handler.h \
uscrn_handler.cc uscrn_handler.h \
$(OPT_PYTHON_SOURCES)

ascii2nc_CPPFLAGS = ${MET_CPPFLAGS} -I../../../basic/vx_log
Expand Down Expand Up @@ -531,6 +535,7 @@ distclean-compile:
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ascii2nc-ndbc_locations.Po@am__quote@ # am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ascii2nc-python_handler.Po@am__quote@ # am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ascii2nc-surfrad_handler.Po@am__quote@ # am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ascii2nc-uscrn_handler.Po@am__quote@ # am--include-marker
@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ascii2nc-wwsis_handler.Po@am__quote@ # am--include-marker

$(am__depfiles_remade):
Expand Down Expand Up @@ -749,6 +754,20 @@ ascii2nc-iabp_handler.obj: iabp_handler.cc
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@
@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o ascii2nc-iabp_handler.obj `if test -f 'iabp_handler.cc'; then $(CYGPATH_W) 'iabp_handler.cc'; else $(CYGPATH_W) '$(srcdir)/iabp_handler.cc'; fi`

ascii2nc-uscrn_handler.o: uscrn_handler.cc
@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT ascii2nc-uscrn_handler.o -MD -MP -MF $(DEPDIR)/ascii2nc-uscrn_handler.Tpo -c -o ascii2nc-uscrn_handler.o `test -f 'uscrn_handler.cc' || echo '$(srcdir)/'`uscrn_handler.cc
@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/ascii2nc-uscrn_handler.Tpo $(DEPDIR)/ascii2nc-uscrn_handler.Po
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='uscrn_handler.cc' object='ascii2nc-uscrn_handler.o' libtool=no @AMDEPBACKSLASH@
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@
@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o ascii2nc-uscrn_handler.o `test -f 'uscrn_handler.cc' || echo '$(srcdir)/'`uscrn_handler.cc

ascii2nc-uscrn_handler.obj: uscrn_handler.cc
@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT ascii2nc-uscrn_handler.obj -MD -MP -MF $(DEPDIR)/ascii2nc-uscrn_handler.Tpo -c -o ascii2nc-uscrn_handler.obj `if test -f 'uscrn_handler.cc'; then $(CYGPATH_W) 'uscrn_handler.cc'; else $(CYGPATH_W) '$(srcdir)/uscrn_handler.cc'; fi`
@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/ascii2nc-uscrn_handler.Tpo $(DEPDIR)/ascii2nc-uscrn_handler.Po
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='uscrn_handler.cc' object='ascii2nc-uscrn_handler.obj' libtool=no @AMDEPBACKSLASH@
@AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@
@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o ascii2nc-uscrn_handler.obj `if test -f 'uscrn_handler.cc'; then $(CYGPATH_W) 'uscrn_handler.cc'; else $(CYGPATH_W) '$(srcdir)/uscrn_handler.cc'; fi`

ascii2nc-python_handler.o: python_handler.cc
@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(ascii2nc_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT ascii2nc-python_handler.o -MD -MP -MF $(DEPDIR)/ascii2nc-python_handler.Tpo -c -o ascii2nc-python_handler.o `test -f 'python_handler.cc' || echo '$(srcdir)/'`python_handler.cc
@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/ascii2nc-python_handler.Tpo $(DEPDIR)/ascii2nc-python_handler.Po
Expand Down Expand Up @@ -904,6 +923,7 @@ distclean: distclean-am
-rm -f ./$(DEPDIR)/ascii2nc-ndbc_locations.Po
-rm -f ./$(DEPDIR)/ascii2nc-python_handler.Po
-rm -f ./$(DEPDIR)/ascii2nc-surfrad_handler.Po
-rm -f ./$(DEPDIR)/ascii2nc-uscrn_handler.Po
-rm -f ./$(DEPDIR)/ascii2nc-wwsis_handler.Po
-rm -f Makefile
distclean-am: clean-am distclean-compile distclean-generic \
Expand Down Expand Up @@ -964,6 +984,7 @@ maintainer-clean: maintainer-clean-am
-rm -f ./$(DEPDIR)/ascii2nc-ndbc_locations.Po
-rm -f ./$(DEPDIR)/ascii2nc-python_handler.Po
-rm -f ./$(DEPDIR)/ascii2nc-surfrad_handler.Po
-rm -f ./$(DEPDIR)/ascii2nc-uscrn_handler.Po
-rm -f ./$(DEPDIR)/ascii2nc-wwsis_handler.Po
-rm -f Makefile
maintainer-clean-am: distclean-am maintainer-clean-generic
Expand Down
60 changes: 43 additions & 17 deletions src/tools/other/ascii2nc/ascii2nc.cc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@
// 021 10/03/22 Prestopnik MET #2227 Remove using namespace std from header files
// 022 10/07/22 Dave Albo MET #2276 Add NDBC buoy data
// 023 11/28/23 Halley Gotway MET #2701 Add ISMN soil moisture data
// 024 01/06/25 Halley Gotway MET #1019 Add USCRN quality controlled data
//
////////////////////////////////////////////////////////////////////////

Expand Down Expand Up @@ -87,6 +88,7 @@
#include "ndbc_handler.h"
#include "ismn_handler.h"
#include "iabp_handler.h"
#include "uscrn_handler.h"

#ifdef ENABLE_PYTHON
#include "global_python.h"
Expand Down Expand Up @@ -119,6 +121,7 @@ enum class ASCIIFormat {
NDBC_standard,
ISMN,
IABP,
USCRN,
Aeronet_v2,
Aeronet_v3,
Python,
Expand Down Expand Up @@ -359,6 +362,10 @@ FileHandler *create_file_handler(const ASCIIFormat format, const ConcatString &a
return((FileHandler *) new IabpHandler(program_name));
}

case ASCIIFormat::USCRN: {
return((FileHandler *) new UscrnHandler(program_name));
}

case ASCIIFormat::Aeronet_v2: {
AeronetHandler *handler = new AeronetHandler(program_name);
handler->setFormatVersion(2);
Expand Down Expand Up @@ -524,6 +531,19 @@ FileHandler *determine_ascii_format(const ConcatString &ascii_filename) {

delete ismn_file;

//
// See if this is a USCRN file.
//
f_in.rewind();
UscrnHandler *uscrn_file = new UscrnHandler(program_name);

if(uscrn_file->isFileType(f_in)) {
f_in.close();
return (FileHandler *) uscrn_file;
}

delete uscrn_file;

//
// If we get here, we didn't recognize the file contents.
//
Expand All @@ -543,7 +563,7 @@ void usage() {
<< program_name << "\n"
<< "\tascii_file1 [ascii_file2 ... ascii_filen]\n"
<< "\tnetcdf_file\n"
<< "\t[-format ASCII_format]\n"
<< "\t[-format type]\n"
<< "\t[-config file]\n"
<< "\t[-mask_grid string]\n"
<< "\t[-mask_poly file]\n"
Expand All @@ -561,26 +581,29 @@ void usage() {
<< "\t\t\"netcdf_file\" indicates the name of the output "
<< "NetCDF file to be written (required).\n"

<< "\t\t\"-format ASCII_format\" may be set to \""
<< MetHandler::getFormatString() << "\", \""
<< LittleRHandler::getFormatString() << "\", \""
<< SurfradHandler::getFormatString() << "\", \""
<< WwsisHandler::getFormatString() << "\", \""
<< AirnowHandler::getFormatStringDailyV2() << "\", \""
<< AirnowHandler::getFormatStringHourlyAqObs() << "\", \""
<< AirnowHandler::getFormatStringHourly() << "\", \""
<< NdbcHandler::getFormatStringStandard() << "\", \""
<< IsmnHandler::getFormatString() << "\", \""
<< IabpHandler::getFormatString() << "\", \""
<< AeronetHandler::getFormatString() << "\", \""
<< AeronetHandler::getFormatString_v2() << "\", \""
<< AeronetHandler::getFormatString_v3() << "\"";
<< "\t\t\"-format type\" may be set to one of the following types (optional).\n"
<< "\t\t "
<< MetHandler::getFormatString() << ", "
<< LittleRHandler::getFormatString() << ", "
<< SurfradHandler::getFormatString() << ", "
<< WwsisHandler::getFormatString() << ",\n\t\t "
<< AirnowHandler::getFormatStringDailyV2() << ", "
<< AirnowHandler::getFormatStringHourlyAqObs() << ", "
<< AirnowHandler::getFormatStringHourly() << ",\n\t\t "
<< NdbcHandler::getFormatStringStandard() << ", "
<< IsmnHandler::getFormatString() << ", "
<< IabpHandler::getFormatString() << ", "
<< UscrnHandler::getFormatString() << ",\n\t\t "
<< AeronetHandler::getFormatString() << ", "
<< AeronetHandler::getFormatString_v2() << ", "
<< AeronetHandler::getFormatString_v3();

#ifdef ENABLE_PYTHON
cout << ", \"" << PythonHandler::getFormatString() << "\"";
cout << ",\n\t\t "
<< PythonHandler::getFormatString();
#endif

cout << " (optional).\n"
cout << "\n"

<< "\t\t\"-config file\" uses the specified configuration file "
<< "to generate summaries of the fields in the ASCII files (optional).\n"
Expand Down Expand Up @@ -666,6 +689,9 @@ void set_format(const StringArray & a) {
else if(IabpHandler::getFormatString() == a[0]) {
ascii_format = ASCIIFormat::IABP;
}
else if(UscrnHandler::getFormatString() == a[0]) {
ascii_format = ASCIIFormat::USCRN;
}
else if(AeronetHandler::getFormatString() == a[0]
|| AeronetHandler::getFormatString_v2() == a[0]) {
ascii_format = ASCIIFormat::Aeronet_v2;
Expand Down
Loading

0 comments on commit 30bf0b7

Please sign in to comment.