Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update NOAA Request function #332

Merged
merged 9 commits into from
Jul 15, 2024
51 changes: 37 additions & 14 deletions mhkit/tidal/io/noaa.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@
This module provides functions to fetch, process, and read NOAA (National
Oceanic and Atmospheric Administration) current data directly from the
NOAA Tides and Currents API (https://api.tidesandcurrents.noaa.gov/api/prod/). It
supports loading data into a pandas DataFrame, handling data in XML and
supports loading data into a pandas DataFrame, handling data in XML and
JSON formats, and writing data to a JSON file.

Functions:
----------
request_noaa_data(station, parameter, start_date, end_date, proxy=None,
request_noaa_data(station, parameter, start_date, end_date, proxy=None,
write_json=None):
Loads NOAA current data from the API into a pandas DataFrame,
Loads NOAA current data from the API into a pandas DataFrame,
with optional support for proxy settings and writing data to a JSON
file.

Expand Down Expand Up @@ -56,9 +56,11 @@ def request_noaa_data(
Parameters
----------
station : str
NOAA current station number (e.g. 'cp0101')
NOAA current station number (e.g. 'cp0101', "s08010", "9446484")
parameter : str
NOAA paramter (e.g. '' for Discharge, cubic feet per second)
NOAA paramter (e.g. "currents", "salinity", "water_level", "water_temperature",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

paramter => parameter

"air_temperature", "wind", "air_pressure")
https://api.tidesandcurrents.noaa.gov/api/prod/
start_date : str
Start date in the format yyyyMMdd
end_date : str
Expand Down Expand Up @@ -158,26 +160,42 @@ def request_noaa_data(
end_date = date_list[i + 1].strftime("%Y%m%d")

api_query = f"begin_date={start_date}&end_date={end_date}&station={station}&product={parameter}&units=metric&time_zone=gmt&application=web_services&format=xml"
# Add datum to water level inquiries
if parameter == "water_level":
api_query += "&datum=MLLW"
data_url = f"https://tidesandcurrents.noaa.gov/api/datagetter?{api_query}"

print("Data request URL: ", data_url)
print(f"Data request URL: {data_url}\n")

# Get response
try:
response = requests.get(url=data_url, proxies=proxy)
response.raise_for_status()
except requests.exceptions.HTTPError as err:
print(f"HTTP error occurred: {err}")
continue
except requests.exceptions.RequestException as err:
print(f"Error occurred: {err}")
continue
# Catch non-exception errors
if "error" in response.content.decode():
raise Exception(response.content.decode())
except Exception as err:
if err.__class__ == requests.exceptions.HTTPError:
print(f"HTTP error occurred: {err}")
print(f"Error message: {response.content.decode()}\n")
continue
elif err.__class__ == requests.exceptions.RequestException:
print(f"Requests error occurred: {err}")
print(f"Error message: {response.content.decode()}\n")
continue
else:
print(f"Requests error occurred: {err}\n")
continue

# Convert to DataFrame and save in data_frames list
df, metadata = _xml_to_dataframe(response)
data_frames.append(df)

# Concatenate all DataFrames
data = pd.concat(data_frames, ignore_index=False)
if data_frames:
data = pd.concat(data_frames, ignore_index=False)
else:
raise ValueError("No data retrieved.")

# Remove duplicated date values
data = data.loc[~data.index.duplicated()]
Expand Down Expand Up @@ -236,7 +254,12 @@ def _xml_to_dataframe(response):
df.drop_duplicates(inplace=True)

# Convert data to float
df[["d", "s"]] = df[["d", "s"]].apply(pd.to_numeric)
cols = list(df.columns)
for var in cols:
try:
df[var] = df[var].apply(pd.to_numeric)
except ValueError:
pass

return df, metadata

Expand Down
Loading