Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pandas 2.0 support #300

Merged
merged 6 commits into from
Apr 6, 2023
Merged

pandas 2.0 support #300

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3b8cc7f
pandas 2.0 support
ocefpaf Apr 4, 2023
6a34222
avoid https://github.com/jupyter/nbconvert/issues/1970
ocefpaf Apr 4, 2023
0e3c70b
don't rely on internal func
ocefpaf Apr 4, 2023
b9cc333
test up to 1.5.3
ocefpaf Apr 4, 2023
517f20e
tested range
ocefpaf Apr 5, 2023
a2631e3
let's try to reduce the scope of the warn-errors
ocefpaf Apr 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 11 additions & 5 deletions erddapy/core/url.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

import httpx
import pytz
from pandas._libs.tslibs.parsing import parse_time_string
from pandas import to_datetime
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gmaze I should've done this a long time ago :-)

Thanks for the ping in

this looks like a recurrent issue for erddapy (#128 #34 #240 )

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome, thanks for the quick fix !

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about extracting the import into a function and then tweaking execution on import errors?

Then we could probably revert the new pin.

def dang_datetimes(date_time: Union[datetime, str], dayfirst: bool=False, yearfirst: bool=False):
    try:
        from pandas import to_datetime

        return to_datetime(datetime, dayfirst=dayfirst, yearfirst=yearfalse).to_pydatetime()
    except ImportError:
        from pandas._libs.tslibs.parsing import parse_time_string

        return parse_time_string(date_time)[0]

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then we could probably revert the new pin.

The new pin (>=1.5.3) is mostly b/c I was lazy and creating old envs to test but I'm 99% sure we can go as low as the old one (>=0.20.3) with the current code. I can get back to this in the afternoon and do more tests, hold on...


ListLike = Union[List[str], Tuple[str]]
OptionalStr = Optional[str]
Expand Down Expand Up @@ -113,7 +113,11 @@ def _check_substrings(constraint):
return any([True for substring in substrings if substring in str(constraint)])


def parse_dates(date_time: Union[datetime, str]) -> float:
def parse_dates(
date_time: Union[datetime, str],
dayfirst=False,
yearfirst=False,
) -> float:
"""
Parse dates to ERDDAP internal format.

Expand All @@ -123,9 +127,11 @@ def parse_dates(date_time: Union[datetime, str]) -> float:

"""
if isinstance(date_time, str):
# pandas returns a tuple with datetime, dateutil, and string representation.
# we want only the datetime obj.
parse_date_time = parse_time_string(date_time)[0]
parse_date_time = to_datetime(
date_time,
dayfirst=dayfirst,
yearfirst=yearfirst,
).to_pydatetime()
else:
parse_date_time = date_time

Expand Down
3 changes: 2 additions & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,14 +34,15 @@ dependencies = {file = ["requirements.txt"]}
[tool.setuptools_scm]
write_to = "erddapy/_version.py"
write_to_template = "__version__ = '{version}'"
tag_regex = "^(?P<prefix>v)?(?P<version>[^\\+]+)(?P<suffix>.*)?$"

[tool.pytest.ini_options]
markers = [
"web: marks tests require connection (deselect with '-m \"not web\"')",
"serial: marks tests that cannot be run in parallel (deselect with '-m \"not serial\"')",
]
filterwarnings = [
"error",
"error:::erddapy.*",
"ignore::UserWarning",
"ignore::RuntimeWarning",
]
Expand Down
1 change: 1 addition & 0 deletions requirements-dev.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ geopandas
interrogate
joblib
jupyter
nbconvert<7.3.0
ocefpaf marked this conversation as resolved.
Show resolved Hide resolved
nbsphinx
netcdf4
pendulum>=2.0.1
Expand Down
2 changes: 1 addition & 1 deletion requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
httpx
pandas>=0.20.3
pandas>=0.25.2,<3
ocefpaf marked this conversation as resolved.
Show resolved Hide resolved
pytz