Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adapt exception handling logic in CFTimeIndex.__sub__ and __rsub__ #5006

Merged

Conversation

spencerkclark
Copy link
Member

@spencerkclark spencerkclark commented Mar 7, 2021

The exception that was raised in pandas when a datetime.timedelta object outside the range that could be expressed in units of nanoseconds was passed to the pandas.TimedeltaIndex constructor changed from an OverflowError to an OutOfBoundsTimedelta error in the development version of pandas. This PR adjusts our exception handling logic in CFTimeIndex.__sub__ and CFTimeIndex.__rsub__ to account for this.

Previous versions of pandas:

>>> import pandas as pd; from datetime import timedelta
>>> pd.TimedeltaIndex([timedelta(days=300 * 365)])
Traceback (most recent call last):
 File "pandas/_libs/tslibs/timedeltas.pyx", line 263, in pandas._libs.tslibs.timedeltas.array_to_timedelta64
TypeError: Expected unicode, got datetime.timedelta

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/Users/spencer/Software/miniconda3/envs/xarray-tests/lib/python3.7/site-packages/pandas/core/indexes/timedeltas.py", line 157, in __new__
   data, freq=freq, unit=unit, dtype=dtype, copy=copy
 File "/Users/spencer/Software/miniconda3/envs/xarray-tests/lib/python3.7/site-packages/pandas/core/arrays/timedeltas.py", line 216, in _from_sequence
   data, inferred_freq = sequence_to_td64ns(data, copy=copy, unit=unit)
 File "/Users/spencer/Software/miniconda3/envs/xarray-tests/lib/python3.7/site-packages/pandas/core/arrays/timedeltas.py", line 926, in sequence_to_td64ns
   data = objects_to_td64ns(data, unit=unit, errors=errors)
 File "/Users/spencer/Software/miniconda3/envs/xarray-tests/lib/python3.7/site-packages/pandas/core/arrays/timedeltas.py", line 1036, in objects_to_td64ns
   result = array_to_timedelta64(values, unit=unit, errors=errors)
 File "pandas/_libs/tslibs/timedeltas.pyx", line 268, in pandas._libs.tslibs.timedeltas.array_to_timedelta64
 File "pandas/_libs/tslibs/timedeltas.pyx", line 221, in pandas._libs.tslibs.timedeltas.convert_to_timedelta64
 File "pandas/_libs/tslibs/timedeltas.pyx", line 166, in pandas._libs.tslibs.timedeltas.delta_to_nanoseconds
OverflowError: Python int too large to convert to C long

Development version of pandas:

>>> import pandas as pd; from datetime import timedelta
>>> pd.TimedeltaIndex([timedelta(days=300 * 365)])
Traceback (most recent call last):
 File "pandas/_libs/tslibs/timedeltas.pyx", line 348, in pandas._libs.tslibs.timedeltas.array_to_timedelta64
TypeError: Expected unicode, got datetime.timedelta

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
 File "pandas/_libs/tslibs/timedeltas.pyx", line 186, in pandas._libs.tslibs.timedeltas.delta_to_nanoseconds
OverflowError: Python int too large to convert to C long

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/Users/spencer/software/pandas/pandas/core/indexes/timedeltas.py", line 161, in __new__
   tdarr = TimedeltaArray._from_sequence_not_strict(
 File "/Users/spencer/software/pandas/pandas/core/arrays/timedeltas.py", line 270, in _from_sequence_not_strict
   data, inferred_freq = sequence_to_td64ns(data, copy=copy, unit=unit)
 File "/Users/spencer/software/pandas/pandas/core/arrays/timedeltas.py", line 970, in sequence_to_td64ns
   data = objects_to_td64ns(data, unit=unit, errors=errors)
 File "/Users/spencer/software/pandas/pandas/core/arrays/timedeltas.py", line 1079, in objects_to_td64ns
   result = array_to_timedelta64(values, unit=unit, errors=errors)
 File "pandas/_libs/tslibs/timedeltas.pyx", line 362, in pandas._libs.tslibs.timedeltas.array_to_timedelta64
 File "pandas/_libs/tslibs/timedeltas.pyx", line 353, in pandas._libs.tslibs.timedeltas.array_to_timedelta64
 File "pandas/_libs/tslibs/timedeltas.pyx", line 306, in pandas._libs.tslibs.timedeltas.convert_to_timedelta64
 File "pandas/_libs/tslibs/timedeltas.pyx", line 189, in pandas._libs.tslibs.timedeltas.delta_to_nanoseconds
pandas._libs.tslibs.conversion.OutOfBoundsTimedelta: Python int too large to convert to C long

… and __rsub__

The latest version of pandas raises an OutOfBoundsTimedelta error instead
of an Overflow error.
@spencerkclark spencerkclark merged commit b610a3c into pydata:master Mar 7, 2021
@spencerkclark spencerkclark deleted the fix-cftimeindex-sub-error branch March 7, 2021 13:22
dcherian added a commit to dcherian/xarray that referenced this pull request Mar 8, 2021
* upstream/master: (46 commits)
  pin netCDF4=1.5.3 in min-all-deps (pydata#4982)
  fix matplotlib errors for single level discrete colormaps (pydata#4256)
  Adapt exception handling in CFTimeIndex.__sub__ and __rsub__ (pydata#5006)
  Update options.py (pydata#5000)
  Adjust tests to use updated pandas syntax for offsets (pydata#4537)
  add a combine_attrs parameter to Dataset.merge (pydata#4895)
  Support for dask.graph_manipulation (pydata#4965)
  raise on passing axis to Dataset.reduce methods (pydata#4940)
  Whatsnew for 0.17.1 (pydata#4963)
  Refinements to how-to-release (pydata#4964)
  DOC: add example for reindex (pydata#4956)
  DOC: rm np import (pydata#4949)
  Add 0.17.0 release notes (pydata#4953)
  document update as inplace (pydata#4932)
  bump the dependencies (pydata#4942)
  Upstream CI: limit runtime (pydata#4946)
  typing for numpy 1.20 (pydata#4878)
  Use definition of DTypeLike from Numpy if found (pydata#4941)
  autoupdate mypy (pydata#4943)
  Add DataArrayCoarsen.reduce and DatasetCoarsen.reduce methods (pydata#4939)
  ...
dcherian added a commit to dcherian/xarray that referenced this pull request Mar 18, 2021
…indow

* upstream/master:
  add polyval to polyfit see also (pydata#5020)
  mention map_blocks in the docstring of apply_ufunc (pydata#5011)
  Switch backend API to v2 (pydata#4989)
  WIP: add new backend api documentation (pydata#4810)
  pin netCDF4=1.5.3 in min-all-deps (pydata#4982)
  fix matplotlib errors for single level discrete colormaps (pydata#4256)
  Adapt exception handling in CFTimeIndex.__sub__ and __rsub__ (pydata#5006)
  Update options.py (pydata#5000)
  Adjust tests to use updated pandas syntax for offsets (pydata#4537)
  add a combine_attrs parameter to Dataset.merge (pydata#4895)
  Support for dask.graph_manipulation (pydata#4965)
  raise on passing axis to Dataset.reduce methods (pydata#4940)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

⚠️ Nightly upstream-dev CI failed ⚠️
3 participants