Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API: silent overflow in Series(bigints, dtype="int8") #40114

Conversation

jbrockmendel
Copy link
Member

  • closes #xxxx
  • tests added / passed
  • Ensure all linting tests pass, see here for how to run them
  • whatsnew entry

This is made complicated by a) #40110 and b) MaskedArray behavior, which i need to check but may be changeable.

@jorisvandenbossche
Copy link
Member

So the current behaviour is to wrap around, which follows numpy:

In [5]: pd.Series([1, 1000], dtype="int8")
Out[5]: 
0     1
1   -24
dtype: int8

In [6]: np.array([1, 1000], dtype="int8")
Out[6]: array([  1, -24], dtype=int8)

Personally, I think it would be good to be more strict (and only allow safe casting by default) in the constructors, as you are doing in this PR.
But I think this could (relatively easily) be done with a deprecation warning that it will raise in the future?

@jorisvandenbossche jorisvandenbossche changed the title BUG: silent overflow in Series(bigints, dtype="int8") API: silent overflow in Series(bigints, dtype="int8") Mar 1, 2021
@jbrockmendel
Copy link
Member Author

But I think this could (relatively easily) be done with a deprecation warning that it will raise in the future?

It could, but leaving it until 2.0 will block most remaining simplification of the construction/casting code, which is bound to uncover more bugs.

We do explicitly catch OverflowError when passing negative integers and specifying a uint dtype, so there's a precedent for not matching numpy's behavior.

@jreback jreback added the Dtype Conversions Unexpected or buggy dtype conversions label Mar 10, 2021
@jbrockmendel
Copy link
Member Author

mothballing to clear the queue

@jbrockmendel jbrockmendel added the Mothballed Temporarily-closed PR the author plans to return to label Mar 23, 2021
@jbrockmendel jbrockmendel deleted the bug-maybe_cast_to_integer_array branch May 30, 2021 16:37
@jbrockmendel jbrockmendel removed the Mothballed Temporarily-closed PR the author plans to return to label May 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Dtype Conversions Unexpected or buggy dtype conversions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants