Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[backport 2.3.x] String dtype: enable in SQL IO + resolve all xfails (#60255) #60315

Merged

Conversation

jorisvandenbossche
Copy link
Member

(cherry picked from commit ba4d1cf)

Backport of #60255

@WillAyd
Copy link
Member

WillAyd commented Nov 14, 2024

Thanks for opening. I'll take a look at the failures

@WillAyd
Copy link
Member

WillAyd commented Nov 14, 2024

As far as I can tell this issue might exist at a deeper level on the backport branch. On main, I see this behavior:

>>> import pandas as pd
>>> import pandas._libs.lib as lib
>>> import numpy as np

>>> arr = np.array(["a", "b", None])
>>> lib.maybe_convert_objects(arr, convert_to_nullable_dtype=True, convert_non_numeric=True)
<StringArray>
['a', 'b', <NA>]
Length: 3, dtype: string

that same call on the backport branch yields np.nan as the missing value sentinel:

>>> lib.maybe_convert_objects(arr, convert_to_nullable_dtype=True, convert_non_numeric=True)
<StringArrayNumpySemantics>
['a', 'b', nan]
Length: 3, dtype: str

that is unexpected right?

@WillAyd WillAyd self-requested a review as a code owner November 14, 2024 18:15
@WillAyd
Copy link
Member

WillAyd commented Nov 14, 2024

I think the backport branch was missing the change in #59487 shown in the second commit here; adding that back in locally gets the tests to pass

Not sure where that got removed, but assuming during the setup of the backport branch accidentally

@jorisvandenbossche
Copy link
Member Author

Thanks for figuring that out!

There is still something related to the datetime64 resolution, will take a look at that

@WillAyd
Copy link
Member

WillAyd commented Nov 14, 2024

I think the datetime issues were caused by a very subtle backport issue. Hopefully resolved by latest commit

@WillAyd
Copy link
Member

WillAyd commented Nov 14, 2024

Is there a way to restart the failed pre-commit job? I think that is spurious

@jorisvandenbossche
Copy link
Member Author

Thanks for the fix!

Is there a way to restart the failed pre-commit job? I think that is spurious

No idea. I typically just ignore it if it times out

@jorisvandenbossche jorisvandenbossche merged commit aa8adfa into pandas-dev:2.3.x Nov 14, 2024
61 of 62 checks passed
@jorisvandenbossche jorisvandenbossche deleted the backport-60255 branch November 14, 2024 22:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants