Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(rust): Impl serde for array dtype #13168

Merged
merged 3 commits into from
Dec 27, 2023
Merged

Conversation

reswqa
Copy link
Collaborator

@reswqa reswqa commented Dec 21, 2023

This closes #13151.

@github-actions github-actions bot added enhancement New feature or an improvement of an existing feature rust Related to Rust Polars labels Dec 21, 2023
@reswqa reswqa changed the title feat(rust): Impl serde for array dtype series feat(rust): Impl serde for array dtype Dec 21, 2023
@reswqa reswqa marked this pull request as ready for review December 21, 2023 15:37
Comment on lines +38 to 42
self.arrays
.push(NullArray::new(ArrowDataType::Null, self.width).boxed());
match &mut self.validity {
Some(validity) => validity.push(false),
None => self.init_validity(),
Copy link
Collaborator Author

@reswqa reswqa Dec 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like I've found the culprit :). It seems we only consider the validity array in push_null. But unlike LargeListArray, there is no offsets array for FixedSizeListArray, causing our values to be confused with null value. After changing this, the test passed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah.. yes. It hurts that we need to box that every time, but that's what it is. :(

@ritchie46 ritchie46 merged commit 01217d2 into pola-rs:main Dec 27, 2023
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or an improvement of an existing feature rust Related to Rust Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Json serialization dataframes containing Array type fails.
2 participants