Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change default serialization format for serialize/deserialize methods to binary #17108

Closed
stinodego opened this issue Jun 21, 2024 · 2 comments · Fixed by #17223
Closed

Change default serialization format for serialize/deserialize methods to binary #17108

stinodego opened this issue Jun 21, 2024 · 2 comments · Fixed by #17223
Assignees
Labels
accepted Ready for implementation enhancement New feature or an improvement of an existing feature P-goal Priority: aligns with long-term Polars goals
Milestone

Comments

@stinodego
Copy link
Member

stinodego commented Jun 21, 2024

Our serialization methods currently all work with JSON strings. This is useful as it's human readable and certain protocols will require JSON structures, but we should also have a more optimized binary serialization format.

bincode seems like a good candidate, since it's already in our dependencies.

The intended API would be to add a format parameter to (de)serialization methods with the options "json" and "bincode". Other useful formats could potentially be added in the future.

@stinodego stinodego added enhancement New feature or an improvement of an existing feature accepted Ready for implementation P-goal Priority: aligns with long-term Polars goals labels Jun 21, 2024
@github-project-automation github-project-automation bot moved this to Ready in Backlog Jun 21, 2024
@stinodego stinodego moved this from Ready to Next in Backlog Jun 21, 2024
@stinodego stinodego self-assigned this Jun 21, 2024
@ritchie46
Copy link
Member

All for this. Though I do think we should mark the other formats as unstable.

@stinodego stinodego moved this from Next to In progress in Backlog Jun 25, 2024
@stinodego stinodego changed the title Add bincode serialization format to serialize/deserialize methods Change default serialization format for serialize/deserialize methods to binary Jun 26, 2024
@stinodego stinodego added this to the 1.0.0 milestone Jun 26, 2024
@stinodego
Copy link
Member Author

Update: we will use the ciborium crate for binary serialization:
https://crates.io/crates/ciborium

The associated format will be named "binary". And this will become the default format.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accepted Ready for implementation enhancement New feature or an improvement of an existing feature P-goal Priority: aligns with long-term Polars goals
Projects
Archived in project
2 participants