Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use LevelHistogram throughout Parquet metadata #6134

Closed
etseidl opened this issue Jul 26, 2024 · 1 comment · Fixed by #6135
Closed

Use LevelHistogram throughout Parquet metadata #6134

etseidl opened this issue Jul 26, 2024 · 1 comment · Fixed by #6135
Labels
enhancement Any new improvement worthy of a entry in the changelog parquet Changes to the parquet crate

Comments

@etseidl
Copy link
Contributor

etseidl commented Jul 26, 2024

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Related to #6129. #6105 added a new LevelHistogram struct to represent histograms recently added to the spec. There remain places where this histogram data is still represented with a Vec<i64>.

Describe the solution you'd like
Replace the remaining bare vectors (in PageIndex and ColumnIndexBuilder with the new struct.

Describe alternatives you've considered
The vectors in ColumnIndexBuilder could be left as is since they are closer to the thrift representation, but I think it's cleaner for all level histograms outside actual thrift structs to share the same interface.

Additional context
See #6105 (comment)

@etseidl etseidl added the enhancement Any new improvement worthy of a entry in the changelog label Jul 26, 2024
@alamb alamb added the parquet Changes to the parquet crate label Aug 31, 2024
@alamb
Copy link
Contributor

alamb commented Aug 31, 2024

label_issue.py automatically added labels {'parquet'} from #6135

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Any new improvement worthy of a entry in the changelog parquet Changes to the parquet crate
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants