Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for Iceberg #1028

Open
andygrove opened this issue Oct 22, 2024 · 1 comment
Open

Add support for Iceberg #1028

andygrove opened this issue Oct 22, 2024 · 1 comment
Labels
enhancement New feature or request
Milestone
0.4.0

Comments

@andygrove
Copy link
Member

andygrove commented Oct 22, 2024 •
edited
Loading

What is the problem the feature request solves?

This issue is to track the status of Iceberg support in Comet.

There is a PR open against Iceberg to make use of Comet's native Parquet decoding:

apache/iceberg#9841

Once this is merged and a new Iceberg release is available, Comet will be updated to support reading from Iceberg.

Describe the potential solution

No response

Additional context

It maybe possible to read from Iceberg today by enabling spark.comet.sparkToColumnar.enabled and adding the Iceberg scan operator to spark.comet.sparkToColumnar.supportedOperatorList but this has not been tested, and may not perform well.
@andygrove andygrove added the enhancement New feature or request label Oct 22, 2024
@andygrove andygrove added this to the 0.4.0 milestone Oct 22, 2024
@viirya
Copy link
Member

viirya commented Oct 23, 2024

Related issue: #974

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
0.4.0
Development

No branches or pull requests
2 participants
@viirya @andygrove