Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Optimize on non-append-only tables #1125

Closed
wjones127 opened this issue Feb 5, 2023 · 3 comments · Fixed by #1318
Closed

Support Optimize on non-append-only tables #1125

wjones127 opened this issue Feb 5, 2023 · 3 comments · Fixed by #1318
Labels
binding/rust Issues for the Rust crate enhancement New feature or request

Comments

@wjones127
Copy link
Collaborator

Description

We note here that optimize only works on append-only tables. We should get it to work for all tables.

//! *WARNING:* Currently Optimize only supports append-only workflows. Use with
//! other workflows may corrupt your table state.

Use Case

Related Issue(s)

@wjones127 wjones127 added enhancement New feature or request binding/rust Issues for the Rust crate labels Feb 5, 2023
@Matthieusalor
Copy link

Do we have more details on scenarios where the table might get corrupted besides when the table uses deletion vectors ?

@roeap
Copy link
Collaborator

roeap commented Apr 20, 2023

Don't quite remember what the specific issue was when this was created. My assumption would be that the corruption was due to the fact that we were not doing conflict resolution on failing commits. This would not have been an issue for append only tables.

Since then we added conflict resolution, but not sure if that is in the latest python release. So i think this could be resolved now.

@wjones127
Copy link
Collaborator Author

I think we can consider this fixed by #632, but we need to update the documentation for Optimize, since it currently claims it could corrupt tables.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
binding/rust Issues for the Rust crate enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants