Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JS API Table snapshot should support specific rows, columns #1039

Closed
niloc132 opened this issue Aug 11, 2021 · 4 comments · Fixed by #1973
Closed

JS API Table snapshot should support specific rows, columns #1039

niloc132 opened this issue Aug 11, 2021 · 4 comments · Fixed by #1973
Assignees
Milestone

Comments

@niloc132
Copy link
Member

Without this, we will either need to modify how the JS UI handles CSV download of very large tables, or will need to find another way to slice up content, like with head/tail calls before requesting a snapshot.

DoGet does not support extra metadata, so we can't just attach a custom barrage object with an index for rows and a bitset for columns.

@nbauernfeind
Copy link
Member

It would also be reasonable (and desirable) to support a filterable DoGet over DoExchange. See #925.

@nbauernfeind
Copy link
Member

Head and Tail is probably not enough. The selected rows do not need to be contiguous; if you break this up into one DoGet per contiguous range then you still can't get a consistent snapshot on a ticking table.

@nbauernfeind
Copy link
Member

Also, one parameter I want to add to the "DoGet over DoExchange" is the ability to control chunk size. Since DoGet/DoExchange are modeled as a stream of record batches, it becomes very easy to respect a user requested batch size.

@niloc132
Copy link
Member Author

This is also the issue behind input table delete row deleting all rows.

niloc132 added a commit that referenced this issue Mar 1, 2022
…#1973)

This uses the flight DoExchange call with the BarrageSnapshotRequest
payload.

Also fixes leak of rowsets in BarrageStreamGenerator, and correctly
reports the RecordBatch length for viewport snapshots.

Fixes #1678
Fixes #1039
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants