Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EPIC: OpenAPI Input table support #1271

Closed
pete-petey opened this issue Sep 13, 2021 · 1 comment
Closed

EPIC: OpenAPI Input table support #1271

pete-petey opened this issue Sep 13, 2021 · 1 comment
Assignees
Labels
epic Feature Epic (User Story) input-tables open api
Milestone

Comments

@pete-petey
Copy link
Member

No description provided.

@pete-petey pete-petey added open api epic Feature Epic (User Story) input-tables labels Sep 13, 2021
@pete-petey pete-petey added this to the Sept 2021 milestone Sep 13, 2021
@devinrsmith
Copy link
Member

devinrsmith commented Oct 4, 2021

I'd like to have support for

io.deephaven.db.v2.utils.AppendOnlyArrayBackedMutableTable#make(io.deephaven.db.tables.TableDefinition)

via table.proto/BatchTableRequest. The interesting part of this will likely be serializing TableDefinition into an appropriate grpc structure. I'm assuming we'd want to use the flight schema? (Is flight schema opaque bytes? If so, does it make sense for use to capture it in a more structured form?)

(EDIT) There was some mention of supporting creation of input tables implicitly via a doPut - but I'd like to make sure we have an explicit create accessible via BatchTableRequest. The creation via BatchTableRequest will not take any data, just a definition/schema.

Note: it would probably be good to have support for io.deephaven.db.v2.utils.KeyedArrayBackedMutableTable#make(io.deephaven.db.tables.TableDefinition, java.lang.String...) too, if it's not too much more work.

Off-topic(?): I've chatted with @nbauernfeind and I think that doPut's against existing input table tickets should trigger io.deephaven.db.util.config.MutableInputTable#add(Table newData)

niloc132 added a commit that referenced this issue Nov 8, 2021
This pull request is broken into three commits to start (with JS API client code to follow in a later pull request):

 * ClassUtil improvements
 * Proposed MutableInputTable api enhancements
 * Proposed gRPC api and implementation

ClassUtil got its lookupClass from the dataobjects ClassUtil, but this implementation isn't thread safe, and while it supports both Class.getName() and Class.getCanonicalName() array notation, it only supports getName() inner type notation. Additionally, it initializes the class on first mention, which could mean for example that an arbitrary doPut could force the server to load a class, even if not serializable. This commit still supports (ignores) generic arguments provided to the class, but otherwise delegates to common-lang3 to actually find the class, and provide the class instance without initializing it.

The MutableInputTable changes are intended to make it easier to use from gRPC on the server, and removes some details where the DHE openapi was used to provide implementation details of the input table.

The gRPC API adds a createInputTable call in table service (still has some TODOs for discussion), and provides a separate gRPC service to interact with the input table itself.

Partial #1271
This was referenced Jan 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
epic Feature Epic (User Story) input-tables open api
Projects
None yet
Development

No branches or pull requests

3 participants