Skip to content

Commit

Permalink
feat: Implement ExecuteQuery API for SQL support (#2280)
Browse files Browse the repository at this point in the history
* feat: publish the Cloud Bigtable ExecuteQuery API

The ExecuteQuery API will allow users to query Bigtable using SQL

PiperOrigin-RevId: 650660213

Source-Link: googleapis/googleapis@f681f79

Source-Link: googleapis/googleapis-gen@3180845
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMzE4MDg0NTQ4NzEzNjc5NDk1MmI4ZjM2NWZlNmM2ODY4OTk5ZDljMCJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* feat: publish ProtoRows Message

This is needed to parse ExecuteQuery responses

PiperOrigin-RevId: 651386373

Source-Link: googleapis/googleapis@a5be6fa

Source-Link: googleapis/googleapis-gen@d467ce8
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZDQ2N2NlODkzYTA0YzQxZTUwNDk4MzM0NmMyMTVkNDFmZDI2MzY1MCJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* Temporarily allow method level tests to pass for ExecuteQuery

I will re-add these once the client supports the appropriate behavior for readRows

Change-Id: I4b5b0861e93f0a8b97d51556c74bccfe0b1da521

* Copy SqlRowMerger from internal code

This moves things mostly as is. I will move everything to appropriate packages in a follow on (e.g. utils will move to internal). Wanted to keep the changes as minimal as possible

Renames ProtoRowsBatchMergingStateMachine to ProtoRowsMergingStateMachine and removes the obsolete ProtoRowsMergingStateMachine

This simplifies the internal code by removing support for the old api. It should not change any behavior for he ProtoRowsBatch API

Change-Id: I8e8267a9cfeb59e0f8455876d1bccdb7c3778b76

* Implement main interfaces for interacting with ResultSets

This adds the main interfaces for interacting with data:
ResultSet, StructReader, and Struct. StructReader is used
by ResultSet, SqlRow, and Struct to provide a standard set
of accessors for the data.

I'm a little uncertain about the package structure right
now. I've put interfaces that users will interact with
directly in 'models' and everything else in 'internal'.
Internal will include both internal interfaces and the
implementations of the public interfaces. Otherwise I have
kept the package structure pretty flat but happy to group
the sql stuff together more.

This includes thorough documentation for the public interfaces.

Change-Id: I4535daffc596e9355da64b5655c58a68e5e7a95f

* Implement proto to java conversion via AbstractProtoStructReader

This will be re-used for structs in the following commit. The current
design implements this at the Row/Struct level instead of ResultSet &
Struct so that result set can operate on rows of different types when we
support arrow. ResultSet will then be a light wrapper where each getType
call is passed through to the current row.

Change-Id: I21fb4cc0cd830110d5455581491c5b1861efbcb1

* Add implementation for ProtoStruct

Change-Id: Ie9d3892fe92c779a6b81d457c31b534bdce20f38

* Implement ResultSet

This also removes the metadata helpers from the ResultSet interface.
These helpers will live on ResultSetMetadata instead.

Change-Id: Iba60b42b4e62f2d72c5a21206edf716c0ad5ea73

* Implement ResultSetMetadata and use it everwhere we operate on metadata

Also fix AbstractProtoStructReaderTest which was using the wrong runner
and not executing any tests. Fixes a couple test bugs as well.

Change-Id: Ia59459f93dd38046e9ac22995191becaad6ea44e

* Create execute query callables and wire up api to data client

This uses the ExecuteQueryCallContext to pass the metadata back
to the user using a future. Note that we need to be very careful
here that errors always close this future. That is why watchdog
happens higher in the call chain that for other streaming RPCs

Also note i decided to unwrap executeExceptions so that when
possible the future throws the same exceptions as the row stream

Change-Id: Ie9dd523a980efecb2cff010a2ffd7ac52b5f8ac7

* Add statement API for ExecuteQuery requests

Change-Id: Id9113adb24419da18f6e43e47c0c73cddf4b622e

* Implement java representation of Type protobuf

The admin types in admin.v2.models.Type will be migrated to
1) share the same implementation in common, while defining a
narrow interface like SqlType 2) be renamed to SchemaType.
We aren't going to block the Sql launch on this though. It
can happen after.

Change-Id: I9dea84fe9b7b2e0b46e8ece4cb666651183dc639

* Remove unsupported query param types and add doc

Change-Id: I41a28069865df6aa688ac5bc96181aae9f18deb4

* Add clirr exclusion for new proto method. Fix dropped import

Change-Id: I0cf3bd45a5f2bb3e2ad980d4170049d69e701d7e

* Small doc update and remove obsolete TODOs

Change-Id: Ie08217d025df3ec3c97dc96268f1c4e1d20b33bf

* Minor documentation fix for executeQuery

Change-Id: I6073e41b5a5b74b91671002cb93dc42c4023f10f

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
  • Loading branch information
2 people authored and lqiu96 committed Jul 26, 2024
1 parent 4db00ea commit 5149168
Show file tree
Hide file tree
Showing 89 changed files with 43,769 additions and 368 deletions.
5 changes: 5 additions & 0 deletions google-cloud-bigtable/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -326,6 +326,11 @@
<artifactId>mockito-core</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava-testlib</artifactId>
<scope>test</scope>
</dependency>
</dependencies>

<profiles>
Expand Down
Loading

0 comments on commit 5149168

Please sign in to comment.