Skip to content

Commit

Permalink
feat: Allocateid tracing (#1488)
Browse files Browse the repository at this point in the history
* feat: Adding tracing for AllocateIds RPC

* formatting

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
  • Loading branch information
jimit-j-shah and gcf-owl-bot[bot] committed Aug 5, 2024
1 parent 24779b9 commit 552d04e
Show file tree
Hide file tree
Showing 6 changed files with 41 additions and 107 deletions.
106 changes: 5 additions & 101 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ If you are using Maven with [BOM][libraries-bom], add this to your pom.xml file:
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>libraries-bom</artifactId>
<version>26.41.0</version>
<version>26.40.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
Expand Down Expand Up @@ -50,20 +50,20 @@ If you are using Maven without the BOM, add this to your dependencies:
If you are using Gradle 5.x or later, add this to your dependencies:

```Groovy
implementation platform('com.google.cloud:libraries-bom:26.42.0')
implementation platform('com.google.cloud:libraries-bom:26.40.0')
implementation 'com.google.cloud:google-cloud-datastore'
```
If you are using Gradle without BOM, add this to your dependencies:

```Groovy
implementation 'com.google.cloud:google-cloud-datastore:2.20.2'
implementation 'com.google.cloud:google-cloud-datastore:2.20.0'
```

If you are using SBT, add this to your dependencies:

```Scala
libraryDependencies += "com.google.cloud" % "google-cloud-datastore" % "2.20.2"
libraryDependencies += "com.google.cloud" % "google-cloud-datastore" % "2.20.0"
```
<!-- {x-version-update-end} -->

Expand Down Expand Up @@ -210,102 +210,6 @@ running on Compute Engine or from your own desktop. To run the example on App En
the code from the main method to your application's servlet class and change the print statements to
display on your webpage.

gRPC Java Datastore Client User Guide
-------
In this feature launch, the [Java Datastore client](https://github.com/googleapis/java-datastore) now offers gRPC as a transport layer option with experimental support. Using [gRPC connection pooling](https://grpc.io/docs/guides/performance/) enables distributing RPCs over multiple connections which may improve performance.

#### Download Instructions
Instructions:
1. Clone the grpc-experimental branch from GitHub:
```python
git clone -b grpc-experimental https://github.com/googleapis/java-datastore.git
```
2. Run the following commands to build the library:
```python
# Go to the directory the code was downloaded to
cd java-datastore/

# Build the library
mvn clean install -DskipTests=true
```
3. Add the following dependency to your project:
```xml
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-datastore</artifactId>
<version>2.20.0-grpc-experimental-1-SNAPSHOT</version>
</dependency>
```

#### How to Use
To opt-in to the gRPC transport behavior, simply add the below line of code (`setTransportOptions`) to your Datastore client instantiation.

Example:
```java
DatastoreOptions datastoreOptions =
DatastoreOptions.newBuilder()
.setProjectId("my-project")
.setDatabaseId("my-database")
.setTransportOptions(GrpcTransportOptions.newBuilder().build())
.build();
```
Setting the transport options explicitly to `GrpcTransportOptions` will signal the client to use gRPC instead of HTTP when making calls to the server.

To revert back to the existing stable behavior and transport, simply remove the transport options line or replace it with `HttpTransportOptions`. Please note this will require an application rebuild and restart.
Example:
```java
// will default to existing HTTP transport behavior
DatastoreOptions datastoreOptions = DatastoreOptions.newBuilder()
.setProjectId("my-project")
.setDatabaseId("my-database")
.build();

// will also default to existing HTTP transport behavior
DatastoreOptions datastoreOptions =
DatastoreOptions.newBuilder()
.setProjectId("my-project")
.setDatabaseId("my-database")
.setTransportOptions(HttpTransportOptions.newBuilder()
.setConnectTimeout(1000)
.build()).build();
```

Note: client instantiations that already use `setTransportOptions` with `HttpTransportOptions` will continue to have the same behavior. Only transports that are explicitly set to gRPC will change.

#### Verify Datastore Transport Options Type
To verify which type of TransportOptions you have successfully configured, you can use the below lines of code to compare transport options type:
```java
// checks if using gRPC transport options
boolean isGRPC = datastore.getOptions().getTransportOptions() instanceof GrpcTransportOptions;

// checks if using HTTP transport options
boolean isHTTP = datastore.getOptions().getTransportOptions() instanceof HTTPTransportOptions;
```

#### New Features
There are new gRPC specific features available to use in this update.

##### Channel Pooling
To customize the number of channels your client uses, you can update the channel provider in the DatastoreOptions.
See [ChannelPoolSettings](https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.grpc.ChannelPoolSettings) and [Performance Best Practices](https://grpc.io/docs/guides/performance/) for more information on channel pools and best practices for performance.

Example:
```java
InstantiatingGrpcChannelProvider channelProvider =
DatastoreSettings.defaultGrpcTransportProviderBuilder()
.setChannelPoolSettings(
ChannelPoolSettings.builder()
.setInitialChannelCount(MIN_VAL)
.setMaxChannelCount(MAX_VAL)
.build())
.build();

DatastoreOptions options = DatastoreOptions.newBuilder()
.setProjectId("my-project")
.setChannelProvider(channelProvider)
.setTransportOptions(GrpcTransportOptions.newBuilder().build())
.build();
```
Testing
-------

Expand Down Expand Up @@ -480,7 +384,7 @@ Java is a registered trademark of Oracle and/or its affiliates.
[kokoro-badge-link-5]: http://storage.googleapis.com/cloud-devrel-public/java/badges/java-datastore/java11.html
[stability-image]: https://img.shields.io/badge/stability-stable-green
[maven-version-image]: https://img.shields.io/maven-central/v/com.google.cloud/google-cloud-datastore.svg
[maven-version-link]: https://central.sonatype.com/artifact/com.google.cloud/google-cloud-datastore/2.20.2
[maven-version-link]: https://central.sonatype.com/artifact/com.google.cloud/google-cloud-datastore/2.20.0
[authentication]: https://github.com/googleapis/google-cloud-java#authentication
[auth-scopes]: https://developers.google.com/identity/protocols/oauth2/scopes
[predefined-iam-roles]: https://cloud.google.com/iam/docs/understanding-roles#predefined_roles
Expand Down
2 changes: 1 addition & 1 deletion google-cloud-datastore/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
</parent>
<properties>
<site.installationModule>google-cloud-datastore</site.installationModule>
<opentelemetry.version>1.37.0</opentelemetry.version>
<opentelemetry.version>1.38.0</opentelemetry.version>
</properties>
<dependencyManagement>
<dependencies>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -327,8 +327,10 @@ public List<Key> allocateId(IncompleteKey... keys) {

private com.google.datastore.v1.AllocateIdsResponse allocateIds(
final com.google.datastore.v1.AllocateIdsRequest requestPb) {
Span span = traceUtil.startSpan(TraceUtil.SPAN_NAME_ALLOCATEIDS);
try (Scope scope = traceUtil.getTracer().withSpan(span)) {
com.google.cloud.datastore.telemetry.TraceUtil.Span span =
otelTraceUtil.startSpan(
com.google.cloud.datastore.telemetry.TraceUtil.SPAN_NAME_ALLOCATE_IDS);
try (com.google.cloud.datastore.telemetry.TraceUtil.Scope ignored = span.makeCurrent()) {
return RetryHelper.runWithRetries(
new Callable<com.google.datastore.v1.AllocateIdsResponse>() {
@Override
Expand All @@ -340,10 +342,10 @@ public com.google.datastore.v1.AllocateIdsResponse call() throws DatastoreExcept
EXCEPTION_HANDLER,
getOptions().getClock());
} catch (RetryHelperException e) {
span.setStatus(Status.UNKNOWN.withDescription(e.getMessage()));
span.end(e);
throw DatastoreException.translateAndThrow(e);
} finally {
span.end(TraceUtil.END_SPAN_OPTIONS);
span.end();
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@
public class TraceUtil {
private final Tracer tracer = Tracing.getTracer();
private static final TraceUtil traceUtil = new TraceUtil();
static final String SPAN_NAME_ALLOCATEIDS = "CloudDatastoreOperation.allocateIds";
static final String SPAN_NAME_TRANSACTION = "CloudDatastoreOperation.readWriteTransaction";
static final String SPAN_NAME_BEGINTRANSACTION = "CloudDatastoreOperation.beginTransaction";
static final String SPAN_NAME_COMMIT = "CloudDatastoreOperation.commit";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ public interface TraceUtil {
static final String ENABLE_TRACING_ENV_VAR = "DATASTORE_ENABLE_TRACING";
static final String LIBRARY_NAME = "com.google.cloud.datastore";
static final String SPAN_NAME_LOOKUP = "Lookup";
static final String SPAN_NAME_ALLOCATE_IDS = "AllocateIds";
static final String SPAN_NAME_COMMIT = "Commit";
static final String SPAN_NAME_RUN_QUERY = "RunQuery";
static final String SPAN_NAME_RUN_AGGREGATION_QUERY = "RunAggregationQuery";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
package com.google.cloud.datastore.it;

import static com.google.cloud.datastore.aggregation.Aggregation.count;
import static com.google.cloud.datastore.telemetry.TraceUtil.SPAN_NAME_ALLOCATE_IDS;
import static com.google.cloud.datastore.telemetry.TraceUtil.SPAN_NAME_BEGIN_TRANSACTION;
import static com.google.cloud.datastore.telemetry.TraceUtil.SPAN_NAME_COMMIT;
import static com.google.cloud.datastore.telemetry.TraceUtil.SPAN_NAME_LOOKUP;
Expand All @@ -40,7 +41,9 @@
import com.google.cloud.datastore.Datastore;
import com.google.cloud.datastore.DatastoreOptions;
import com.google.cloud.datastore.Entity;
import com.google.cloud.datastore.IncompleteKey;
import com.google.cloud.datastore.Key;
import com.google.cloud.datastore.KeyFactory;
import com.google.cloud.datastore.Query;
import com.google.cloud.datastore.QueryResults;
import com.google.cloud.datastore.ReadOption;
Expand Down Expand Up @@ -574,6 +577,31 @@ public void lookupTraceTest() throws Exception {
fetchAndValidateTrace(customSpanContext.getTraceId(), SPAN_NAME_LOOKUP);
}

@Test
public void allocateIdsTraceTest() throws Exception {
assertNotNull(customSpanContext);

Span rootSpan = getNewRootSpanWithContext();
try (Scope ignored = rootSpan.makeCurrent()) {
String kind1 = "kind1";
KeyFactory keyFactory = datastore.newKeyFactory().setKind(kind1);
IncompleteKey pk1 = keyFactory.newKey();
Key key1 = datastore.allocateId(pk1);
assertEquals(key1.getProjectId(), pk1.getProjectId());
assertEquals(key1.getNamespace(), pk1.getNamespace());
assertEquals(key1.getAncestors(), pk1.getAncestors());
assertEquals(key1.getKind(), pk1.getKind());
assertTrue(key1.hasId());
assertFalse(key1.hasName());
assertEquals(Key.newBuilder(pk1, key1.getId()).build(), key1);
} finally {
rootSpan.end();
}
waitForTracesToComplete();

fetchAndValidateTrace(customSpanContext.getTraceId(), SPAN_NAME_ALLOCATE_IDS);
}

@Test
public void commitTraceTest() throws Exception {
assertNotNull(customSpanContext);
Expand Down

0 comments on commit 552d04e

Please sign in to comment.