Skip to content

Java libraries and HBase client extensions for accessing Google Cloud Bigtable

License

Notifications You must be signed in to change notification settings

mutianf/java-bigtable-hbase

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Maven Stack Overflow

Google Cloud Bigtable is Google's NoSQL Big Data database service. It's the same database that powers many core Google services, including Search, Analytics, Maps, and Gmail.

Bigtable is designed to handle massive workloads at consistent low latency and high throughput, so it's a great choice for both operational and analytical applications, including IoT, user analytics, and financial data analysis.

Bigtable provisions and scales to hundreds of petabytes automatically, and can smoothly handle millions of operations per second. Changes to the deployment configuration are immediate, so there is no downtime during reconfiguration.

Bigtable integrates easily with popular Big Data tools like Hadoop, as well as Google Cloud Platform products like Cloud Dataflow and Dataproc. Plus, Bigtable supports the open-source, industry-standard HBase API, which makes it easy for development teams to get started.

Note: Please use google-cloud-bigtable to access Bigtable APIs instead of bigtable-core-client. These artifacts are meant to wrap HBase over Bigtable API.

Project setup, installation, and configuration

Prerequisites

Using the Java client

  • Add the appropriate Cloud Bigtable artifact dependencies to your Maven project.
    • bigtable-hbase-1.x: use for standalone applications where you are in control of your dependencies.
    • bigtable-hbase-1.x-hadoop: use in hadoop environments.
    • bigtable-hbase-1.x-mapreduce: use for map/reduce utilities.
    • bigtable-hbase-1.x-shaded: use in environments (other than hadoop) that require older versions of protobuf, guava, etc.
    • bigtable-hbase-2.x: use for standalone applications where you are in control of your dependencies. This includes an HBase async client.
    • bigtable-hbase-2.x-hadoop: use in hadoop environments.
    • bigtable-hbase-2.x-shaded: use in environments (other than hadoop) that require older versions of protobuf, guava, etc.

Maven:

<dependency>
  <groupId>com.google.cloud.bigtable</groupId>
  <artifactId>bigtable-hbase-1.x</artifactId>
  <version>1.18.0</version>
</dependency>

Gradle:

compile 'com.google.cloud.bigtable:bigtable-hbase-1.x:1.18.0'

SBT:

libraryDependencies += "com.google.cloud.bigtable" % "bigtable-hbase-1.x" % "1.18.0"

OpenCensus Integration

The Bigtable HBase Client supports OpenCensus telemetry, specifically exporting gRPC metrics to Stats and supporting Tracing.

Stats

The code example below shows how to enable metrics. For more details, see the gRPC Java Guide.

Maven Setup

If you are not using the shaded Bigtable HBase Client artifact, you need to define the OpenCensus dependencies.

<!-- OpenCensus dependencies -->
<dependency>
    <groupId>com.google.cloud.bigtable</groupId>
    <artifactId>bigtable-hbase-1.x</artifactId>
    <version>1.18.0</version>
</dependency>
<dependency>
    <groupId>io.opencensus</groupId>
    <artifactId>opencensus-impl</artifactId>
    <version>0.24.0</version>
    <scope>runtime</scope>
</dependency>
<dependency>
    <groupId>io.opencensus</groupId>
    <artifactId>opencensus-exporter-stats-stackdriver</artifactId>
    <version>0.24.0</version>
    <exclusions>
        <exclusion>
            <groupId>io.grpc</groupId>
            <artifactId>*</artifactId>
        </exclusion>
        <exclusion>
            <groupId>com.google.auth</groupId>
            <artifactId>*</artifactId>
        </exclusion>
    </exclusions>
</dependency>

If you are using the shaded Bigtable HBase Client artifact, then the OpenCensus dependencies are embedded in the shaded artifact; i.e. nothing additional for you to do.

<!-- OpenCensus dependencies -->
<dependency>
    <groupId>com.google.cloud.bigtable</groupId>
    <artifactId>bigtable-hbase-1.x-shaded</artifactId>
    <version>1.18.0</version>
</dependency>
Java Example
// For the non-shaded client, remove the package prefix "com.google.bigtable.repackaged."
import com.google.bigtable.repackaged.io.opencensus.contrib.grpc.metrics.RpcViews;
import com.google.bigtable.repackaged.io.opencensus.exporter.stats.stackdriver.StackdriverStatsConfiguration;
import com.google.bigtable.repackaged.io.opencensus.exporter.stats.stackdriver.StackdriverStatsExporter;

import java.io.IOException;

public class OpenCensusExample {
    String projectId = "your-project-id";

    void setupStatsExport() throws Exception {
        // Option 1: Automatic Configuration (from GCP Resources only):
        // If you are running from a GCP Resource (e.g. a GCE VM), the Stackdriver metrics are automatically
        // configured to upload to your resource.
        // For examples of monitored resources, see here: https://cloud.google.com/monitoring/api/resources
        StackdriverStatsExporter.createAndRegister();

        // Then register your gRPC views in OpenCensus.
        RpcViews.registerClientGrpcViews();


        // Option 2: Manual Configuration
        // If you are not running from a GCP Resource (e.g. if you are running on-prem), then you should
        // configure the monitored resource yourself.
        // Use the code snippet below as a starting example.
        // For examples of monitored resources, see here: https://cloud.google.com/monitoring/api/resources
        StackdriverStatsExporter.createAndRegister(
                StackdriverStatsConfiguration.builder()
                        .setProjectId(projectId)
                        // This example uses generic_node as the MonitoredResource, with your host name as the node ID.
                        .setMonitoredResource(MonitoredResource.newBuilder()
                                .setType("generic_node")
                                .putLabels("project_id", projectId)
                                .putLabels("location", "us-west1-b")  // Specify the region in which your service is running (e.g. us-west1-b). 
                                .putLabels("namespace", "anyNamespaceYouChoose")
                                .putLabels("node_id", InetAddress.getLocalHost().getHostName())  // Specify any node you choose (e.g. the local hostname).
                                .build())
                        .build()
        );
        
        // Then register your gRPC views in OpenCensus.
        RpcViews.registerClientGrpcViews();
    }
}
Viewing Your Metrics in Google Cloud Console

The above steps will expose Bigtable's gRPC metrics under the custom.googleapis.com/opencensus/grpc.io/client prefix.

Follow these instructions for viewing the metrics in Google Cloud Console.

Be sure to choose your Resource Type as the one you defined in your Stackdriver configuration in the code.

Tracing

The code example below shows how to enable tracing. For more details, see here.

Maven Setup

If you are not using the shaded Bigtable HBase Client artifact, you need to define the OpenCensus dependencies.

<!-- OpenCensus dependencies -->
<dependency>
    <groupId>com.google.cloud.bigtable</groupId>
    <artifactId>bigtable-hbase-1.x</artifactId>
    <version>1.18.0</version>
</dependency>
<dependency>
    <groupId>io.opencensus</groupId>
    <artifactId>opencensus-impl</artifactId>
    <version>0.24.0</version>
    <scope>runtime</scope>
</dependency>
<dependency>
    <groupId>io.opencensus</groupId>
    <artifactId>opencensus-exporter-trace-stackdriver</artifactId>
    <version>0.24.0</version>
    <exclusions>
        <exclusion>
            <groupId>io.grpc</groupId>
            <artifactId>*</artifactId>
        </exclusion>
        <exclusion>
            <groupId>com.google.auth</groupId>
            <artifactId>*</artifactId>
        </exclusion>
    </exclusions>
</dependency>

If you are using the shaded Bigtable HBase Client artifact, then the OpenCensus dependencies are embedded in the shaded artifact; i.e. nothing additional for you to do.

<!-- OpenCensus dependencies -->
<dependency>
    <groupId>com.google.cloud.bigtable</groupId>
    <artifactId>bigtable-hbase-1.x-shaded</artifactId>
    <version>1.18.0</version>
</dependency>
Java Example
// For the non-shaded client, remove the package prefix "com.google.bigtable.repackaged."
import com.google.bigtable.repackaged.io.opencensus.exporter.trace.stackdriver.StackdriverTraceConfiguration;
import com.google.bigtable.repackaged.io.opencensus.exporter.trace.stackdriver.StackdriverTraceExporter;
import com.google.bigtable.repackaged.io.opencensus.trace.Tracing;
import com.google.bigtable.repackaged.io.opencensus.trace.samplers.Samplers;

import java.io.IOException;

public class OpenCensusExample {
    String projectId = "your-project-id";

    void setupTracing() throws Exception {
        // Setup tracing.
        StackdriverTraceExporter.createAndRegister(
                StackdriverTraceConfiguration.builder()
                        .setProjectId(projectId)
                        .build()
        );
        Tracing.getTraceConfig().updateActiveTraceParams(
                Tracing.getTraceConfig().getActiveTraceParams().toBuilder()
                        // Adjust the sampling rate as you see fit.
                        .setSampler(Samplers.probabilitySampler(0.01))
                        .build()
        );
    }
}

Questions and discussions

If you have questions or run into issues with Google Cloud Bigtable or the client libraries, use any of the following forums:

You can also subscribe to google-cloud-bigtable-announce@ list to receive infrequent product and client library announcements.

Clients and Repositories

Name Language Repository latest version status
Cloud Bigtable Examples Java, others GoogleCloudPlatform/cloud-bigtable-examples
HBase client Java googleapis/cloud-bigtable-client Maven GA
Cloud Bigtable GoLang Go googleapis/google-cloud-go N/A GA
Cloud Bigtable Java Java googleapis/java-bigtable Maven GA
Cloud Bigtable Python Python googleapis/google-cloud-python PyPI version Beta
Cloud Bigtable Node.js Node googleaps/nodejs-bigtable npm version GA
Cloud Bigtable C# C# googleapis/google-cloud-dotnet NuGet version GA
Cloud Bigtable C++ C++ googleapis/google-cloud-cpp 0.9.0 Beta
Cloud Bigtable Ruby Ruby googleapis/google-cloud-ruby Gem Version Beta
Cloud Bigtable PHP PHP googleapis/google-cloud-php Latest Stable Version Beta

Contributing changes

See CONTRIBUTING.md for more information on how to contribute to this project.

License

Apache 2.0; see LICENSE for details.

About

Java libraries and HBase client extensions for accessing Google Cloud Bigtable

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Java 99.2%
  • Other 0.8%