Hugging Face Text Generation

hf_text_generation is an Hugging Face Text Generation API client for Java 11 or later. It is generated from the OpenAPI spec using the excellent OpenAPI Generator.

It can be used in Android or any Java and Kotlin Project.

Add Dependency

Gradle

To use library in your gradle project follow the steps below:

1. Add this in your root build.gradle at the end of repositories:

   allprojects {
       repositories {
           ...
           maven { url 'https://jitpack.io' }
       }
   }

2. Add the dependency

   dependencies {
       def HF_TEXT_GENERATION_VERSION = "..."
       implementation "llmjava:hf_text_generation:$HF_TEXT_GENERATION_VERSION"
   }

Maven

To use the library in your Maven project, follow the steps below:

1. Add the JitPack repository to your build file:

   <repositories>
       <repository>
           <id>jitpack.io</id>
           <url>https://jitpack.io</url>
       </repository>
   </repositories>

2. Add the dependency

   <dependency>
       <groupId>llmjava</groupId>
       <artifactId>hf_text_generation</artifactId>
       <version>${HF_TEXT_GENERATION_VERSION}</version>
   </dependency>

Usage

This section provide some examples for interacting with HuggingFace Text Generation API in java, see also huggingface-examples.

First, run a Text Generation Inference endpoint, e.g. locally it will be available at 127.0.0.1:8080.

See documentation to run an endpoint https://huggingface.co/docs/text-generation-inference

Set HF_API_URL with the endpoint url of your choice:

String HF_API_URL = "https://api-inference.huggingface.co";

Note: not all APIs are available if you are using HuggingFace public endpoint at https://huggingface.co/inference-api

Next, get an API token. If using HuggingFace public endpoint then you can get one at https://huggingface.co/settings/tokens

Now, we create a client to interact with the endpoint and be able to send requests

// create API client
String HF_TOKEN = System.getenv("HF_API_KEY");

ApiClient client = new ApiClient()
.setRequestInterceptor(new Consumer<HttpRequest.Builder>() {
    @Override public void accept(HttpRequest.Builder builder) {
        builder.header("Authorization", "Bearer " + HF_TOKEN);
    }
});
client.updateBaseUri(HF_API_URL);
TextGenerationInferenceApi api = new TextGenerationInferenceApi(client);

The rest of this section provide examples for submit different types of requests

Compact Text Generation request

Submit Compat Text Generation request

GenerateParameters params = new GenerateParameters()
    .seed(123L)
    .temperature(0.7f)
    .topK(3)
    .maxNewTokens(100);

// Compat Generation request
GenerateRequest request = new GenerateRequest()
    .inputs("Hi")
    .parameters(params);

GenerateResponse compatResponse = api.compatGenerate(compatRequest);
System.out.println("Generated text" + compatResponse.getGeneratedText());

Text Generation request

Submit Text Generation request

GenerateParameters params = new GenerateParameters()
    .seed(123L)
    .temperature(0.7f)
    .topK(3)
    .maxNewTokens(100);

// Generation request
GenerateRequest request = new GenerateRequest()
    .inputs("Hi")
    .parameters(params);

GenerateResponse response = api.generate(request);
System.out.println("Generated text" + response.getGeneratedText());

Streaming Text Generation request

Submit streaming Text Generation request

GenerateParameters params = new GenerateParameters()
    .seed(123L)
    .temperature(0.7f)
    .topK(3)
    .maxNewTokens(100);

// Generate stream
GenerateRequest streamRequest = new GenerateRequest()
    .inputs("Hi")
    .parameters(params);

StreamResponse streamResponse = api.generateStream(streamRequest);
System.out.println("Generated text" + streamResponse.getGeneratedText());

Model info request

Submit model info request

Info modelInfoResponse = api.getModelInfo();
System.out.println("Model info:\n" + modelInfoResponse);

Health request

Submit health request

api.health();

Metrics request

Submit metrics request

String metricsResponse = api.metrics();
System.out.println("Metrics response:\n" + metricsResponse);

Build Project

Clone the repository and import as Maven project in IntelliJ IDEA or Eclipse

Before building the project, make sure you have the following things installed.

• Maven
• Java 11

To install the library to your local Maven repository, simply execute:

mvn install

To build the library using Gradle, execute the following command

./gradlew build

Refer to the official documentation for more information.