-
Notifications
You must be signed in to change notification settings - Fork 1
Getting Started
Add jitpack and hosp-ai maven dependency
<dependency>
<groupId>com.github.r7b7</groupId>
<artifactId>hosp-ai</artifactId>
<version>v1.0.0-alpha.1</version>
</dependency>
<repository>
<id>jitpack.io</id>
<url>https://jitpack.io</url>
</repository>
When no example is provided in the prompt, it is called a zero-shot prompt. In this approach, the model is given only an instruction or question without any demonstrations of the desired output format. The model relies solely on its pre-trained knowledge to generate a response.
Invoke chat completion API from a provider of your liking - We use OpenAI here
public static void main( String[] args )
{
ILLMService service = LLMServiceFactory.createService(Provider.OPENAI, "<OPENAI-API-KEY>", "gpt-4");
PromptEngine promptEngine = new PromptEngine(service);
CompletionResponse response1 = promptEngine.sendQuery("what's the stock symbol of Palantir technology?");
System.out.println("Response from promptEngine: " + response1);
}
Response
Response from promptEngine: CompletionResponse[messages=[Message[role=assistant, content=The stock symbol for Palantir Technologies
is PLTR.]], metaData={total_tokens=30, id=chatcmpl-Ab9i1eXMZp6wffPGQVK56h83wfH8Y, prompt_tokens=18, model=gpt-4-0613, provider=OpenAi,
completion_tokens=12}, error=null]
Few-Shot Prompt involves providing the model with a few examples (typically 2 to 5) of a task to help it understand how to generate the desired output.
Invoke chat completion API by passing additional params (optional) and a system message (optional) along with a few examples.
public static void main( String[] args )
{
ILLMService service = LLMServiceFactory.createService(Provider.OPENAI, "<OPENAI-API-KEY>", "gpt-4");
PromptBuilder builder = new PromptBuilder()
.addMessage(new Message(Role.system, "Give output in consistent format"))
.addMessage(new Message(Role.user, "what's the stock symbol of ARCHER Aviation?"))
.addMessage(new Message(Role.assistant, "{\"company\":\"Archer\", \"symbol\":\"ACHR\"}"))
.addMessage(new Message(Role.user, "what's the stock symbol of Palantir technology?"))
.addParam("temperature", 0.7)
.addParam("max_tokens", 150);
PromptEngine engine = builder.build(service);
CompletionResponse response = engine.sendQuery();
System.out.println("Response from engine: " + response);
}
Response
Response from engine: CompletionResponse[messages=[Message[role=assistant, content={"company":"Palantir Technology",
"symbol":"PLTR"}]], metaData={total_tokens=71, id=chatcmpl-Ab9i0RltCR3C8PHzZyOmRphQdMcGz, prompt_tokens=57, model=gpt-4-0613,
provider=OpenAi, completion_tokens=14}, error=null]
- Follow any of the above examples
- Update synchronous call with an Asynchronous call
promptEngine.sendQueryAsync("<input_query>") .thenAccept(asyncResponse -> System.out.println("Async Response: " + asyncResponse)) .join(); //Prompt Builder as Input PromptEngine engine = builder.build(service); engine.sendQueryAsync() .thenAccept(asyncResponse -> System.out.println("Async Response: " + asyncResponse)) .join();
Update the values of model, apiKey (where applicable, for example, Ollama doesn't need an API key) and Provider.
For example, to move from Ollama to OpenAI, change this line
ILLMService service = LLMServiceFactory.createService(Provider.OLLAMA,"gemma2");
to
LLMService service = LLMServiceFactory.createService(Provider.OPENAI, "<OPEN-AI-KEY>", "gpt-4");
To add tools to a chat completion API, add the tool name, description and parameters to ToolFunction class.
ToolFunction function = new ToolFunction("get_current_weather",
"Get the current weather in a given location in fahrenheit",
parameters);
Next, pass the tool to PromptBuilder along with query and other parameters (if any).
PromptEngine engine = new PromptBuilder()
.addMessage(new Message(Role.user, "What's the weather in Chicago today?"))
.addTool(function)
.addToolChoice("auto")
.build(service);
If, the following parameters were passed in the tools field,
private static Map<String, Object> getParameters() {
Map<String, Object> parameters = new HashMap<>();
parameters.put("type", "object");
Map<String, Object> location = new HashMap<>();
location.put("type", "string");
location.put("description", "The city and state");
Map<String, Object> unit = new HashMap<>();
unit.put("type", "string");
unit.put("enum", List.of("celsius", "fahrenheit"));
parameters.put("properties", Map.of("location",location,"unit", unit));
parameters.put("required", List.of("location", "unit"));
return parameters;
}
We would get a similar response from the LLMs,
-
Groq
CompletionResponse[messages=[Message[role=assistant, content=null, toolCalls=[{id=call_8ed1, type=function, function= {name=get_current_weather, arguments={"location":"Chicago","unit":"fahrenheit"}}}]]], metaData={completion_tokens=51, provider=Groq, model=llama3-70b-8192, prompt_tokens=947, id=chatcmpl-f0895dda-d0de-4b34-a109-d4cbd07ef397, total_tokens=998}, error=null]
-
Ollama
CompletionResponse[messages=[Message[role=assistant, content=, toolCalls=[{function={name=get_current_weather, arguments=. {location=Chicago, IL, unit=fahrenheit}}}]]], metaData={provider=Ollama, model=mistral, eval_duration=2309000000, total_duration=6780714000}, error=null]
ILLMService service = LLMServiceFactory.createService(Provider.OPENAI,"<API_KEY>", "gpt-4o");
//sample content array
List<Map<String, Object>> content = new ArrayList<>();
Map<String, Object> item1 = new HashMap<>();
item1.put("type", "text");
item1.put("text", "What's in this image?");
Map<String, Object> item2 = new HashMap<>();
item2.put("type", "image_url");
item2.put("image_url", Map.of("url","https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-
boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"));
content.add(item1);
content.add(item2);
PromptEngine engine = new PromptBuilder().addMessage(new Message(Role.user, content)).build(service);
CompletionResponse response = engine.sendQuery();
System.out.println(response);
Response from OPENAI:
CompletionResponse[messages=[Message[role=assistant, content=The image shows a wooden boardwalk path through a grassy field. The sky above
is blue with a few clouds. There are trees and shrubs in the background. The scene conveys a peaceful, natural setting., toolCalls=null]],
metaData={prompt_tokens=1150, id=chatcmpl-AmlNEteMqqyeOU6tuE4AqKQjh42ZT, total_tokens=1192, completion_tokens=42, provider=OpenAi,
model=gpt-4o-2024-08-06}, error=null]
Please note - Not all models available on Groq support image inputs. Refer Groq documentation to find out the supported models. We use "llama-3.2-11b-vision-preview" in this example.
From the above sample code, update the params passed to LLMServiceFactory and run the code.
ILLMService service = LLMServiceFactory.createService(Provider.GROQ, "<API_KEY>", "llama-3.2-11b-vision-preview");
Response from Groq:
CompletionResponse[messages=[Message[role=assistant, content=Based on the image, what appears to be in the image is a wooden boardwalk
that runs through a field of tall grass., toolCalls=null]], metaData={prompt_tokens=17, model=llama-3.2-11b-vision-preview, provider=Groq,
completion_tokens=27, total_tokens=44, id=chatcmpl-2e99d2e0-e8ee-496c-8a34-b25ea84bf047}, error=null]
Unlike Groq and OpenAI, Claude model doesn't support image url as input(not yet that is). To pass an image as input, we would need to convert the image to base64 source type.
To add "content" of type array instead of a string, we can use code similar to the one shown below:
List<Map<String, Object>> content = new ArrayList<>();
Map<String, Object> item1 = new HashMap<>();
item1.put("type", "text");
item1.put("text", "What's in this image?");
Map<String, Object> item2 = new HashMap<>();
item2.put("type", "image");
item2.put("source", Map.of("type","base64","media_type","image/jpeg","data",imageToBase64Alt()));
content.add(item1);
content.add(item2);
where,"imageToBase64Alt" is the method that reads input file and converts to base64
public static String imageToBase64Alt() throws IOException {
byte[] imageBytes = Files.readAllBytes(Path.of("<path-to-image>"));
return Base64.getEncoder().encodeToString(imageBytes);
}
Pass the right params to LLMServiceFactory class. Rest of the code remains the same as the above OpenAI sample code.
ILLMService service = LLMServiceFactory.createService(Provider.ANTHROPIC,"<API_KEY>", "claude-3-5-sonnet-20241022");
NB: Image used here is that of two robot toys in front of a computer.
Response from Claude:
CompletionResponse[messages=[Message[role=assistant, content=This image shows two toy figurines on what appears to be a desk or table
surface. One is an R2-D2 style droid from Star Wars, and the other appears to be a playful robot or character figure with a more
whimsical design, featuring extended arms and a colorful appearance. There's a blurred computer or screen display visible in the
background, creating a nice depth of field in the photograph., toolCalls=null]], metaData={model=claude-3-5-sonnet-20241022,
id=msg_012pLAsa8dXJegkXZ1aCKMp5, provider=Anthropic, output_tokens=92, input_tokens=384}, error=null]
Ollama has a slightly simple structure to pass image in chat completion API request body, hence, we will need to tweak the input a little.
Since not all the models on Ollama support image as input, we need to pick one that supports this functionality. In the sample code given below, we use moondream
First, we would need to convert the images to base64 type and add them to an ArrayList as Ollama expects an array of base64 strings
List<String> images = new ArrayList<>();
images.add(imageToBase64Alt());
Next, we pass this array to Message class as shown below,
PromptEngine engine = new PromptBuilder().addMessage(new Message(Role.user, "What's in the image", images)).build(service);
Response from Ollama:
CompletionResponse[messages=[Message[role=assistant, content=
The image features a desk with two toy robots positioned on it. One of the robots is holding a cup, while the other robot has its arms
outstretched as if waving or greeting someone. The scene appears to be set up for a photo session, capturing the playful interaction
between these two toys., toolCalls=null]], metaData={total_duration=8717397083, eval_duration=1193000000, model=moondream,
provider=Ollama}, error=null]