diff --git a/docs/core_docs/docs/integrations/chat/ibm.ipynb b/docs/core_docs/docs/integrations/chat/ibm.ipynb new file mode 100644 index 000000000000..5c86adad492f --- /dev/null +++ b/docs/core_docs/docs/integrations/chat/ibm.ipynb @@ -0,0 +1,592 @@ +{ + "cells": [ + { + "cell_type": "raw", + "id": "afaf8039", + "metadata": { + "vscode": { + "languageId": "raw" + } + }, + "source": [ + "---\n", + "sidebar_label: IBM watsonx.ai\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "e49f1e0d", + "metadata": {}, + "source": [ + "# IBM watsonx.ai\n", + "\n", + "This will help you getting started with IBM watsonx.ai [chat models](/docs/concepts/#chat-models). For detailed documentation of all `IBM watsonx.ai` features and configurations head to the [IBM watsonx.ai](https://api.js.langchain.com/classes/_langchain_community.chat_models_ibm.html).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "| Class | Package | Local | Serializable | [PY support](https://python.langchain.com/docs/integrations/chat/ibm_watsonx/) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [`ChatWatsonx`](https://api.js.langchain.com/classes/_langchain_community.chat_models_ibm.html) | [@langchain/community](https://api.js.langchain.com/modules/langchain_community_llms_ibm.html) | ❌ | ✅ | ✅ | ![NPM - Downloads](https://img.shields.io/npm/dm/@langchain/community?style=flat-square&label=%20&) | ![NPM - Version](https://img.shields.io/npm/v/@langchain/community?style=flat-square&label=%20&) |\n", + "\n", + "### Model features\n", + "\n", + "\n", + "| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n", + "| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n", + "| ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | \n", + "\n", + "## Setup\n", + "\n", + "To access IBM watsonx.ai models you'll need to create a/an IBM watsonx.ai account, get an API key, and install the `@langchain/community` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "\n", + "Head to [IBM Cloud](https://cloud.ibm.com/login) to sign up to IBM watsonx.ai and generate an API key or provide any other authentication form as presented below.\n", + "\n", + "#### IAM authentication\n", + "\n", + "```bash\n", + "export WATSONX_AI_AUTH_TYPE=iam\n", + "export WATSONX_AI_APIKEY=\n", + "```\n", + "\n", + "#### Bearer token authentication\n", + "\n", + "```bash\n", + "export WATSONX_AI_AUTH_TYPE=bearertoken\n", + "export WATSONX_AI_BEARER_TOKEN=\n", + "```\n", + "\n", + "#### CP4D authentication\n", + "\n", + "```bash\n", + "export WATSONX_AI_AUTH_TYPE=cp4d\n", + "export WATSONX_AI_USERNAME=\n", + "export WATSONX_AI_PASSWORD=\n", + "export WATSONX_AI_URL=\n", + "```\n", + "\n", + "Once these are places in your enviromental variables and object is initialized authentication will proceed automatically.\n", + "\n", + "Authentication can also be accomplished by passing these values as parameters to a new instance.\n", + "\n", + "## IAM authentication\n", + "\n", + "```typescript\n", + "import { WatsonxLLM } from \"@langchain/community/llms/ibm\";\n", + "\n", + "const props = {\n", + " version: \"YYYY-MM-DD\",\n", + " serviceUrl: \"\",\n", + " projectId: \"\",\n", + " watsonxAIAuthType: \"iam\",\n", + " watsonxAIApikey: \"\",\n", + "};\n", + "const instance = new WatsonxLLM(props);\n", + "```\n", + "\n", + "## Bearer token authentication\n", + "\n", + "```typescript\n", + "import { WatsonxLLM } from \"@langchain/community/llms/ibm\";\n", + "\n", + "const props = {\n", + " version: \"YYYY-MM-DD\",\n", + " serviceUrl: \"\",\n", + " projectId: \"\",\n", + " watsonxAIAuthType: \"bearertoken\",\n", + " watsonxAIBearerToken: \"\",\n", + "};\n", + "const instance = new WatsonxLLM(props);\n", + "```\n", + "\n", + "### CP4D authentication\n", + "\n", + "```typescript\n", + "import { WatsonxLLM } from \"@langchain/community/llms/ibm\";\n", + "\n", + "const props = {\n", + " version: \"YYYY-MM-DD\",\n", + " serviceUrl: \"\",\n", + " projectId: \"\",\n", + " watsonxAIAuthType: \"cp4d\",\n", + " watsonxAIUsername: \"\",\n", + " watsonxAIPassword: \"\",\n", + " watsonxAIUrl: \"\",\n", + "};\n", + "const instance = new WatsonxLLM(props);\n", + "```\n", + "\n", + "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:\n", + "\n", + "```bash\n", + "# export LANGCHAIN_TRACING_V2=\"true\"\n", + "# export LANGCHAIN_API_KEY=\"your-api-key\"\n", + "```\n", + "\n", + "### Installation\n", + "\n", + "The LangChain IBM watsonx.ai integration lives in the `@langchain/community` package:\n", + "\n", + "```{=mdx}\n", + "import IntegrationInstallTooltip from \"@mdx_components/integration_install_tooltip.mdx\";\n", + "import Npm2Yarn from \"@theme/Npm2Yarn\";\n", + "\n", + "\n", + "\n", + "\n", + " __package_name__ @langchain/core\n", + "\n", + "\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "a38cde65-254d-4219-a441-068766c0d4b5", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae", + "metadata": {}, + "outputs": [], + "source": [ + "import { ChatWatsonx } from \"@langchain/community/chat_models/ibm\";\n", + "const props = {\n", + " maxTokens: 200,\n", + " temperature: 0.5\n", + "};\n", + "\n", + "const instance = new ChatWatsonx({\n", + " version: \"YYYY-MM-DD\",\n", + " serviceUrl: process.env.API_URL,\n", + " projectId: \"\",\n", + " spaceId: \"\",\n", + " model: \"\",\n", + " ...props\n", + "});" + ] + }, + { + "cell_type": "markdown", + "id": "30cb3968", + "metadata": {}, + "source": [ + "Note:\n", + "\n", + "- You must provide `spaceId` or `projectId` in order to proceed.\n", + "- Depending on the region of your provisioned service instance, use correct serviceUrl." + ] + }, + { + "cell_type": "markdown", + "id": "2b4f3e15", + "metadata": {}, + "source": [ + "## Invocation" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "62e0dbc3", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "AIMessage {\n", + " \"id\": \"chat-c5341b2062dc42f091e5ae2558e905e3\",\n", + " \"content\": \" J'adore la programmation.\",\n", + " \"additional_kwargs\": {\n", + " \"tool_calls\": []\n", + " },\n", + " \"response_metadata\": {\n", + " \"tokenUsage\": {\n", + " \"completion_tokens\": 10,\n", + " \"prompt_tokens\": 28,\n", + " \"total_tokens\": 38\n", + " },\n", + " \"finish_reason\": \"stop\"\n", + " },\n", + " \"tool_calls\": [],\n", + " \"invalid_tool_calls\": [],\n", + " \"usage_metadata\": {\n", + " \"input_tokens\": 28,\n", + " \"output_tokens\": 10,\n", + " \"total_tokens\": 38\n", + " }\n", + "}\n" + ] + } + ], + "source": [ + "const aiMsg = await instance.invoke([{\n", + " role: \"system\",\n", + " content: \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n", + "},\n", + "{\n", + " role: \"user\",\n", + " content: \"I love programming.\"\n", + "}]);\n", + "console.log(aiMsg)" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "d86145b3-bfef-46e8-b227-4dda5c9c2705", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " J'adore la programmation.\n" + ] + } + ], + "source": [ + "console.log(aiMsg.content)" + ] + }, + { + "cell_type": "markdown", + "id": "18e2bfc0-7e78-4528-a73f-499ac150dca8", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "AIMessage {\n", + " \"id\": \"chat-c5c2c08d3c984254acc48225c39c6a08\",\n", + " \"content\": \" Ich liebe Programmieren.\",\n", + " \"additional_kwargs\": {\n", + " \"tool_calls\": []\n", + " },\n", + " \"response_metadata\": {\n", + " \"tokenUsage\": {\n", + " \"completion_tokens\": 8,\n", + " \"prompt_tokens\": 22,\n", + " \"total_tokens\": 30\n", + " },\n", + " \"finish_reason\": \"stop\"\n", + " },\n", + " \"tool_calls\": [],\n", + " \"invalid_tool_calls\": [],\n", + " \"usage_metadata\": {\n", + " \"input_tokens\": 22,\n", + " \"output_tokens\": 8,\n", + " \"total_tokens\": 30\n", + " }\n", + "}\n" + ] + } + ], + "source": [ + "import { ChatPromptTemplate } from \"@langchain/core/prompts\";\n", + "\n", + "const prompt = ChatPromptTemplate.fromMessages(\n", + " [\n", + " [\n", + " \"system\",\n", + " \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n", + " ],\n", + " [\"human\", \"{input}\"],\n", + " ]\n", + ")\n", + "const chain = prompt.pipe(instance);\n", + "await chain.invoke(\n", + " {\n", + " input_language: \"English\",\n", + " output_language: \"German\",\n", + " input: \"I love programming.\",\n", + " }\n", + " )" + ] + }, + { + "cell_type": "markdown", + "id": "2896aae5", + "metadata": {}, + "source": [ + "## Streaming the Model output" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "cd21e356", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " The\n", + " Moon\n", + " is\n", + " Earth\n", + "'\n", + "s\n", + " only\n", + " natural\n", + " satellite\n", + " and\n", + " the\n", + " fifth\n", + " largest\n", + " satellite\n", + " in\n", + " the\n", + " Solar\n", + " System\n", + ".\n", + " It\n", + " or\n", + "bits\n", + " Earth\n", + " every\n", + " \n", + "2\n", + "7\n", + ".\n", + "3\n", + " days\n", + " and\n", + " rot\n", + "ates\n", + " on\n", + " its\n", + " axis\n", + " in\n", + " the\n", + " same\n", + " amount\n", + " of\n", + " time\n", + ",\n", + " which\n", + " is\n", + " why\n", + " we\n", + " always\n", + " see\n", + " the\n", + " same\n", + " side\n", + " of\n", + " it\n", + ".\n", + " The\n", + " Moon\n", + "'\n", + "s\n", + " phases\n", + " change\n", + " as\n", + " it\n", + " or\n", + "bits\n", + " Earth\n", + ",\n", + " going\n", + " through\n", + " cycles\n", + " of\n", + " new\n", + ",\n", + " c\n", + "res\n", + "cent\n", + ",\n", + " half\n", + ",\n", + " g\n", + "ib\n", + "b\n", + "ous\n", + ",\n", + " and\n", + " full\n", + " phases\n", + ".\n", + " Its\n", + " gravity\n", + " influences\n", + " Earth\n", + "'\n", + "s\n", + " t\n", + "ides\n", + " and\n", + " stabil\n", + "izes\n", + " our\n" + ] + } + ], + "source": [ + "import { HumanMessage, SystemMessage } from \"@langchain/core/messages\";\n", + "\n", + "const messages = [\n", + " new SystemMessage('You are a helpful assistant which telling short-info about provided topic.'),\n", + " new HumanMessage(\"moon\")\n", + "]\n", + "const stream = await instance.stream(messages);\n", + "for await(const chunk of stream){\n", + " console.log(chunk)\n", + "}" + ] + }, + { + "cell_type": "markdown", + "id": "65ed0609", + "metadata": {}, + "source": [ + "## Tool calling" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "f32f8cb0", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "AIMessage {\n", + " \"id\": \"chat-d2214d0bdb794483a213b3211cf0d819\",\n", + " \"content\": \"\",\n", + " \"additional_kwargs\": {\n", + " \"tool_calls\": [\n", + " {\n", + " \"id\": \"chatcmpl-tool-257f3d39532141b89178c2120f81f0cb\",\n", + " \"type\": \"function\",\n", + " \"function\": \"[Object]\"\n", + " }\n", + " ]\n", + " },\n", + " \"response_metadata\": {\n", + " \"tokenUsage\": {\n", + " \"completion_tokens\": 38,\n", + " \"prompt_tokens\": 177,\n", + " \"total_tokens\": 215\n", + " },\n", + " \"finish_reason\": \"tool_calls\"\n", + " },\n", + " \"tool_calls\": [\n", + " {\n", + " \"name\": \"calculator\",\n", + " \"args\": {\n", + " \"number1\": 3,\n", + " \"number2\": 12,\n", + " \"operation\": \"multiply\"\n", + " },\n", + " \"type\": \"tool_call\",\n", + " \"id\": \"chatcmpl-tool-257f3d39532141b89178c2120f81f0cb\"\n", + " }\n", + " ],\n", + " \"invalid_tool_calls\": [],\n", + " \"usage_metadata\": {\n", + " \"input_tokens\": 177,\n", + " \"output_tokens\": 38,\n", + " \"total_tokens\": 215\n", + " }\n", + "}\n" + ] + } + ], + "source": [ + "import { tool } from \"@langchain/core/tools\";\n", + "import { z } from \"zod\";\n", + "\n", + "const calculatorSchema = z.object({\n", + " operation: z\n", + " .enum([\"add\", \"subtract\", \"multiply\", \"divide\"])\n", + " .describe(\"The type of operation to execute.\"),\n", + " number1: z.number().describe(\"The first number to operate on.\"),\n", + " number2: z.number().describe(\"The second number to operate on.\"),\n", + " });\n", + " \n", + "const calculatorTool = tool(\n", + "async ({ operation, number1, number2 }) => {\n", + " if (operation === \"add\") {\n", + " return `${number1 + number2}`;\n", + " } else if (operation === \"subtract\") {\n", + " return `${number1 - number2}`;\n", + " } else if (operation === \"multiply\") {\n", + " return `${number1 * number2}`;\n", + " } else if (operation === \"divide\") {\n", + " return `${number1 / number2}`;\n", + " } else {\n", + " throw new Error(\"Invalid operation.\");\n", + " }\n", + "},\n", + "{\n", + " name: \"calculator\",\n", + " description: \"Can perform mathematical operations.\",\n", + " schema: calculatorSchema,\n", + "}\n", + ");\n", + "\n", + "const instanceWithTools = instance.bindTools([calculatorTool]);\n", + "\n", + "const res = await instanceWithTools.invoke(\"What is 3 * 12\");\n", + "console.log(res)" + ] + }, + { + "cell_type": "markdown", + "id": "6339db97", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all `IBM watsonx.ai` features and configurations head to the API reference: [API docs](https://api.js.langchain.com/modules/_langchain_community.embeddings_ibm.html)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "JavaScript (Node.js)", + "language": "javascript", + "name": "javascript" + }, + "language_info": { + "file_extension": ".js", + "mimetype": "application/javascript", + "name": "javascript", + "version": "20.17.0" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/core_docs/docs/integrations/llms/ibm.ipynb b/docs/core_docs/docs/integrations/llms/ibm.ipynb new file mode 100644 index 000000000000..81e57aceb7d6 --- /dev/null +++ b/docs/core_docs/docs/integrations/llms/ibm.ipynb @@ -0,0 +1,361 @@ +{ + "cells": [ + { + "cell_type": "raw", + "id": "67db2992", + "metadata": { + "vscode": { + "languageId": "raw" + } + }, + "source": [ + "---\n", + "sidebar_label: IBM watsonx.ai\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "9597802c", + "metadata": {}, + "source": [ + "# IBM watsonx.ai\n", + "\n", + "\n", + "This will help you get started with IBM [text completion models (LLMs)](/docs/concepts#llms) using LangChain. For detailed documentation on `IBM watsonx.ai` features and configuration options, please refer to the [IBM watsonx.ai](https://api.js.langchain.com/classes/_langchain_community.llms_ibm.html).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "\n", + "| Class | Package | Local | Serializable | [PY support](https://python.langchain.com/docs/integrations/llms/ibm_watsonx/) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n", + "| [`IBM watsonx.ai`](https://api.js.langchain.com/modules/_langchain_community.llms_ibm.html) | [@langchain/community](https://api.js.langchain.com/modules/langchain_community_llms_ibm.html) | ❌ | ✅ | ✅ | ![NPM - Downloads](https://img.shields.io/npm/dm/@langchain/community?style=flat-square&label=%20&) | ![NPM - Version](https://img.shields.io/npm/v/@langchain/community?style=flat-square&label=%20&) |\n", + "\n", + "## Setup\n", + "\n", + "\n", + "To access IBM WatsonxAI models you'll need to create an IBM watsonx.ai account, get an API key or any other type of credentials, and install the `@langchain/community` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "\n", + "Head to [IBM Cloud](https://cloud.ibm.com/login) to sign up to IBM watsonx.ai and generate an API key or provide any other authentication form as presented below.\n", + "\n", + "#### IAM authentication\n", + "\n", + "```bash\n", + "export WATSONX_AI_AUTH_TYPE=iam\n", + "export WATSONX_AI_APIKEY=\n", + "```\n", + "\n", + "#### Bearer token authentication\n", + "\n", + "```bash\n", + "export WATSONX_AI_AUTH_TYPE=bearertoken\n", + "export WATSONX_AI_BEARER_TOKEN=\n", + "```\n", + "\n", + "#### CP4D authentication\n", + "\n", + "```bash\n", + "export WATSONX_AI_AUTH_TYPE=cp4d\n", + "export WATSONX_AI_USERNAME=\n", + "export WATSONX_AI_PASSWORD=\n", + "export WATSONX_AI_URL=\n", + "```\n", + "\n", + "Once these are placed in your environment variables and object is initialized authentication will proceed automatically.\n", + "\n", + "Authentication can also be accomplished by passing these values as parameters to a new instance.\n", + "\n", + "## IAM authentication\n", + "\n", + "```typescript\n", + "import { WatsonxLLM } from \"@langchain/community/llms/ibm\";\n", + "\n", + "const props = {\n", + " version: \"YYYY-MM-DD\",\n", + " serviceUrl: \"\",\n", + " projectId: \"\",\n", + " watsonxAIAuthType: \"iam\",\n", + " watsonxAIApikey: \"\",\n", + "};\n", + "const instance = new WatsonxLLM(props);\n", + "```\n", + "\n", + "## Bearer token authentication\n", + "\n", + "```typescript\n", + "import { WatsonxLLM } from \"@langchain/community/llms/ibm\";\n", + "\n", + "const props = {\n", + " version: \"YYYY-MM-DD\",\n", + " serviceUrl: \"\",\n", + " projectId: \"\",\n", + " watsonxAIAuthType: \"bearertoken\",\n", + " watsonxAIBearerToken: \"\",\n", + "};\n", + "const instance = new WatsonxLLM(props);\n", + "```\n", + "\n", + "### CP4D authentication\n", + "\n", + "```typescript\n", + "import { WatsonxLLM } from \"@langchain/community/llms/ibm\";\n", + "\n", + "const props = {\n", + " version: \"YYYY-MM-DD\",\n", + " serviceUrl: \"\",\n", + " projectId: \"\",\n", + " watsonxAIAuthType: \"cp4d\",\n", + " watsonxAIUsername: \"\",\n", + " watsonxAIPassword: \"\",\n", + " watsonxAIUrl: \"\",\n", + "};\n", + "const instance = new WatsonxLLM(props);\n", + "```\n", + "\n", + "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:\n", + "\n", + "```bash\n", + "# export LANGCHAIN_TRACING_V2=\"true\"\n", + "# export LANGCHAIN_API_KEY=\"your-api-key\"\n", + "```\n", + "\n", + "### Installation\n", + "\n", + "The LangChain IBM watsonx.ai integration lives in the `@langchain/community` package:\n", + "\n", + "```{=mdx}\n", + "import IntegrationInstallTooltip from \"@mdx_components/integration_install_tooltip.mdx\";\n", + "import Npm2Yarn from \"@theme/Npm2Yarn\";\n", + "\n", + "\n", + "\n", + "\n", + " @langchain/community @langchain/core\n", + "\n", + "\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "0a760037", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and generate chat completions:\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a0562a13", + "metadata": {}, + "outputs": [], + "source": [ + "import { WatsonxLLM } from \"@langchain/community/llms/ibm\";\n", + "\n", + "const props = {\n", + " decoding_method: \"sample\",\n", + " max_new_tokens: 100,\n", + " min_new_tokens: 1,\n", + " temperature: 0.5,\n", + " top_k: 50,\n", + " top_p: 1,\n", + "};\n", + "const instance = new WatsonxLLM({\n", + " version: \"YYYY-MM-DD\",\n", + " serviceUrl: process.env.API_URL,\n", + " projectId: \"\",\n", + " spaceId: \"\",\n", + " idOrName: \"\",\n", + " model: \"\",\n", + " ...props,\n", + "});" + ] + }, + { + "cell_type": "markdown", + "id": "f7498103", + "metadata": {}, + "source": [ + "Note:\n", + "\n", + "- You must provide `spaceId`, `projectId` or `idOrName`(deployment id) in order to proceed.\n", + "- Depending on the region of your provisioned service instance, use correct serviceUrl.\n", + "- You need to specify the model you want to use for inferencing through model_id." + ] + }, + { + "cell_type": "markdown", + "id": "0ee90032", + "metadata": {}, + "source": [ + "## Invocation and generation\n" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "035dea0f", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "print('Hello world.')<|endoftext|>\n", + "{\n", + " generations: [ [ [Object] ], [ [Object] ] ],\n", + " llmOutput: { tokenUsage: { generated_token_count: 28, input_token_count: 10 } }\n", + "}\n" + ] + } + ], + "source": [ + "const result = await instance.invoke(\"Print hello world.\");\n", + "console.log(result);\n", + "\n", + "const results = await instance.generate([\n", + " \"Print hello world.\",\n", + " \"Print bye, bye world!\",\n", + "]);\n", + "console.log(results);" + ] + }, + { + "cell_type": "markdown", + "id": "add38532", + "metadata": {}, + "source": [ + "## Chaining\n", + "\n", + "We can chain our completion model with a prompt template like so:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "078e9db2", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Ich liebe Programmieren.\n", + "\n", + "To express that you are passionate about programming in German,\n" + ] + } + ], + "source": [ + "import { PromptTemplate } from \"@langchain/core/prompts\"\n", + "\n", + "const prompt = PromptTemplate.fromTemplate(\"How to say {input} in {output_language}:\\n\")\n", + "\n", + "const chain = prompt.pipe(instance);\n", + "await chain.invoke(\n", + " {\n", + " output_language: \"German\",\n", + " input: \"I love programming.\",\n", + " }\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "0c305670", + "metadata": {}, + "source": [ + "## Props overwriting\n", + "\n", + "Passed props at initialization will last for the whole life cycle of the object, however you may overwrite them for a single method's call by passing second argument as below\n" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "bb53235c", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "print('Hello world.')<|endoftext|>\n" + ] + } + ], + "source": [ + "const result2 = await instance.invoke(\"Print hello world.\", {\n", + " parameters: {\n", + " max_new_tokens: 20,\n", + " },\n", + "});\n", + "console.log(result2);" + ] + }, + { + "cell_type": "markdown", + "id": "577a0583", + "metadata": {}, + "source": [ + "## Tokenization\n", + "This package has it's custom getNumTokens implementation which returns exact amount of tokens that would be used.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "339e237c", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "4\n" + ] + } + ], + "source": [ + "const tokens = await instance.getNumTokens(\"Print hello world.\");\n", + "console.log(tokens);" + ] + }, + { + "cell_type": "markdown", + "id": "e9bdfcef", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all `IBM watsonx.ai` features and configurations head to the API reference: [API docs](https://api.js.langchain.com/modules/_langchain_community.embeddings_ibm.html)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "JavaScript (Node.js)", + "language": "javascript", + "name": "javascript" + }, + "language_info": { + "file_extension": ".js", + "mimetype": "application/javascript", + "name": "javascript", + "version": "20.17.0" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/core_docs/docs/integrations/text_embedding/ibm.ipynb b/docs/core_docs/docs/integrations/text_embedding/ibm.ipynb new file mode 100644 index 000000000000..ecfb9ac677b4 --- /dev/null +++ b/docs/core_docs/docs/integrations/text_embedding/ibm.ipynb @@ -0,0 +1,389 @@ +{ + "cells": [ + { + "cell_type": "raw", + "id": "afaf8039", + "metadata": { + "vscode": { + "languageId": "raw" + } + }, + "source": [ + "---\n", + "sidebar_label: IBM watsonx.ai\n", + "---" + ] + }, + { + "cell_type": "markdown", + "id": "9a3d6f34", + "metadata": {}, + "source": [ + "# IBM watsonx.ai\n", + "\n", + "\n", + "This will help you get started with IBM watsonx.ai [embedding models](/docs/concepts#embedding-models) using LangChain. For detailed documentation on `IBM watsonx.ai` features and configuration options, please refer to the [API reference](https://api.js.langchain.com/classes/_langchain_community.embeddings_ibm.html).\n", + "\n", + "## Overview\n", + "### Integration details\n", + "\n", + "\n", + "| Class | Package | Local | [Py support](https://python.langchain.com/docs/integrations/text_embedding/ibm_watsonx/) | Package downloads | Package latest |\n", + "| :--- | :--- | :---: | :---: | :---: | :---: |\n", + "| [`IBM watsonx.ai`](https://api.js.langchain.com/classes/_langchain_community.embeddings_ibm.WatsonxEmbeddings.html) | [@langchain/community](https://api.js.langchain.com/modules/langchain_community_llms_ibm.html)| ❌ | ✅ | ![NPM - Downloads](https://img.shields.io/npm/dm/@langchain/community?style=flat-square&label=%20&) | ![NPM - Version](https://img.shields.io/npm/v/@langchain/community?style=flat-square&label=%20&) |\n", + "\n", + "## Setup\n", + "\n", + "To access IBM WatsonxAI embeddings you'll need to create an IBM watsonx.ai account, get an API key or any other type of credentials, and install the `@langchain/community` integration package.\n", + "\n", + "### Credentials\n", + "\n", + "\n", + "Head to [IBM Cloud](https://cloud.ibm.com/login) to sign up to IBM watsonx.ai and generate an API key or provide any other authentication form as presented below.\n", + "\n", + "#### IAM authentication\n", + "\n", + "```bash\n", + "export WATSONX_AI_AUTH_TYPE=iam\n", + "export WATSONX_AI_APIKEY=\n", + "```\n", + "\n", + "#### Bearer token authentication\n", + "\n", + "```bash\n", + "export WATSONX_AI_AUTH_TYPE=bearertoken\n", + "export WATSONX_AI_BEARER_TOKEN=\n", + "```\n", + "\n", + "#### CP4D authentication\n", + "\n", + "```bash\n", + "export WATSONX_AI_AUTH_TYPE=cp4d\n", + "export WATSONX_AI_USERNAME=\n", + "export WATSONX_AI_PASSWORD=\n", + "export WATSONX_AI_URL=\n", + "```\n", + "\n", + "Once these are placed in your environment variables and object is initialized authentication will proceed automatically.\n", + "\n", + "Authentication can also be accomplished by passing these values as parameters to a new instance.\n", + "\n", + "## IAM authentication\n", + "\n", + "```typescript\n", + "import { WatsonxEmbeddings } from \"@langchain/community/embeddings/ibm\";\n", + "\n", + "const props = {\n", + " version: \"YYYY-MM-DD\",\n", + " serviceUrl: \"\",\n", + " projectId: \"\",\n", + " watsonxAIAuthType: \"iam\",\n", + " watsonxAIApikey: \"\",\n", + "};\n", + "const instance = new WatsonxEmbeddings(props);\n", + "```\n", + "\n", + "## Bearer token authentication\n", + "\n", + "```typescript\n", + "import { WatsonxEmbeddings } from \"@langchain/community/embeddings/ibm\";\n", + "\n", + "const props = {\n", + " version: \"YYYY-MM-DD\",\n", + " serviceUrl: \"\",\n", + " projectId: \"\",\n", + " watsonxAIAuthType: \"bearertoken\",\n", + " watsonxAIBearerToken: \"\",\n", + "};\n", + "const instance = new WatsonxEmbeddings(props);\n", + "```\n", + "\n", + "### CP4D authentication\n", + "\n", + "```typescript\n", + "import { WatsonxEmbeddings } from \"@langchain/community/embeddings/ibm\";\n", + "\n", + "const props = {\n", + " version: \"YYYY-MM-DD\",\n", + " serviceUrl: \"\",\n", + " projectId: \"\",\n", + " watsonxAIAuthType: \"cp4d\",\n", + " watsonxAIUsername: \"\",\n", + " watsonxAIPassword: \"\",\n", + " watsonxAIUrl: \"\",\n", + "};\n", + "const instance = new WatsonxEmbeddings(props);\n", + "```\n", + "\n", + "If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:\n", + "\n", + "```bash\n", + "# export LANGCHAIN_TRACING_V2=\"true\"\n", + "# export LANGCHAIN_API_KEY=\"your-api-key\"\n", + "```\n", + "\n", + "### Installation\n", + "\n", + "The LangChain IBM watsonx.ai integration lives in the `@langchain/community` package:\n", + "\n", + "```{=mdx}\n", + "import IntegrationInstallTooltip from \"@mdx_components/integration_install_tooltip.mdx\";\n", + "import Npm2Yarn from \"@theme/Npm2Yarn\";\n", + "\n", + "\n", + "\n", + "\n", + " @langchain/community @langchain/core\n", + "\n", + "\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "45dd1724", + "metadata": {}, + "source": [ + "## Instantiation\n", + "\n", + "Now we can instantiate our model object and embed text:\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9ea7a09b", + "metadata": {}, + "outputs": [], + "source": [ + "import { WatsonxEmbeddings } from \"@langchain/community/embeddings/ibm\";\n", + "\n", + "const embeddings = new WatsonxEmbeddings({\n", + " version: \"YYYY-MM-DD\",\n", + " serviceUrl: process.env.API_URL,\n", + " projectId: \"\",\n", + " spaceId: \"\",\n", + " idOrName: \"\",\n", + " model: \"\",\n", + "});" + ] + }, + { + "cell_type": "markdown", + "id": "ba7f5c8a", + "metadata": {}, + "source": [ + "Note:\n", + "\n", + "- You must provide `spaceId`, `projectId` or `idOrName`(deployment id) in order to proceed.\n", + "- Depending on the region of your provisioned service instance, use correct serviceUrl." + ] + }, + { + "cell_type": "markdown", + "id": "77d271b6", + "metadata": {}, + "source": [ + "## Indexing and Retrieval\n", + "\n", + "Embedding models are often used in retrieval-augmented generation (RAG) flows, both as part of indexing data as well as later retrieving it. For more detailed instructions, please see our RAG tutorials under the [working with external knowledge tutorials](/docs/tutorials/#working-with-external-knowledge).\n", + "\n", + "Below, see how to index and retrieve data using the `embeddings` object we initialized above. In this example, we will index and retrieve a sample document using the demo [`MemoryVectorStore`](/docs/integrations/vectorstores/memory)." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "d817716b", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "LangChain is the framework for building context-aware reasoning applications\n" + ] + } + ], + "source": [ + "// Create a vector store with a sample text\n", + "import { MemoryVectorStore } from \"langchain/vectorstores/memory\";\n", + "\n", + "const text = \"LangChain is the framework for building context-aware reasoning applications\";\n", + "\n", + "const vectorstore = await MemoryVectorStore.fromDocuments(\n", + " [{ pageContent: text, metadata: {} }],\n", + " embeddings,\n", + ");\n", + "\n", + "// Use the vector store as a retriever that returns a single document\n", + "const retriever = vectorstore.asRetriever(1);\n", + "\n", + "// Retrieve the most similar text\n", + "const retrievedDocuments = await retriever.invoke(\"What is LangChain?\");\n", + "\n", + "retrievedDocuments[0].pageContent;" + ] + }, + { + "cell_type": "markdown", + "id": "e02b9855", + "metadata": {}, + "source": [ + "## Direct Usage\n", + "\n", + "Under the hood, the vectorstore and retriever implementations are calling `embeddings.embedDocument(...)` and `embeddings.embedQuery(...)` to create embeddings for the text(s) used in `fromDocuments` and the retriever's `invoke` operations, respectively.\n", + "\n", + "You can directly call these methods to get embeddings for your own use cases.\n", + "\n", + "### Embed single texts\n", + "\n", + "You can embed queries for search with `embedQuery`. This generates a vector representation specific to the query:" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "0d2befcd", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[\n", + " -0.017436018, -0.01469498, -0.015685871, -0.013543149, -0.0011519607,\n", + " -0.008123747, 0.015286108, -0.023845721, -0.02454774, 0.07235078,\n", + " -0.032333843, -0.0035843418, -0.015389036, 0.0455373, -0.021119863,\n", + " -0.022039745, 0.021746712, -0.017774817, -0.008232582, -0.036727764,\n", + " -0.015734928, 0.03606811, -0.005108186, -0.036052454, 0.024462992,\n", + " 0.02359307, 0.03273164, 0.009195497, -0.0077208397, -0.0127943,\n", + " -0.023869334, -0.029473905, -0.0080457395, -0.0021337876, 0.04949132,\n", + " 0.013950589, -0.010046689, 0.021029025, -0.031725302, 0.004251065,\n", + " -0.034171984, -0.03696642, -0.014253629, -0.017757406, -0.007531065,\n", + " 0.07187789, 0.009661725, 0.041889492, -0.04660478, 0.028036641,\n", + " 0.059334517, -0.04561291, 0.056029715, -0.00676024, 0.026493236,\n", + " 0.0116374, 0.050126843, -0.018036349, -0.013711887, 0.042252757,\n", + " -0.04453391, 0.04705777, -0.00044598224, -0.030227259, 0.029286578,\n", + " 0.0252211, 0.011694125, -0.031404093, 0.02951232, 0.08812359,\n", + " 0.023539362, -0.011082862, 0.008024676, 0.00084492035, -0.007984158,\n", + " -0.0005008702, -0.025189219, 0.021000557, -0.0065513053, 0.036524914,\n", + " 0.0015150858, -0.0042383806, 0.049065087, 0.000941666, 0.04447001,\n", + " 0.012942205, -0.078316726, -0.03004237, -0.025807172, -0.03446275,\n", + " -0.00932942, -0.044925686, 0.03190307, 0.010136769, -0.048854534,\n", + " 0.025738232, -0.017840309, 0.023738133, 0.014214792, 0.030452395\n", + "]\n" + ] + } + ], + "source": [ + " const singleVector = await embeddings.embedQuery(text);\n", + " singleVector.slice(0, 100);" + ] + }, + { + "cell_type": "markdown", + "id": "1b5a7d03", + "metadata": {}, + "source": [ + "### Embed multiple texts\n", + "\n", + "You can embed multiple texts for indexing with `embedDocuments`. The internals used for this method may (but do not have to) differ from embedding queries:" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "2f4d6e97", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[\n", + " -0.017436024, -0.014695002, -0.01568589, -0.013543164, -0.001151976,\n", + " -0.008123703, 0.015286064, -0.023845702, -0.024547677, 0.07235076,\n", + " -0.032333862, -0.0035843418, -0.015389038, 0.045537304, -0.021119865,\n", + " -0.02203975, 0.021746716, -0.01777481, -0.008232588, -0.03672781,\n", + " -0.015734889, 0.036068108, -0.0051082, -0.036052432, 0.024462998,\n", + " 0.023593083, 0.03273162, 0.009195521, -0.007720828, -0.012794304,\n", + " -0.023869323, -0.029473891, -0.008045726, -0.002133793, 0.049491342,\n", + " 0.013950573, -0.010046691, 0.02102898, -0.03172528, 0.0042510596,\n", + " -0.034171965, -0.036966413, -0.014253668, -0.017757434, -0.007531062,\n", + " 0.07187787, 0.009661732, 0.041889492, -0.04660476, 0.028036654,\n", + " 0.059334517, -0.045612894, 0.056029722, -0.00676024, 0.026493296,\n", + " 0.0116374055, 0.050126873, -0.018036384, -0.013711868, 0.0422528,\n", + " -0.044533912, 0.047057763, -0.00044596897, -0.030227251, 0.029286569,\n", + " 0.025221113, 0.011694138, -0.03140413, 0.029512335, 0.08812357,\n", + " 0.023539348, -0.011082865, 0.008024677, 0.00084490055, -0.007984145,\n", + " -0.0005008745, -0.025189226, 0.021000564, -0.0065513197, 0.036524955,\n", + " 0.0015150585, -0.0042383634, 0.049065102, 0.000941638, 0.044469994,\n", + " 0.012942193, -0.078316696, -0.0300424, -0.025807157, -0.0344627,\n", + " -0.009329439, -0.04492573, 0.031903077, 0.010136808, -0.048854522,\n", + " 0.025738247, -0.01784033, 0.023738142, 0.014214801, 0.030452369\n", + "]\n", + "[\n", + " 0.03278884, -0.017893745, -0.0027520044, 0.016506646, 0.028271576,\n", + " -0.01284331, 0.014344065, -0.007968607, -0.03899479, 0.039327156,\n", + " -0.047726233, 0.009559004, -0.05302522, 0.011498492, -0.0055542476,\n", + " -0.0020940166, -0.029262392, -0.025919685, 0.024261741, -0.0010863725,\n", + " 0.0074619935, 0.014191284, -0.009054746, -0.038633537, 0.039744128,\n", + " 0.012625762, 0.030490868, 0.013526139, -0.024638629, -0.011268263,\n", + " -0.012759613, -0.04693565, -0.013087251, -0.01971696, 0.0125782555,\n", + " 0.024156926, -0.011638484, 0.017364893, -0.0405832, -0.0032466082,\n", + " -0.01611277, -0.022583133, 0.019492855, -0.03664484, -0.022627067,\n", + " 0.011026938, -0.014631298, 0.043255687, -0.029447634, 0.017212389,\n", + " 0.029366229, -0.041978795, 0.005347565, -0.0106230285, -0.008334342,\n", + " -0.008841154, 0.045096103, 0.03996879, -0.002039457, -0.0051824683,\n", + " -0.019464444, 0.092018366, -0.009283633, -0.020052811, 0.0043408144,\n", + " -0.029403884, 0.02587689, -0.027253918, 0.0159064, 0.0421537,\n", + " 0.05078811, -0.012380686, -0.018032575, 0.01711449, 0.03636163,\n", + " -0.014590949, -0.015076142, 0.00018201554, 0.002490666, 0.044776678,\n", + " 0.05301749, -0.007891316, 0.028668318, -0.0016632816, 0.04487743,\n", + " -0.032529455, -0.040372133, -0.020566158, -0.011109745, -0.01724949,\n", + " -0.0047519016, -0.041635286, 0.0068111843, 0.039498538, -0.02491227,\n", + " 0.016853934, -0.017926402, -0.006154979, 0.025893573, 0.015262395\n", + "]\n" + ] + } + ], + "source": [ + "\n", + "\n", + " const text2 = \"LangGraph is a library for building stateful, multi-actor applications with LLMs\";\n", + "\n", + " const vectors = await embeddings.embedDocuments([text, text2]);\n", + " \n", + " console.log(vectors[0].slice(0, 100));\n", + " console.log(vectors[1].slice(0, 100));\n", + " " + ] + }, + { + "cell_type": "markdown", + "id": "8938e581", + "metadata": {}, + "source": [ + "## API reference\n", + "\n", + "For detailed documentation of all __module_name__ features and configurations head to the API reference: __api_ref_module__" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "JavaScript (Node.js)", + "language": "javascript", + "name": "javascript" + }, + "language_info": { + "file_extension": ".js", + "mimetype": "application/javascript", + "name": "javascript", + "version": "20.17.0" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/libs/langchain-community/.gitignore b/libs/langchain-community/.gitignore index 676b316638e8..890c93717dea 100644 --- a/libs/langchain-community/.gitignore +++ b/libs/langchain-community/.gitignore @@ -162,6 +162,10 @@ embeddings/hf_transformers.cjs embeddings/hf_transformers.js embeddings/hf_transformers.d.ts embeddings/hf_transformers.d.cts +embeddings/ibm.cjs +embeddings/ibm.js +embeddings/ibm.d.ts +embeddings/ibm.d.cts embeddings/jina.cjs embeddings/jina.js embeddings/jina.d.ts @@ -254,6 +258,10 @@ llms/hf.cjs llms/hf.js llms/hf.d.ts llms/hf.d.cts +llms/ibm.cjs +llms/ibm.js +llms/ibm.d.ts +llms/ibm.d.cts llms/llama_cpp.cjs llms/llama_cpp.js llms/llama_cpp.d.ts @@ -522,6 +530,10 @@ chat_models/friendli.cjs chat_models/friendli.js chat_models/friendli.d.ts chat_models/friendli.d.cts +chat_models/ibm.cjs +chat_models/ibm.js +chat_models/ibm.d.ts +chat_models/ibm.d.cts chat_models/iflytek_xinghuo.cjs chat_models/iflytek_xinghuo.js chat_models/iflytek_xinghuo.d.ts diff --git a/libs/langchain-community/langchain.config.js b/libs/langchain-community/langchain.config.js index 78118b4d1d5e..63b495f92f2c 100644 --- a/libs/langchain-community/langchain.config.js +++ b/libs/langchain-community/langchain.config.js @@ -29,6 +29,7 @@ export const config = { "notion-to-md/build/utils/notion.js", "@getzep/zep-cloud/api", "@supabase/postgrest-js", + "@ibm-cloud/watsonx-ai/dist/watsonx-ai-ml/vml_v1.js", ], entrypoints: { load: "load/index", @@ -75,6 +76,7 @@ export const config = { "embeddings/gradient_ai": "embeddings/gradient_ai", "embeddings/hf": "embeddings/hf", "embeddings/hf_transformers": "embeddings/hf_transformers", + "embeddings/ibm": "embeddings/ibm", "embeddings/jina": "embeddings/jina", "embeddings/llama_cpp": "embeddings/llama_cpp", "embeddings/minimax": "embeddings/minimax", @@ -99,6 +101,7 @@ export const config = { "llms/friendli": "llms/friendli", "llms/gradient_ai": "llms/gradient_ai", "llms/hf": "llms/hf", + "llms/ibm": "llms/ibm", "llms/llama_cpp": "llms/llama_cpp", "llms/ollama": "llms/ollama", "llms/portkey": "llms/portkey", @@ -168,6 +171,7 @@ export const config = { "chat_models/deepinfra": "chat_models/deepinfra", "chat_models/fireworks": "chat_models/fireworks", "chat_models/friendli": "chat_models/friendli", + "chat_models/ibm": "chat_models/ibm", "chat_models/iflytek_xinghuo": "chat_models/iflytek_xinghuo/index", "chat_models/iflytek_xinghuo/web": "chat_models/iflytek_xinghuo/web", "chat_models/llama_cpp": "chat_models/llama_cpp", @@ -336,6 +340,7 @@ export const config = { "embeddings/tensorflow", "embeddings/hf", "embeddings/hf_transformers", + "embeddings/ibm", "embeddings/jina", "embeddings/llama_cpp", "embeddings/gradient_ai", @@ -349,6 +354,7 @@ export const config = { "llms/gradient_ai", "llms/hf", "llms/raycast", + "llms/ibm", "llms/replicate", "llms/sagemaker_endpoint", "llms/watsonx_ai", @@ -410,6 +416,7 @@ export const config = { "chat_models/premai", "chat_models/tencent_hunyuan", "chat_models/tencent_hunyuan/web", + "chat_models/ibm", "chat_models/iflytek_xinghuo", "chat_models/iflytek_xinghuo/web", "chat_models/webllm", diff --git a/libs/langchain-community/package.json b/libs/langchain-community/package.json index 158b27368022..cf2340e1779f 100644 --- a/libs/langchain-community/package.json +++ b/libs/langchain-community/package.json @@ -77,6 +77,7 @@ "@google-cloud/storage": "^7.7.0", "@gradientai/nodejs-sdk": "^1.2.0", "@huggingface/inference": "^2.6.4", + "@ibm-cloud/watsonx-ai": "^1.1.0", "@jest/globals": "^29.5.0", "@langchain/core": "workspace:*", "@langchain/scripts": ">=0.1.0 <0.2.0", @@ -170,6 +171,7 @@ "hdb": "0.19.8", "hnswlib-node": "^3.0.0", "html-to-text": "^9.0.5", + "ibm-cloud-sdk-core": "^5.0.2", "ignore": "^5.2.0", "interface-datastore": "^8.2.11", "ioredis": "^5.3.2", @@ -243,6 +245,7 @@ "@google-cloud/storage": "^6.10.1 || ^7.7.0", "@gradientai/nodejs-sdk": "^1.2.0", "@huggingface/inference": "^2.6.4", + "@ibm-cloud/watsonx-ai": "*", "@langchain/core": ">=0.2.21 <0.4.0", "@layerup/layerup-security": "^1.5.12", "@libsql/client": "^0.14.0", @@ -300,6 +303,7 @@ "googleapis": "*", "hnswlib-node": "^3.0.0", "html-to-text": "^9.0.5", + "ibm-cloud-sdk-core": "*", "ignore": "^5.2.0", "interface-datastore": "^8.2.11", "ioredis": "^5.3.2", @@ -1073,6 +1077,15 @@ "import": "./embeddings/hf_transformers.js", "require": "./embeddings/hf_transformers.cjs" }, + "./embeddings/ibm": { + "types": { + "import": "./embeddings/ibm.d.ts", + "require": "./embeddings/ibm.d.cts", + "default": "./embeddings/ibm.d.ts" + }, + "import": "./embeddings/ibm.js", + "require": "./embeddings/ibm.cjs" + }, "./embeddings/jina": { "types": { "import": "./embeddings/jina.d.ts", @@ -1280,6 +1293,15 @@ "import": "./llms/hf.js", "require": "./llms/hf.cjs" }, + "./llms/ibm": { + "types": { + "import": "./llms/ibm.d.ts", + "require": "./llms/ibm.d.cts", + "default": "./llms/ibm.d.ts" + }, + "import": "./llms/ibm.js", + "require": "./llms/ibm.cjs" + }, "./llms/llama_cpp": { "types": { "import": "./llms/llama_cpp.d.ts", @@ -1883,6 +1905,15 @@ "import": "./chat_models/friendli.js", "require": "./chat_models/friendli.cjs" }, + "./chat_models/ibm": { + "types": { + "import": "./chat_models/ibm.d.ts", + "require": "./chat_models/ibm.d.cts", + "default": "./chat_models/ibm.d.ts" + }, + "import": "./chat_models/ibm.js", + "require": "./chat_models/ibm.cjs" + }, "./chat_models/iflytek_xinghuo": { "types": { "import": "./chat_models/iflytek_xinghuo.d.ts", @@ -3194,6 +3225,10 @@ "embeddings/hf_transformers.js", "embeddings/hf_transformers.d.ts", "embeddings/hf_transformers.d.cts", + "embeddings/ibm.cjs", + "embeddings/ibm.js", + "embeddings/ibm.d.ts", + "embeddings/ibm.d.cts", "embeddings/jina.cjs", "embeddings/jina.js", "embeddings/jina.d.ts", @@ -3286,6 +3321,10 @@ "llms/hf.js", "llms/hf.d.ts", "llms/hf.d.cts", + "llms/ibm.cjs", + "llms/ibm.js", + "llms/ibm.d.ts", + "llms/ibm.d.cts", "llms/llama_cpp.cjs", "llms/llama_cpp.js", "llms/llama_cpp.d.ts", @@ -3554,6 +3593,10 @@ "chat_models/friendli.js", "chat_models/friendli.d.ts", "chat_models/friendli.d.cts", + "chat_models/ibm.cjs", + "chat_models/ibm.js", + "chat_models/ibm.d.ts", + "chat_models/ibm.d.cts", "chat_models/iflytek_xinghuo.cjs", "chat_models/iflytek_xinghuo.js", "chat_models/iflytek_xinghuo.d.ts", diff --git a/libs/langchain-community/src/chat_models/ibm.ts b/libs/langchain-community/src/chat_models/ibm.ts new file mode 100644 index 000000000000..d0fae3ac15ce --- /dev/null +++ b/libs/langchain-community/src/chat_models/ibm.ts @@ -0,0 +1,860 @@ +import { + AIMessage, + AIMessageChunk, + ChatMessage, + ChatMessageChunk, + FunctionMessageChunk, + HumanMessageChunk, + isAIMessage, + MessageType, + ToolMessageChunk, + UsageMetadata, + type BaseMessage, +} from "@langchain/core/messages"; +import { + BaseLanguageModelInput, + FunctionDefinition, + StructuredOutputMethodOptions, + type BaseLanguageModelCallOptions, +} from "@langchain/core/language_models/base"; +import { CallbackManagerForLLMRun } from "@langchain/core/callbacks/manager"; +import { + BaseChatModel, + BindToolsInput, + LangSmithParams, + type BaseChatModelParams, +} from "@langchain/core/language_models/chat_models"; +import { + ChatGeneration, + ChatGenerationChunk, + ChatResult, +} from "@langchain/core/outputs"; +import { AsyncCaller } from "@langchain/core/utils/async_caller"; +import { + TextChatConstants, + TextChatMessagesTextChatMessageAssistant, + TextChatParameterTools, + TextChatParams, + TextChatResponse, + TextChatResponseFormat, + TextChatResultChoice, + TextChatResultMessage, + TextChatToolCall, + TextChatToolChoiceTool, + TextChatUsage, +} from "@ibm-cloud/watsonx-ai/dist/watsonx-ai-ml/vml_v1.js"; +import { WatsonXAI } from "@ibm-cloud/watsonx-ai"; +import { + convertLangChainToolCallToOpenAI, + makeInvalidToolCall, + parseToolCall, +} from "@langchain/core/output_parsers/openai_tools"; +import { ToolCallChunk } from "@langchain/core/messages/tool"; +import { + Runnable, + RunnablePassthrough, + RunnableSequence, +} from "@langchain/core/runnables"; +import { z } from "zod"; +import { + BaseLLMOutputParser, + JsonOutputParser, + StructuredOutputParser, +} from "@langchain/core/output_parsers"; +import { isZodSchema } from "@langchain/core/utils/types"; +import { zodToJsonSchema } from "zod-to-json-schema"; +import { NewTokenIndices } from "@langchain/core/callbacks/base"; +import { WatsonxAuth, WatsonxParams } from "../types/ibm.js"; +import { + _convertToolCallIdToMistralCompatible, + authenticateAndSetInstance, + WatsonxToolsOutputParser, +} from "../utils/ibm.js"; + +export interface WatsonxDeltaStream { + role?: string; + content?: string; + tool_calls?: TextChatToolCall[]; + refusal?: string; +} + +export interface WatsonxCallParams + extends Partial< + Omit< + TextChatParams, + | "toolChoiceOption" + | "toolChoice" + | "frequencyPenalty" + | "topLogprobs" + | "maxTokens" + | "presencePenalty" + | "responseFormat" + | "timeLimit" + | "modelId" + > + > { + maxRetries?: number; + tool_choice?: TextChatToolChoiceTool; + tool_choice_option?: TextChatConstants.ToolChoiceOption | string; + frequency_penalty?: number; + top_logprobs?: number; + max_new_tokens?: number; + presence_penalty?: number; + top_p?: number; + time_limit?: number; + response_format?: TextChatResponseFormat; +} +export interface WatsonxCallOptionsChat + extends Omit, + WatsonxCallParams { + promptIndex?: number; +} + +type ChatWatsonxToolType = BindToolsInput | TextChatParameterTools; + +export interface ChatWatsonxInput extends BaseChatModelParams, WatsonxParams { + streaming?: boolean; +} + +function _convertToValidToolId(modelId: string, tool_call_id: string) { + if (modelId.startsWith("mistralai")) + return _convertToolCallIdToMistralCompatible(tool_call_id); + else return tool_call_id; +} + +function _convertToolToWatsonxTool( + tools: ChatWatsonxToolType[] +): WatsonXAI.TextChatParameterTools[] { + return tools.map((tool) => { + if ("type" in tool) { + return tool as WatsonXAI.TextChatParameterTools; + } + return { + type: "function", + function: { + name: tool.name, + description: tool.description ?? "Tool: " + tool.name, + parameters: zodToJsonSchema(tool.schema), + }, + }; + }); +} + +function _convertMessagesToWatsonxMessages( + messages: BaseMessage[], + modelId: string +): TextChatResultMessage[] { + const getRole = (role: MessageType) => { + switch (role) { + case "human": + return "user"; + case "ai": + return "assistant"; + case "system": + return "system"; + case "tool": + return "tool"; + case "function": + return "function"; + default: + throw new Error(`Unknown message type: ${role}`); + } + }; + + const getTools = (message: BaseMessage): TextChatToolCall[] | undefined => { + if (isAIMessage(message) && message.tool_calls?.length) { + return message.tool_calls + .map((toolCall) => ({ + ...toolCall, + id: _convertToValidToolId(modelId, toolCall.id ?? ""), + })) + .map(convertLangChainToolCallToOpenAI) as TextChatToolCall[]; + } + return undefined; + }; + + return messages.map((message) => { + const toolCalls = getTools(message); + const content = toolCalls === undefined ? message.content : ""; + if ("tool_call_id" in message && typeof message.tool_call_id === "string") { + return { + role: getRole(message._getType()), + content, + name: message.name, + tool_call_id: _convertToValidToolId(modelId, message.tool_call_id), + }; + } + + return { + role: getRole(message._getType()), + content, + tool_calls: toolCalls, + }; + }) as TextChatResultMessage[]; +} + +function _watsonxResponseToChatMessage( + choice: TextChatResultChoice, + rawDataId: string, + usage?: TextChatUsage +): BaseMessage { + const { message } = choice; + if (!message) throw new Error("No message presented"); + const rawToolCalls: TextChatToolCall[] = message.tool_calls ?? []; + + switch (message.role) { + case "assistant": { + const toolCalls = []; + const invalidToolCalls = []; + for (const rawToolCall of rawToolCalls) { + try { + const parsed = parseToolCall(rawToolCall, { returnId: true }); + toolCalls.push(parsed); + } catch (e: any) { + invalidToolCalls.push(makeInvalidToolCall(rawToolCall, e.message)); + } + } + const additional_kwargs: Record = { + tool_calls: rawToolCalls.map((toolCall) => ({ + ...toolCall, + type: "function", + })), + }; + + return new AIMessage({ + id: rawDataId, + content: message.content ?? "", + tool_calls: toolCalls, + invalid_tool_calls: invalidToolCalls, + additional_kwargs, + usage_metadata: usage + ? { + input_tokens: usage.prompt_tokens ?? 0, + output_tokens: usage.completion_tokens ?? 0, + total_tokens: usage.total_tokens ?? 0, + } + : undefined, + }); + } + default: + return new ChatMessage(message.content ?? "", message.role ?? "unknown"); + } +} + +function _convertDeltaToMessageChunk( + delta: WatsonxDeltaStream, + rawData: TextChatResponse, + modelId: string, + usage?: TextChatUsage, + defaultRole?: TextChatMessagesTextChatMessageAssistant.Constants.Role +) { + if (delta.refusal) throw new Error(delta.refusal); + const rawToolCalls = delta.tool_calls?.length + ? delta.tool_calls?.map( + ( + toolCall, + index + ): TextChatToolCall & { + index: number; + type: "function"; + } => ({ + ...toolCall, + index, + id: _convertToValidToolId(modelId, toolCall.id), + type: "function", + }) + ) + : undefined; + + let role = "assistant"; + if (delta.role) { + role = delta.role; + } else if (defaultRole) { + role = defaultRole; + } + const content = delta.content ?? ""; + let additional_kwargs; + if (rawToolCalls) { + additional_kwargs = { + tool_calls: rawToolCalls, + }; + } else { + additional_kwargs = {}; + } + + if (role === "user") { + return new HumanMessageChunk({ content }); + } else if (role === "assistant") { + const toolCallChunks: ToolCallChunk[] = []; + if (rawToolCalls && rawToolCalls.length > 0) + for (const rawToolCallChunk of rawToolCalls) { + toolCallChunks.push({ + name: rawToolCallChunk.function?.name, + args: rawToolCallChunk.function?.arguments, + id: rawToolCallChunk.id, + index: rawToolCallChunk.index, + type: "tool_call_chunk", + }); + } + + return new AIMessageChunk({ + content, + tool_call_chunks: toolCallChunks, + additional_kwargs, + usage_metadata: { + input_tokens: usage?.prompt_tokens ?? 0, + output_tokens: usage?.completion_tokens ?? 0, + total_tokens: usage?.total_tokens ?? 0, + }, + id: rawData.id, + }); + } else if (role === "tool") { + if (rawToolCalls) + return new ToolMessageChunk({ + content, + additional_kwargs, + tool_call_id: _convertToValidToolId(modelId, rawToolCalls?.[0].id), + }); + } else if (role === "function") { + return new FunctionMessageChunk({ + content, + additional_kwargs, + }); + } else { + return new ChatMessageChunk({ content, role }); + } + return null; +} + +export class ChatWatsonx< + CallOptions extends WatsonxCallOptionsChat = WatsonxCallOptionsChat + > + extends BaseChatModel + implements ChatWatsonxInput +{ + static lc_name() { + return "ChatWatsonx"; + } + + lc_serializable = true; + + get lc_secrets(): { [key: string]: string } { + return { + authenticator: "AUTHENTICATOR", + apiKey: "WATSONX_AI_APIKEY", + apikey: "WATSONX_AI_APIKEY", + watsonxAIAuthType: "WATSONX_AI_AUTH_TYPE", + watsonxAIApikey: "WATSONX_AI_APIKEY", + watsonxAIBearerToken: "WATSONX_AI_BEARER_TOKEN", + watsonxAIUsername: "WATSONX_AI_USERNAME", + watsonxAIPassword: "WATSONX_AI_PASSWORD", + watsonxAIUrl: "WATSONX_AI_URL", + }; + } + + get lc_aliases(): { [key: string]: string } { + return { + authenticator: "authenticator", + apikey: "watsonx_ai_apikey", + apiKey: "watsonx_ai_apikey", + watsonxAIAuthType: "watsonx_ai_auth_type", + watsonxAIApikey: "watsonx_ai_apikey", + watsonxAIBearerToken: "watsonx_ai_bearer_token", + watsonxAIUsername: "watsonx_ai_username", + watsonxAIPassword: "watsonx_ai_password", + watsonxAIUrl: "watsonx_ai_url", + }; + } + + getLsParams(options: this["ParsedCallOptions"]): LangSmithParams { + const params = this.invocationParams(options); + return { + ls_provider: "watsonx", + ls_model_name: this.model, + ls_model_type: "chat", + ls_temperature: params.temperature ?? undefined, + ls_max_tokens: params.maxTokens ?? undefined, + }; + } + + model = "mistralai/mistral-large"; + + version = "2024-05-31"; + + max_new_tokens = 100; + + maxRetries = 0; + + serviceUrl: string; + + spaceId?: string; + + projectId?: string; + + frequency_penalty?: number; + + logprobs?: boolean; + + top_logprobs?: number; + + n?: number; + + presence_penalty?: number; + + temperature?: number; + + top_p?: number; + + time_limit?: number; + + maxConcurrency?: number; + + service: WatsonXAI; + + response_format?: TextChatResponseFormat | string; + + streaming: boolean; + + constructor( + fields: ChatWatsonxInput & + WatsonxAuth & + Partial> + ) { + super(fields); + if ( + (fields.projectId && fields.spaceId) || + (fields.idOrName && fields.projectId) || + (fields.spaceId && fields.idOrName) + ) + throw new Error("Maximum 1 id type can be specified per instance"); + + if (!fields.projectId && !fields.spaceId && !fields.idOrName) + throw new Error( + "No id specified! At least ide of 1 type has to be specified" + ); + this.projectId = fields?.projectId; + this.spaceId = fields?.spaceId; + this.temperature = fields?.temperature; + this.maxRetries = fields?.maxRetries || this.maxRetries; + this.maxConcurrency = fields?.maxConcurrency; + this.frequency_penalty = fields?.frequency_penalty; + this.top_logprobs = fields?.top_logprobs; + this.max_new_tokens = fields?.max_new_tokens ?? this.max_new_tokens; + this.presence_penalty = fields?.presence_penalty; + this.top_p = fields?.top_p; + this.time_limit = fields?.time_limit; + this.response_format = fields?.response_format ?? this.response_format; + this.serviceUrl = fields?.serviceUrl; + this.streaming = fields?.streaming ?? this.streaming; + this.n = fields?.n ?? this.n; + this.model = fields?.model ?? this.model; + this.version = fields?.version ?? this.version; + + const { + watsonxAIApikey, + watsonxAIAuthType, + watsonxAIBearerToken, + watsonxAIUsername, + watsonxAIPassword, + watsonxAIUrl, + version, + serviceUrl, + } = fields; + + const auth = authenticateAndSetInstance({ + watsonxAIApikey, + watsonxAIAuthType, + watsonxAIBearerToken, + watsonxAIUsername, + watsonxAIPassword, + watsonxAIUrl, + version, + serviceUrl, + }); + if (auth) this.service = auth; + else throw new Error("You have not provided one type of authentication"); + } + + _llmType() { + return "watsonx"; + } + + invocationParams(options: this["ParsedCallOptions"]) { + return { + maxTokens: options.max_new_tokens ?? this.max_new_tokens, + temperature: options?.temperature ?? this.temperature, + timeLimit: options?.time_limit ?? this.time_limit, + topP: options?.top_p ?? this.top_p, + presencePenalty: options?.presence_penalty ?? this.presence_penalty, + n: options?.n ?? this.n, + topLogprobs: options?.top_logprobs ?? this.top_logprobs, + logprobs: options?.logprobs ?? this?.logprobs, + frequencyPenalty: options?.frequency_penalty ?? this.frequency_penalty, + tools: options.tools + ? _convertToolToWatsonxTool(options.tools) + : undefined, + toolChoice: options.tool_choice, + responseFormat: options.response_format, + toolChoiceOption: options.tool_choice_option, + }; + } + + override bindTools( + tools: ChatWatsonxToolType[], + kwargs?: Partial + ): Runnable { + return this.bind({ + tools: _convertToolToWatsonxTool(tools), + ...kwargs, + } as CallOptions); + } + + scopeId() { + if (this.projectId) + return { projectId: this.projectId, modelId: this.model }; + else return { spaceId: this.spaceId, modelId: this.model }; + } + + async completionWithRetry( + callback: () => T, + options?: this["ParsedCallOptions"] + ) { + const caller = new AsyncCaller({ + maxConcurrency: options?.maxConcurrency || this.maxConcurrency, + maxRetries: this.maxRetries, + }); + const result = options + ? caller.callWithOptions( + { + signal: options.signal, + }, + async () => callback() + ) + : caller.call(async () => callback()); + + return result; + } + + async _generate( + messages: BaseMessage[], + options: this["ParsedCallOptions"], + runManager?: CallbackManagerForLLMRun + ): Promise { + if (this.streaming) { + const stream = this._streamResponseChunks(messages, options, runManager); + const finalChunks: Record = {}; + let tokenUsage: UsageMetadata = { + input_tokens: 0, + output_tokens: 0, + total_tokens: 0, + }; + const tokenUsages: UsageMetadata[] = []; + for await (const chunk of stream) { + const message = chunk.message as AIMessageChunk; + if (message?.usage_metadata) { + const completion = chunk.generationInfo?.completion; + if (tokenUsages[completion]) + tokenUsages[completion].output_tokens += + message.usage_metadata.output_tokens; + else tokenUsages[completion] = message.usage_metadata; + } + chunk.message.response_metadata = { + ...chunk.generationInfo, + ...chunk.message.response_metadata, + }; + + const index = + (chunk.generationInfo as NewTokenIndices)?.completion ?? 0; + if (finalChunks[index] === undefined) { + finalChunks[index] = chunk; + } else { + finalChunks[index] = finalChunks[index].concat(chunk); + } + } + tokenUsage = tokenUsages.reduce((acc, curr) => { + return { + input_tokens: acc.input_tokens + curr.input_tokens, + output_tokens: acc.output_tokens + curr.output_tokens, + total_tokens: acc.total_tokens + curr.total_tokens, + }; + }); + const generations = Object.entries(finalChunks) + .sort(([aKey], [bKey]) => parseInt(aKey, 10) - parseInt(bKey, 10)) + .map(([_, value]) => value); + return { generations, llmOutput: { tokenUsage } }; + } else { + const params: Omit = { + ...this.invocationParams(options), + ...this.scopeId(), + }; + const watsonxMessages = _convertMessagesToWatsonxMessages( + messages, + this.model + ); + const callback = () => + this.service.textChat({ + ...params, + messages: watsonxMessages, + }); + const { result } = await this.completionWithRetry(callback, options); + + const generations: ChatGeneration[] = []; + for (const part of result.choices) { + const generation: ChatGeneration = { + text: part.message?.content ?? "", + message: _watsonxResponseToChatMessage( + part, + result.id, + result?.usage + ), + }; + if (part.finish_reason) { + generation.generationInfo = { finish_reason: part.finish_reason }; + } + generations.push(generation); + } + if (options.signal?.aborted) { + throw new Error("AbortError"); + } + + return { + generations, + llmOutput: { + tokenUsage: result?.usage, + }, + }; + } + } + + async *_streamResponseChunks( + messages: BaseMessage[], + options: this["ParsedCallOptions"], + _runManager?: CallbackManagerForLLMRun + ): AsyncGenerator { + const params = { ...this.invocationParams(options), ...this.scopeId() }; + const watsonxMessages = _convertMessagesToWatsonxMessages( + messages, + this.model + ); + const callback = () => + this.service.textChatStream({ + ...params, + messages: watsonxMessages, + returnObject: true, + }); + const stream = await this.completionWithRetry(callback, options); + let defaultRole; + for await (const chunk of stream) { + if (options.signal?.aborted) { + throw new Error("AbortError"); + } + const { data } = chunk; + const choice = data.choices[0] as TextChatResultChoice & + Record<"delta", TextChatResultMessage>; + if (choice && !("delta" in choice)) { + continue; + } + const delta = choice?.delta; + + if (!delta) { + continue; + } + + const newTokenIndices = { + prompt: options.promptIndex ?? 0, + completion: choice.index ?? 0, + }; + + const generationInfo = { + ...newTokenIndices, + finish_reason: choice.finish_reason, + }; + + const message = _convertDeltaToMessageChunk( + delta, + data, + this.model, + chunk.data.usage, + defaultRole + ); + + defaultRole = + (delta.role as TextChatMessagesTextChatMessageAssistant.Constants.Role) ?? + defaultRole; + + if (message === null || (!delta.content && !delta.tool_calls)) { + continue; + } + + const generationChunk = new ChatGenerationChunk({ + message, + text: delta.content ?? "", + generationInfo, + }); + + yield generationChunk; + + void _runManager?.handleLLMNewToken( + generationChunk.text ?? "", + newTokenIndices, + undefined, + undefined, + undefined, + { chunk: generationChunk } + ); + } + } + + /** @ignore */ + _combineLLMOutput() { + return []; + } + + withStructuredOutput< + // eslint-disable-next-line @typescript-eslint/no-explicit-any + RunOutput extends Record = Record + >( + outputSchema: + | z.ZodType + // eslint-disable-next-line @typescript-eslint/no-explicit-any + | Record, + config?: StructuredOutputMethodOptions + ): Runnable; + + withStructuredOutput< + // eslint-disable-next-line @typescript-eslint/no-explicit-any + RunOutput extends Record = Record + >( + outputSchema: + | z.ZodType + // eslint-disable-next-line @typescript-eslint/no-explicit-any + | Record, + config?: StructuredOutputMethodOptions + ): Runnable; + + withStructuredOutput< + // eslint-disable-next-line @typescript-eslint/no-explicit-any + RunOutput extends Record = Record + >( + outputSchema: + | z.ZodType + // eslint-disable-next-line @typescript-eslint/no-explicit-any + | Record, + config?: StructuredOutputMethodOptions + ): + | Runnable + | Runnable< + BaseLanguageModelInput, + { raw: BaseMessage; parsed: RunOutput } + > { + // eslint-disable-next-line @typescript-eslint/no-explicit-any + const schema: z.ZodType | Record = outputSchema; + const name = config?.name; + const method = config?.method; + const includeRaw = config?.includeRaw; + let functionName = name ?? "extract"; + let outputParser: BaseLLMOutputParser; + let llm: Runnable; + if (method === "jsonMode") { + const options = { + response_format: { type: "json_object" }, + } as Partial; + llm = this.bind(options); + + if (isZodSchema(schema)) { + outputParser = StructuredOutputParser.fromZodSchema(schema); + } else { + outputParser = new JsonOutputParser(); + } + } else { + if (isZodSchema(schema)) { + const asJsonSchema = zodToJsonSchema(schema); + llm = this.bind({ + tools: [ + { + type: "function" as const, + function: { + name: functionName, + description: asJsonSchema.description, + parameters: asJsonSchema, + }, + }, + ], + // Ideally that would be set to required but this is not supported yet + tool_choice: { + type: "function", + function: { + name: functionName, + }, + }, + } as Partial); + outputParser = new WatsonxToolsOutputParser({ + returnSingle: true, + keyName: functionName, + zodSchema: schema, + }); + } else { + let openAIFunctionDefinition: FunctionDefinition; + if ( + typeof schema.name === "string" && + typeof schema.parameters === "object" && + schema.parameters != null + ) { + openAIFunctionDefinition = schema as FunctionDefinition; + functionName = schema.name; + } else { + openAIFunctionDefinition = { + name: functionName, + description: schema.description ?? "", + parameters: schema, + }; + } + llm = this.bind({ + tools: [ + { + type: "function" as const, + function: openAIFunctionDefinition, + }, + ], + // Ideally that would be set to required but this is not supported yet + tool_choice: { + type: "function", + function: { + name: functionName, + }, + }, + } as Partial); + outputParser = new WatsonxToolsOutputParser({ + returnSingle: true, + keyName: functionName, + }); + } + } + + if (!includeRaw) { + return llm.pipe(outputParser) as Runnable< + BaseLanguageModelInput, + RunOutput + >; + } + + const parserAssign = RunnablePassthrough.assign({ + parsed: (input: any, config) => outputParser.invoke(input.raw, config), + }); + const parserNone = RunnablePassthrough.assign({ + parsed: () => null, + }); + const parsedWithFallback = parserAssign.withFallbacks({ + fallbacks: [parserNone], + }); + return RunnableSequence.from< + BaseLanguageModelInput, + { raw: BaseMessage; parsed: RunOutput } + >([ + { + raw: llm, + }, + parsedWithFallback, + ]); + } +} diff --git a/libs/langchain-community/src/chat_models/tests/ibm.int.test.ts b/libs/langchain-community/src/chat_models/tests/ibm.int.test.ts new file mode 100644 index 000000000000..85f33d733e6d --- /dev/null +++ b/libs/langchain-community/src/chat_models/tests/ibm.int.test.ts @@ -0,0 +1,844 @@ +/* eslint-disable no-process-env */ +import { + AIMessage, + AIMessageChunk, + HumanMessage, + SystemMessage, +} from "@langchain/core/messages"; +import { z } from "zod"; +import { StringOutputParser } from "@langchain/core/output_parsers"; +import { CallbackManager } from "@langchain/core/callbacks/manager"; +import { LLMResult } from "@langchain/core/outputs"; +import { ChatPromptTemplate } from "@langchain/core/prompts"; +import { tool } from "@langchain/core/tools"; +import { NewTokenIndices } from "@langchain/core/callbacks/base"; +import * as fs from "node:fs/promises"; +import { fileURLToPath } from "node:url"; +import * as path from "node:path"; +import { ChatWatsonx } from "../ibm.js"; + +describe("Tests for chat", () => { + describe("Test ChatWatsonx invoke and generate", () => { + test("Basic invoke", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + const res = await service.invoke("Print hello world"); + expect(res).toBeInstanceOf(AIMessage); + }); + test("Basic generate", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + const message = new HumanMessage("Hello"); + const res = await service.generate([[message], [message]]); + expect(res.generations.length).toBe(2); + }); + test("Invoke with system message", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + const messages = [ + new SystemMessage("Translate the following from English into Italian"), + new HumanMessage("hi!"), + ]; + const res = await service.invoke(messages); + expect(res).toBeInstanceOf(AIMessage); + }); + test("Invoke with output parser", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + const parser = new StringOutputParser(); + const messages = [ + new SystemMessage("Translate the following from English into Italian"), + new HumanMessage("hi!"), + ]; + const res = await service.invoke(messages); + const parsed = await parser.invoke(res); + expect(typeof parsed).toBe("string"); + }); + test("Invoke with prompt", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + const systemTemplate = "Translate the following into {language}:"; + const promptTemplate = ChatPromptTemplate.fromMessages([ + ["system", systemTemplate], + ["user", "{text}"], + ]); + const llmChain = promptTemplate.pipe(service); + const res = await llmChain.invoke({ language: "italian", text: "hi" }); + expect(res).toBeInstanceOf(AIMessage); + }); + test("Invoke with chat conversation", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + const res = await service.invoke([ + { role: "user", content: "Hi! I'm Bob" }, + { + role: "assistant", + content: "Hello Bob! How can I assist you today?", + }, + { role: "user", content: "What's my name?" }, + ]); + expect(res).toBeInstanceOf(AIMessage); + }); + test("Token usage", async () => { + let tokenUsage = { + completion_tokens: 0, + prompt_tokens: 0, + totalTokens: 0, + }; + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + callbackManager: CallbackManager.fromHandlers({ + async handleLLMEnd(output: LLMResult) { + tokenUsage = output.llmOutput?.tokenUsage; + }, + }), + }); + + const message = new HumanMessage("Hello"); + await service.invoke([message]); + expect(tokenUsage.prompt_tokens).toBeGreaterThan(0); + }); + test("Timeout", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + await expect(() => + service.invoke("Print hello world", { + timeout: 10, + }) + ).rejects.toThrow(); + }, 5000); + test("Controller options", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + const controller = new AbortController(); + await expect(() => { + const res = service.invoke("Print hello world", { + signal: controller.signal, + }); + controller.abort(); + return res; + }).rejects.toThrow(); + }, 5000); + }); + + describe("Test ChatWatsonx invoke and generate with stream mode", () => { + test("Basic invoke", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + const res = await service.invoke("Print hello world"); + + expect(res).toBeInstanceOf(AIMessage); + }); + test("Basic generate", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + const message = new HumanMessage("Hello"); + const res = await service.generate([[message], [message]]); + expect(res.generations.length).toBe(2); + }); + test("Generate with n>1", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + n: 3, + }); + const message = new HumanMessage("Print hello world"); + const res = await service.generate([[message]]); + for (const generation of res.generations) { + expect(generation.length).toBe(3); + for (const gen of generation) { + expect(typeof gen.text).toBe("string"); + } + } + }); + test("Generate with n>1 token count", async () => { + process.env.LANGCHAIN_CALLBACKS_BACKGROUND = "false"; + + let tokenUsage = { + input_tokens: 0, + output_tokens: 0, + total_tokens: 0, + }; + const generationsStreamed = [ + ["", ""], + ["", ""], + ]; + let tokenUsed = 0; + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + n: 2, + max_new_tokens: 5, + streaming: true, + callbackManager: CallbackManager.fromHandlers({ + async handleLLMEnd(output: LLMResult) { + const usage = output.llmOutput?.tokenUsage; + tokenUsage = { + input_tokens: usage.input_tokens + tokenUsage.input_tokens, + output_tokens: usage.output_tokens + tokenUsage.output_tokens, + total_tokens: usage.total_tokens + tokenUsage.total_tokens, + }; + }, + async handleLLMNewToken(token: string, idx: NewTokenIndices) { + const { prompt, completion } = idx; + generationsStreamed[prompt][completion] += token; + tokenUsed += 1; + }, + }), + }); + const message = new HumanMessage("Print hello world"); + const res = await service.generate([[message], [message]]); + + for (const generation of res.generations) { + expect(generation.length).toBe(2); + for (const gen of generation) { + expect(typeof gen.text).toBe("string"); + } + } + expect(tokenUsed).toBe(tokenUsage.output_tokens); + expect(res.generations.map((g) => g.map((gg) => gg.text))).toEqual( + generationsStreamed + ); + }); + test("Invoke with system message", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + const messages = [ + new SystemMessage("Translate the following from English into Italian"), + new HumanMessage("hi!"), + ]; + const res = await service.invoke(messages); + expect(res).toBeInstanceOf(AIMessage); + }); + test("Invoke with output parser", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + const parser = new StringOutputParser(); + const messages = [ + new SystemMessage("Translate the following from English into Italian"), + new HumanMessage("hi!"), + ]; + const res = await service.invoke(messages); + const parsed = await parser.invoke(res); + expect(typeof parsed).toBe("string"); + }); + test("Invoke with prompt", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + const systemTemplate = "Translate the following into {language}:"; + const promptTemplate = ChatPromptTemplate.fromMessages([ + ["system", systemTemplate], + ["user", "{text}"], + ]); + const llmChain = promptTemplate.pipe(service); + const res = await llmChain.invoke({ language: "italian", text: "hi" }); + expect(res).toBeInstanceOf(AIMessage); + }); + test("Invoke with chat conversation", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + const res = await service.invoke([ + { role: "user", content: "Hi! I'm Bob" }, + { + role: "assistant", + content: "Hello Bob! How can I assist you today?", + }, + { role: "user", content: "What's my name?" }, + ]); + expect(res).toBeInstanceOf(AIMessage); + }); + test("Token usage", async () => { + let tokenUsage = { + completion_tokens: 0, + prompt_tokens: 0, + totalTokens: 0, + }; + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + callbackManager: CallbackManager.fromHandlers({ + async handleLLMEnd(output: LLMResult) { + tokenUsage = output.llmOutput?.tokenUsage; + }, + }), + }); + + const message = new HumanMessage("Hello"); + await service.invoke([message]); + expect(tokenUsage.prompt_tokens).toBeGreaterThan(0); + }); + test("Timeout", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + await expect(() => + service.invoke("Print hello world", { + timeout: 10, + }) + ).rejects.toThrow(); + }, 5000); + test("Controller options", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + const controller = new AbortController(); + await expect(() => { + const res = service.invoke("Print hello world", { + signal: controller.signal, + }); + controller.abort(); + return res; + }).rejects.toThrow(); + }, 5000); + }); + + describe("Test ChatWatsonx stream", () => { + test("Basic stream", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + const prompt = ChatPromptTemplate.fromMessages([ + ["system", "You are a helpful assistant"], + ["human", "{input}"], + ]); + const res = await prompt.pipe(service).stream({ + input: "Print hello world.", + }); + const chunks = []; + for await (const chunk of res) { + chunks.push(chunk); + } + expect(chunks.length).toBeGreaterThan(1); + expect(chunks.join("").length).toBeGreaterThan(1); + }); + test("Timeout", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + await expect(() => + service.stream("Print hello world", { + timeout: 10, + }) + ).rejects.toThrow(); + }, 5000); + test("Controller options", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + const controller = new AbortController(); + await expect(async () => { + const res = await service.stream("Print hello world", { + signal: controller.signal, + }); + let hasEntered = false; + for await (const chunk of res) { + hasEntered = true; + expect(chunk).toBeDefined(); + controller.abort(); + } + expect(hasEntered).toBe(true); + }).rejects.toThrow(); + }, 5000); + test("Token count and response equality", async () => { + let generation = ""; + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + callbackManager: CallbackManager.fromHandlers({ + async handleLLMEnd(output: LLMResult) { + generation = output.generations[0][0].text; + }, + }), + }); + const prompt = ChatPromptTemplate.fromMessages([ + ["system", "You are a helpful assistant"], + ["human", "{input}"], + ]); + const res = await prompt.pipe(service).stream({ + input: "Print hello world", + }); + let tokenCount = 0; + const chunks = []; + for await (const chunk of res) { + tokenCount += 1; + chunks.push(chunk.content); + } + expect(tokenCount).toBeGreaterThan(1); + expect(chunks.join("")).toBe(generation); + }); + test("Token count usage_metadata", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + let res: AIMessageChunk | null = null; + const stream = await service.stream("Why is the sky blue? Be concise."); + for await (const chunk of stream) { + res = chunk; + } + expect(res?.usage_metadata).toBeDefined(); + if (!res?.usage_metadata) { + return; + } + expect(res.usage_metadata.input_tokens).toBeGreaterThan(1); + expect(res.usage_metadata.output_tokens).toBe(1); + expect(res.usage_metadata.total_tokens).toBe( + res.usage_metadata.input_tokens + res.usage_metadata.output_tokens + ); + }); + }); + + describe("Test tool usage", () => { + test("Passing tool to chat model", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + const calculatorSchema = z.object({ + operation: z + .enum(["add", "subtract", "multiply", "divide"]) + .describe("The type of operation to execute."), + number1: z.number().describe("The first number to operate on."), + number2: z.number().describe("The second number to operate on."), + }); + + const calculatorTool = tool( + async ({ + operation, + number1, + number2, + }: { + operation: string; + number1: number; + number2: number; + }) => { + // Functions must return strings + if (operation === "add") { + return `${number1 + number2}`; + } else if (operation === "subtract") { + return `${number1 - number2}`; + } else if (operation === "multiply") { + return `${number1 * number2}`; + } else if (operation === "divide") { + return `${number1 / number2}`; + } else { + throw new Error("Invalid operation."); + } + }, + { + name: "calculator", + description: "Can perform mathematical operations.", + schema: calculatorSchema, + } + ); + const llmWithTools = service.bindTools([calculatorTool]); + const res = await llmWithTools.invoke("What is 3 * 12"); + + expect(res).toBeInstanceOf(AIMessage); + expect(res.tool_calls?.[0].name).toBe("calculator"); + expect(typeof res.tool_calls?.[0].args?.operation).toBe("string"); + expect(typeof res.tool_calls?.[0].args?.number1).toBe("number"); + expect(typeof res.tool_calls?.[0].args?.number2).toBe("number"); + expect(res.response_metadata.finish_reason).toBe("tool_calls"); + }); + test("Passing tool to chat model extended", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + const calculatorSchema = z.object({ + operation: z + .enum(["add", "subtract", "multiply", "divide"]) + .describe("The type of operation to execute."), + number1: z.number().describe("The first number to operate on."), + number2: z.number().describe("The second number to operate on."), + }); + + const calculatorTool = tool( + async ({ + operation, + number1, + number2, + }: { + operation: string; + number1: number; + number2: number; + }) => { + // Functions must return strings + if (operation === "add") { + return `${number1 + number2}`; + } else if (operation === "subtract") { + return `${number1 - number2}`; + } else if (operation === "multiply") { + return `${number1 * number2}`; + } else if (operation === "divide") { + return `${number1 / number2}`; + } else { + throw new Error("Invalid operation."); + } + }, + { + name: "calculator", + description: "Can perform mathematical operations.", + schema: calculatorSchema, + } + ); + const llmWithTools = service.bindTools([calculatorTool]); + const res = await llmWithTools.invoke( + "What is 3 * 12? Also, what is 11 + 49?" + ); + + expect(res).toBeInstanceOf(AIMessage); + expect(res.tool_calls).toBeDefined(); + if (!res.tool_calls) return; + expect(res.tool_calls.length).toBe(2); + + for (const tool_call of res.tool_calls) { + expect(tool_call.name).toBe("calculator"); + expect(typeof tool_call.args?.operation).toBe("string"); + expect(typeof tool_call.args?.number1).toBe("number"); + expect(typeof tool_call.args?.number2).toBe("number"); + } + }); + test("Binding model-specific formats", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + + const modelWithTools = service.bind({ + tools: [ + { + type: "function", + function: { + name: "calculator", + description: "Can perform mathematical operations.", + parameters: { + type: "object", + properties: { + operation: { + type: "string", + description: "The type of operation to execute.", + enum: ["add", "subtract", "multiply", "divide"], + }, + number1: { type: "number", description: "First integer" }, + number2: { type: "number", description: "Second integer" }, + }, + required: ["number1", "number2"], + }, + }, + }, + ], + }); + const res = await modelWithTools.invoke("What is 32 * 122"); + + expect(res).toBeInstanceOf(AIMessage); + expect(res.tool_calls?.[0].name).toBe("calculator"); + expect(typeof res.tool_calls?.[0].args?.operation).toBe("string"); + expect(typeof res.tool_calls?.[0].args?.number1).toBe("number"); + expect(typeof res.tool_calls?.[0].args?.number2).toBe("number"); + expect(res.response_metadata.finish_reason).toBe("tool_calls"); + }); + test("Passing tool to chat model", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + const addTool = tool( + async (input) => { + return input.a + input.b; + }, + { + name: "add", + description: "Adds a and b.", + schema: z.object({ + a: z.number(), + b: z.number(), + }), + } + ); + + const multiplyTool = tool( + async (input) => { + return input.a * input.b; + }, + { + name: "multiply", + description: "Multiplies a and b.", + schema: z.object({ + a: z.number(), + b: z.number(), + }), + } + ); + const tools = [addTool, multiplyTool]; + + const modelWithTools = service.bindTools(tools); + const res = await modelWithTools.invoke( + "What is 3 * 12? Also, what is 11 + 49?" + ); + + expect(res).toBeInstanceOf(AIMessage); + expect(res.tool_calls).toBeDefined(); + if (!res.tool_calls) return; + expect(res.tool_calls.length).toBe(2); + + expect(res.tool_calls[0].name).not.toBe(res.tool_calls[1].name); + expect(res.tool_calls[0].args.a).not.toBe(res.tool_calls[1].args.a); + expect(res.tool_calls[0].args.b).not.toBe(res.tool_calls[1].args.b); + }); + }); + + describe("Test withStructuredOutput usage", () => { + test("Schema with zod", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + }); + const joke = z.object({ + setup: z.string().describe("The setup of the joke"), + punchline: z.string().describe("The punchline to the joke"), + rating: z + .number() + .optional() + .describe("How funny the joke is, from 1 to 10"), + }); + + const structuredLlm = service.withStructuredOutput(joke); + + const res = await structuredLlm.invoke("Tell me a joke about cats"); + expect("setup" in res).toBe(true); + expect("punchline" in res).toBe(true); + }); + + test("Schema with zod and stream", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + temperature: 0.2, + }); + const joke = z.object({ + setup: z.string().describe("The setup of the joke"), + punchline: z.string().describe("The punchline to the joke"), + rating: z + .number() + .optional() + .describe("How funny the joke is, from 1 to 10"), + }); + + const structuredLlm = service.withStructuredOutput(joke); + const res = await structuredLlm.stream("Tell me a joke about cats"); + let object = {}; + for await (const chunk of res) { + expect(typeof chunk).toBe("object"); + object = chunk; + } + expect("setup" in object).toBe(true); + expect("punchline" in object).toBe(true); + }); + test("Schema with object", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + temperature: 0.2, + }); + const structuredLlm = service.withStructuredOutput({ + name: "joke", + description: "Joke to tell user.", + parameters: { + title: "Joke", + type: "object", + properties: { + setup: { type: "string", description: "The setup for the joke" }, + punchline: { type: "string", description: "The joke's punchline" }, + }, + required: ["setup", "punchline"], + }, + }); + + const res = await structuredLlm.invoke("Tell me a joke about cats"); + expect(res).toBeDefined(); + expect(typeof res.setup).toBe("string"); + expect(typeof res.punchline).toBe("string"); + }); + test("Schema with rawOutput", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + temperature: 0.2, + }); + const structuredLlm = service.withStructuredOutput( + { + name: "joke", + description: "Joke to tell user.", + parameters: { + title: "Joke", + type: "object", + properties: { + setup: { type: "string", description: "The setup for the joke" }, + punchline: { + type: "string", + description: "The joke's punchline", + }, + }, + required: ["setup", "punchline"], + }, + }, + { includeRaw: true } + ); + + const res = await structuredLlm.invoke("Tell me a joke about cats"); + expect(res.raw).toBeInstanceOf(AIMessage); + expect(typeof res.parsed.setup).toBe("string"); + expect(typeof res.parsed.setup).toBe("string"); + }); + test("Schema with zod and JSON mode", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + temperature: 0, + }); + const calculatorSchema = z.object({ + operation: z.enum(["add", "subtract", "multiply", "divide"]), + number1: z.number(), + number2: z.number(), + }); + const modelWithStructuredOutput = service.withStructuredOutput( + calculatorSchema, + { + name: "calculator", + method: "jsonMode", + } + ); + const prompt = ChatPromptTemplate.fromMessages([ + { + role: "system", + content: `Reply structure should be type of JSON as followed: + 'operation': the type of operation to execute, either 'add', 'subtract', 'multiply' or 'divide', + 'number1': the first number to operate on, + 'number2': the second number to operate on. + `, + }, + { role: "human", content: "What is 21 * 12?" }, + ]); + const modelWithStructuredOutoputJson = prompt.pipe( + modelWithStructuredOutput + ); + const result = await modelWithStructuredOutoputJson.invoke(""); + expect(typeof result.operation).toBe("string"); + expect(typeof result.number1).toBe("number"); + expect(typeof result.number2).toBe("number"); + }); + }); + + describe("Test image input", () => { + test("Image input", async () => { + const service = new ChatWatsonx({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + model: "meta-llama/llama-3-2-11b-vision-instruct", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + max_new_tokens: 100, + }); + const __filename = fileURLToPath(import.meta.url); + const __dirname = path.dirname(__filename); + const encodedString = await fs.readFile( + path.join(__dirname, "/data/hotdog.jpg") + ); + const question = "What is on the picture"; + const messages = [ + { + role: "user", + content: [ + { + type: "text", + text: question, + }, + { + type: "image_url", + image_url: { + url: + "data:image/jpeg;base64," + encodedString.toString("base64"), + }, + }, + ], + }, + ]; + const res = await service.stream(messages); + const chunks = []; + for await (const chunk of res) { + expect(chunk).toBeInstanceOf(AIMessageChunk); + chunks.push(chunk.content); + } + expect(typeof chunks.join("")).toBe("string"); + }); + }); +}); diff --git a/libs/langchain-community/src/chat_models/tests/ibm.standard.int.test.ts b/libs/langchain-community/src/chat_models/tests/ibm.standard.int.test.ts new file mode 100644 index 000000000000..03b8eb4b3351 --- /dev/null +++ b/libs/langchain-community/src/chat_models/tests/ibm.standard.int.test.ts @@ -0,0 +1,43 @@ +/* eslint-disable no-process-env */ +import { test, expect } from "@jest/globals"; +import { ChatModelIntegrationTests } from "@langchain/standard-tests"; +import { AIMessageChunk } from "@langchain/core/messages"; +import { + ChatWatsonx, + ChatWatsonxInput, + WatsonxCallOptionsChat, + WatsonxCallParams, +} from "../ibm.js"; +import { WatsonxAuth } from "../../types/ibm.js"; + +class ChatWatsonxStandardIntegrationTests extends ChatModelIntegrationTests< + WatsonxCallOptionsChat, + AIMessageChunk, + ChatWatsonxInput & + WatsonxAuth & + Partial> +> { + constructor() { + if (!process.env.WATSONX_AI_APIKEY) { + throw new Error("Cannot run tests. Api key not provided"); + } + super({ + Cls: ChatWatsonx, + chatModelHasToolCalling: true, + chatModelHasStructuredOutput: true, + constructorArgs: { + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + temperature: 0, + }, + }); + } +} + +const testClass = new ChatWatsonxStandardIntegrationTests(); + +test("ChatWatsonxStandardIntegrationTests", async () => { + const testResults = await testClass.runTests(); + expect(testResults).toBe(true); +}); diff --git a/libs/langchain-community/src/chat_models/tests/ibm.standard.test.ts b/libs/langchain-community/src/chat_models/tests/ibm.standard.test.ts new file mode 100644 index 000000000000..6c7ab7d5576a --- /dev/null +++ b/libs/langchain-community/src/chat_models/tests/ibm.standard.test.ts @@ -0,0 +1,55 @@ +/* eslint-disable no-process-env */ +import { test, expect } from "@jest/globals"; +import { ChatModelUnitTests } from "@langchain/standard-tests"; +import { AIMessageChunk } from "@langchain/core/messages"; +import { LangSmithParams } from "@langchain/core/language_models/chat_models"; +import { + ChatWatsonx, + ChatWatsonxInput, + WatsonxCallOptionsChat, + WatsonxCallParams, +} from "../ibm.js"; +import { WatsonxAuth } from "../../types/ibm.js"; + +class ChatWatsonxStandardTests extends ChatModelUnitTests< + WatsonxCallOptionsChat, + AIMessageChunk, + ChatWatsonxInput & + WatsonxAuth & + Partial> +> { + constructor() { + super({ + Cls: ChatWatsonx, + chatModelHasToolCalling: true, + chatModelHasStructuredOutput: true, + constructorArgs: { + watsonxAIApikey: "testString", + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL ?? "testString", + projectId: process.env.WATSONX_AI_PROJECT_ID ?? "testString", + watsonxAIAuthType: "iam", + }, + }); + } + + expectedLsParams(): Partial { + console.warn( + "ChatWatsonx does not support stop sequences. Overwrite params." + ); + return { + ls_provider: "watsonx", + ls_model_name: "string", + ls_model_type: "chat", + ls_temperature: 0, + ls_max_tokens: 0, + }; + } +} + +const testClass = new ChatWatsonxStandardTests(); + +test("ChatWatsonxStandardTests", () => { + const testResults = testClass.runTests(); + expect(testResults).toBe(true); +}); diff --git a/libs/langchain-community/src/chat_models/tests/ibm.test.ts b/libs/langchain-community/src/chat_models/tests/ibm.test.ts new file mode 100644 index 000000000000..3fea7de8504b --- /dev/null +++ b/libs/langchain-community/src/chat_models/tests/ibm.test.ts @@ -0,0 +1,185 @@ +/* eslint-disable no-process-env */ +import WatsonxAiMlVml_v1 from "@ibm-cloud/watsonx-ai/dist/watsonx-ai-ml/vml_v1.js"; +import { ChatWatsonx, ChatWatsonxInput, WatsonxCallParams } from "../ibm.js"; +import { authenticateAndSetInstance } from "../../utils/ibm.js"; + +const fakeAuthProp = { + watsonxAIAuthType: "iam", + watsonxAIApikey: "fake_key", +}; +export function getKey(key: K): K { + return key; +} +export const testProperties = ( + instance: ChatWatsonx, + testProps: ChatWatsonxInput, + notExTestProps?: { [key: string]: any } +) => { + const checkProperty = ( + testProps: T, + instance: T, + existing = true + ) => { + Object.keys(testProps).forEach((key) => { + const keys = getKey(key); + type Type = Pick; + + if (typeof testProps[key as keyof T] === "object") + checkProperty(testProps[key as keyof T], instance[key], existing); + else { + if (existing) + expect(instance[key as keyof T]).toBe(testProps[key as keyof T]); + else if (instance) expect(instance[key as keyof T]).toBeUndefined(); + } + }); + }; + checkProperty(testProps, instance); + if (notExTestProps) + checkProperty(notExTestProps, instance, false); +}; + +describe("LLM unit tests", () => { + describe("Positive tests", () => { + test("Test authentication function", () => { + const instance = authenticateAndSetInstance({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + ...fakeAuthProp, + }); + expect(instance).toBeInstanceOf(WatsonxAiMlVml_v1); + }); + + test("Test basic properties after init", async () => { + const testProps = { + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID || "testString", + }; + const instance = new ChatWatsonx({ ...testProps, ...fakeAuthProp }); + + testProperties(instance, testProps); + }); + + test("Test methods after init", () => { + const testProps: ChatWatsonxInput = { + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID || "testString", + }; + const instance = new ChatWatsonx({ + ...testProps, + ...fakeAuthProp, + }); + expect(instance.getNumTokens).toBeDefined(); + expect(instance._generate).toBeDefined(); + expect(instance._streamResponseChunks).toBeDefined(); + expect(instance.invocationParams).toBeDefined(); + }); + + test("Test properties after init", async () => { + const testProps: WatsonxCallParams & ChatWatsonxInput = { + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID || "testString", + model: "ibm/granite-13b-chat-v2", + max_new_tokens: 100, + temperature: 0.1, + time_limit: 10000, + top_p: 1, + maxRetries: 3, + maxConcurrency: 3, + }; + const instance = new ChatWatsonx({ ...testProps, ...fakeAuthProp }); + + testProperties(instance, testProps); + }); + }); + + describe("Negative tests", () => { + test("Missing id", async () => { + const testProps: ChatWatsonxInput = { + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + }; + expect( + () => + new ChatWatsonx({ + ...testProps, + ...fakeAuthProp, + }) + ).toThrowError(); + }); + + test("Missing other props", async () => { + // @ts-expect-error Intentionally passing not enough parameters + const testPropsProjectId: ChatWatsonxInput = { + projectId: process.env.WATSONX_AI_PROJECT_ID || "testString", + }; + + expect( + () => + new ChatWatsonx({ + ...testPropsProjectId, + ...fakeAuthProp, + }) + ).toThrowError(); + // @ts-expect-error Intentionally passing not enough parameters + const testPropsServiceUrl: ChatWatsonxInput = { + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + }; + expect( + () => + new ChatWatsonx({ + ...testPropsServiceUrl, + ...fakeAuthProp, + }) + ).toThrowError(); + const testPropsVersion = { + version: "2024-05-31", + }; + expect( + () => + new ChatWatsonx({ + // @ts-expect-error Intentionally passing wrong type of an object + testPropsVersion, + }) + ).toThrowError(); + }); + + test("Passing more than one id", async () => { + const testProps: ChatWatsonxInput = { + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID || "testString", + spaceId: process.env.WATSONX_AI_PROJECT_ID || "testString", + }; + expect( + () => + new ChatWatsonx({ + ...testProps, + ...fakeAuthProp, + }) + ).toThrowError(); + }); + + test("Not existing property passed", async () => { + const testProps = { + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID || "testString", + }; + const notExTestProps = { + notExisting: 12, + notExObj: { + notExProp: 12, + }, + }; + const instance = new ChatWatsonx({ + ...testProps, + ...notExTestProps, + ...fakeAuthProp, + }); + testProperties(instance, testProps, notExTestProps); + }); + }); +}); diff --git a/libs/langchain-community/src/embeddings/ibm.ts b/libs/langchain-community/src/embeddings/ibm.ts new file mode 100644 index 000000000000..fc8fcaebb561 --- /dev/null +++ b/libs/langchain-community/src/embeddings/ibm.ts @@ -0,0 +1,130 @@ +import { Embeddings } from "@langchain/core/embeddings"; +import { + EmbeddingParameters, + TextEmbeddingsParams, +} from "@ibm-cloud/watsonx-ai/dist/watsonx-ai-ml/vml_v1.js"; +import { WatsonXAI } from "@ibm-cloud/watsonx-ai"; +import { AsyncCaller } from "@langchain/core/utils/async_caller"; +import { WatsonxAuth, WatsonxParams } from "../types/ibm.js"; +import { authenticateAndSetInstance } from "../utils/ibm.js"; + +export interface WatsonxEmbeddingsParams + extends Omit, + Pick {} + +export class WatsonxEmbeddings + extends Embeddings + implements WatsonxEmbeddingsParams, WatsonxParams +{ + model = "ibm/slate-125m-english-rtrvr"; + + serviceUrl: string; + + version: string; + + spaceId?: string; + + projectId?: string; + + truncate_input_tokens?: number; + + maxRetries?: number; + + maxConcurrency?: number; + + private service: WatsonXAI; + + constructor(fields: WatsonxEmbeddingsParams & WatsonxAuth & WatsonxParams) { + const superProps = { maxConcurrency: 2, ...fields }; + super(superProps); + this.model = fields?.model ? fields.model : this.model; + this.version = fields.version; + this.serviceUrl = fields.serviceUrl; + this.truncate_input_tokens = fields.truncate_input_tokens; + this.maxConcurrency = fields.maxConcurrency; + this.maxRetries = fields.maxRetries; + if (fields.projectId && fields.spaceId) + throw new Error("Maximum 1 id type can be specified per instance"); + else if (!fields.projectId && !fields.spaceId && !fields.idOrName) + throw new Error( + "No id specified! At least id of 1 type has to be specified" + ); + this.projectId = fields?.projectId; + this.spaceId = fields?.spaceId; + this.serviceUrl = fields?.serviceUrl; + const { + watsonxAIApikey, + watsonxAIAuthType, + watsonxAIBearerToken, + watsonxAIUsername, + watsonxAIPassword, + watsonxAIUrl, + version, + serviceUrl, + } = fields; + const auth = authenticateAndSetInstance({ + watsonxAIApikey, + watsonxAIAuthType, + watsonxAIBearerToken, + watsonxAIUsername, + watsonxAIPassword, + watsonxAIUrl, + version, + serviceUrl, + }); + if (auth) this.service = auth; + else throw new Error("You have not provided one type of authentication"); + } + + scopeId() { + if (this.projectId) return { projectId: this.projectId }; + else return { spaceId: this.spaceId }; + } + + invocationParams(): EmbeddingParameters { + return { + truncate_input_tokens: this.truncate_input_tokens, + }; + } + + async listModels() { + const listModelParams = { + filters: "function_embedding", + }; + const caller = new AsyncCaller({ + maxConcurrency: this.maxConcurrency, + maxRetries: this.maxRetries, + }); + const listModels = await caller.call(() => + this.service.listFoundationModelSpecs(listModelParams) + ); + return listModels.result.resources?.map((item) => item.model_id); + } + + private async embedSingleText(inputs: string[]) { + const textEmbeddingParams: TextEmbeddingsParams = { + inputs, + modelId: this.model, + ...this.scopeId(), + parameters: this.invocationParams(), + }; + const caller = new AsyncCaller({ + maxConcurrency: this.maxConcurrency, + maxRetries: this.maxRetries, + }); + const embeddings = await caller.call(() => + this.service.embedText(textEmbeddingParams) + ); + return embeddings.result.results.map((item) => item.embedding); + } + + async embedDocuments(documents: string[]): Promise { + const data = await this.embedSingleText(documents); + return data; + } + + async embedQuery(document: string): Promise { + const data = await this.embedSingleText([document]); + return data[0]; + } +} diff --git a/libs/langchain-community/src/embeddings/tests/ibm.int.test.ts b/libs/langchain-community/src/embeddings/tests/ibm.int.test.ts new file mode 100644 index 000000000000..9361a7915213 --- /dev/null +++ b/libs/langchain-community/src/embeddings/tests/ibm.int.test.ts @@ -0,0 +1,62 @@ +/* eslint-disable no-process-env */ +import { test } from "@jest/globals"; +import { WatsonxEmbeddings } from "../ibm.js"; + +describe("Test embeddings", () => { + test("embedQuery method", async () => { + const embeddings = new WatsonxEmbeddings({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + }); + const res = await embeddings.embedQuery("Hello world"); + expect(typeof res[0]).toBe("number"); + }); + + test("embedDocuments", async () => { + const embeddings = new WatsonxEmbeddings({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + }); + const res = await embeddings.embedDocuments(["Hello world", "Bye world"]); + expect(res).toHaveLength(2); + expect(typeof res[0][0]).toBe("number"); + expect(typeof res[1][0]).toBe("number"); + }); + + test("Concurrency", async () => { + const embeddings = new WatsonxEmbeddings({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + maxConcurrency: 4, + }); + const res = await embeddings.embedDocuments([ + "Hello world", + "Bye world", + "Hello world", + "Bye world", + "Hello world", + "Bye world", + "Hello world", + "Bye world", + ]); + expect(res).toHaveLength(8); + expect(res.find((embedding) => typeof embedding[0] !== "number")).toBe( + undefined + ); + }); + + test("List models", async () => { + const embeddings = new WatsonxEmbeddings({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + maxConcurrency: 4, + }); + const res = await embeddings.listModels(); + expect(res?.length).toBeGreaterThan(0); + if (res) expect(typeof res[0]).toBe("string"); + }); +}); diff --git a/libs/langchain-community/src/embeddings/tests/ibm.test.ts b/libs/langchain-community/src/embeddings/tests/ibm.test.ts new file mode 100644 index 000000000000..affa8491807f --- /dev/null +++ b/libs/langchain-community/src/embeddings/tests/ibm.test.ts @@ -0,0 +1,122 @@ +/* eslint-disable no-process-env */ +import { testProperties } from "../../llms/tests/ibm.test.js"; +import { WatsonxEmbeddings } from "../ibm.js"; + +const fakeAuthProp = { + watsonxAIAuthType: "iam", + watsonxAIApikey: "fake_key", +}; +describe("Embeddings unit tests", () => { + describe("Positive tests", () => { + test("Basic properties", () => { + const testProps = { + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID || "testString", + }; + const instance = new WatsonxEmbeddings({ ...testProps, ...fakeAuthProp }); + testProperties(instance, testProps); + }); + + test("Basic properties", () => { + const testProps = { + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID || "testString", + truncate_input_tokens: 10, + maxConcurrency: 2, + maxRetries: 2, + model: "ibm/slate-125m-english-rtrvr", + }; + const instance = new WatsonxEmbeddings({ ...testProps, ...fakeAuthProp }); + + testProperties(instance, testProps); + }); + }); + + describe("Negative tests", () => { + test("Missing id", async () => { + const testProps = { + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + }; + expect( + () => + new WatsonxEmbeddings({ + ...testProps, + ...fakeAuthProp, + }) + ).toThrowError(); + }); + + test("Missing other props", async () => { + // @ts-expect-error Intentionally passing wrong value + const testPropsProjectId: WatsonxInputLLM = { + projectId: process.env.WATSONX_AI_PROJECT_ID || "testString", + }; + expect( + () => + new WatsonxEmbeddings({ + ...testPropsProjectId, + }) + ).toThrowError(); + // @ts-expect-error //Intentionally passing wrong value + const testPropsServiceUrl: WatsonxInputLLM = { + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + }; + expect( + () => + new WatsonxEmbeddings({ + ...testPropsServiceUrl, + }) + ).toThrowError(); + const testPropsVersion = { + version: "2024-05-31", + }; + expect( + () => + new WatsonxEmbeddings({ + // @ts-expect-error Intentionally passing wrong props + testPropsVersion, + }) + ).toThrowError(); + }); + + test("Passing more than one id", async () => { + const testProps = { + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID || "testString", + spaceId: process.env.WATSONX_AI_PROJECT_ID || "testString", + }; + expect( + () => + new WatsonxEmbeddings({ + ...testProps, + ...fakeAuthProp, + }) + ).toThrowError(); + }); + + test("Invalid properties", () => { + const testProps = { + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID || "testString", + }; + const notExTestProps = { + notExisting: 12, + notExObj: { + notExProp: 12, + }, + }; + const instance = new WatsonxEmbeddings({ + ...testProps, + ...notExTestProps, + ...fakeAuthProp, + }); + + testProperties(instance, testProps, notExTestProps); + }); + }); +}); diff --git a/libs/langchain-community/src/llms/ibm.ts b/libs/langchain-community/src/llms/ibm.ts new file mode 100644 index 000000000000..302275158d9c --- /dev/null +++ b/libs/langchain-community/src/llms/ibm.ts @@ -0,0 +1,566 @@ +/* eslint-disable @typescript-eslint/no-unused-vars */ +import { CallbackManagerForLLMRun } from "@langchain/core/callbacks/manager"; +import { BaseLLM, BaseLLMParams } from "@langchain/core/language_models/llms"; +import { WatsonXAI } from "@ibm-cloud/watsonx-ai"; +import { + DeploymentsTextGenerationParams, + DeploymentsTextGenerationStreamParams, + DeploymentTextGenProperties, + ReturnOptionProperties, + TextGenerationParams, + TextGenerationStreamParams, + TextGenLengthPenalty, + TextGenParameters, + TextTokenizationParams, + TextTokenizeParameters, +} from "@ibm-cloud/watsonx-ai/dist/watsonx-ai-ml/vml_v1.js"; +import { + Generation, + LLMResult, + GenerationChunk, +} from "@langchain/core/outputs"; +import { BaseLanguageModelCallOptions } from "@langchain/core/language_models/base"; +import { AsyncCaller } from "@langchain/core/utils/async_caller"; +import { authenticateAndSetInstance } from "../utils/ibm.js"; +import { + GenerationInfo, + ResponseChunk, + TokenUsage, + WatsonxAuth, + WatsonxParams, +} from "../types/ibm.js"; + +/** + * Input to LLM class. + */ + +export interface WatsonxCallOptionsLLM + extends BaseLanguageModelCallOptions, + Omit< + Partial< + TextGenerationParams & + TextGenerationStreamParams & + DeploymentsTextGenerationParams & + DeploymentsTextGenerationStreamParams + >, + "input" + > { + maxRetries?: number; +} + +export interface WatsonxInputLLM + extends TextGenParameters, + WatsonxParams, + BaseLLMParams { + streaming?: boolean; +} + +/** + * Integration with an LLM. + */ +export class WatsonxLLM< + CallOptions extends WatsonxCallOptionsLLM = WatsonxCallOptionsLLM + > + extends BaseLLM + implements WatsonxInputLLM +{ + // Used for tracing, replace with the same name as your class + static lc_name() { + return "Watsonx"; + } + + lc_serializable = true; + + streaming = false; + + model = "ibm/granite-13b-chat-v2"; + + maxRetries = 0; + + version = "2024-05-31"; + + serviceUrl: string; + + max_new_tokens?: number; + + spaceId?: string; + + projectId?: string; + + idOrName?: string; + + decoding_method?: TextGenParameters.Constants.DecodingMethod | string; + + length_penalty?: TextGenLengthPenalty; + + min_new_tokens?: number; + + random_seed?: number; + + stop_sequences?: string[]; + + temperature?: number; + + time_limit?: number; + + top_k?: number; + + top_p?: number; + + repetition_penalty?: number; + + truncate_input_tokens?: number; + + return_options?: ReturnOptionProperties; + + include_stop_sequence?: boolean; + + maxConcurrency?: number; + + private service: WatsonXAI; + + constructor(fields: WatsonxInputLLM & WatsonxAuth) { + super(fields); + this.model = fields.model ?? this.model; + this.version = fields.version; + this.max_new_tokens = fields.max_new_tokens ?? this.max_new_tokens; + this.serviceUrl = fields.serviceUrl; + this.decoding_method = fields.decoding_method; + this.length_penalty = fields.length_penalty; + this.min_new_tokens = fields.min_new_tokens; + this.random_seed = fields.random_seed; + this.stop_sequences = fields.stop_sequences; + this.temperature = fields.temperature; + this.time_limit = fields.time_limit; + this.top_k = fields.top_k; + this.top_p = fields.top_p; + this.repetition_penalty = fields.repetition_penalty; + this.truncate_input_tokens = fields.truncate_input_tokens; + this.return_options = fields.return_options; + this.include_stop_sequence = fields.include_stop_sequence; + this.maxRetries = fields.maxRetries || this.maxRetries; + this.maxConcurrency = fields.maxConcurrency; + this.streaming = fields.streaming || this.streaming; + if ( + (fields.projectId && fields.spaceId) || + (fields.idOrName && fields.projectId) || + (fields.spaceId && fields.idOrName) + ) + throw new Error("Maximum 1 id type can be specified per instance"); + + if (!fields.projectId && !fields.spaceId && !fields.idOrName) + throw new Error( + "No id specified! At least ide of 1 type has to be specified" + ); + this.projectId = fields?.projectId; + this.spaceId = fields?.spaceId; + this.idOrName = fields?.idOrName; + + this.serviceUrl = fields?.serviceUrl; + const { + watsonxAIApikey, + watsonxAIAuthType, + watsonxAIBearerToken, + watsonxAIUsername, + watsonxAIPassword, + watsonxAIUrl, + version, + serviceUrl, + } = fields; + + const auth = authenticateAndSetInstance({ + watsonxAIApikey, + watsonxAIAuthType, + watsonxAIBearerToken, + watsonxAIUsername, + watsonxAIPassword, + watsonxAIUrl, + version, + serviceUrl, + }); + if (auth) this.service = auth; + else throw new Error("You have not provided one type of authentication"); + } + + get lc_secrets(): { [key: string]: string } { + return { + authenticator: "AUTHENTICATOR", + apiKey: "WATSONX_AI_APIKEY", + apikey: "WATSONX_AI_APIKEY", + watsonxAIAuthType: "WATSONX_AI_AUTH_TYPE", + watsonxAIApikey: "WATSONX_AI_APIKEY", + watsonxAIBearerToken: "WATSONX_AI_BEARER_TOKEN", + watsonxAIUsername: "WATSONX_AI_USERNAME", + watsonxAIPassword: "WATSONX_AI_PASSWORD", + watsonxAIUrl: "WATSONX_AI_URL", + }; + } + + get lc_aliases(): { [key: string]: string } { + return { + authenticator: "authenticator", + apikey: "watsonx_ai_apikey", + apiKey: "watsonx_ai_apikey", + watsonxAIAuthType: "watsonx_ai_auth_type", + watsonxAIApikey: "watsonx_ai_apikey", + watsonxAIBearerToken: "watsonx_ai_bearer_token", + watsonxAIUsername: "watsonx_ai_username", + watsonxAIPassword: "watsonx_ai_password", + watsonxAIUrl: "watsonx_ai_url", + }; + } + + invocationParams( + options: this["ParsedCallOptions"] + ): TextGenParameters | DeploymentTextGenProperties { + const { parameters } = options; + + return { + max_new_tokens: parameters?.max_new_tokens ?? this.max_new_tokens, + decoding_method: parameters?.decoding_method ?? this.decoding_method, + length_penalty: parameters?.length_penalty ?? this.length_penalty, + min_new_tokens: parameters?.min_new_tokens ?? this.min_new_tokens, + random_seed: parameters?.random_seed ?? this.random_seed, + stop_sequences: options?.stop ?? this.stop_sequences, + temperature: parameters?.temperature ?? this.temperature, + time_limit: parameters?.time_limit ?? this.time_limit, + top_k: parameters?.top_k ?? this.top_k, + top_p: parameters?.top_p ?? this.top_p, + repetition_penalty: + parameters?.repetition_penalty ?? this.repetition_penalty, + truncate_input_tokens: + parameters?.truncate_input_tokens ?? this.truncate_input_tokens, + return_options: parameters?.return_options ?? this.return_options, + include_stop_sequence: + parameters?.include_stop_sequence ?? this.include_stop_sequence, + }; + } + + scopeId() { + if (this.projectId) + return { projectId: this.projectId, modelId: this.model }; + else if (this.spaceId) + return { spaceId: this.spaceId, modelId: this.model }; + else if (this.idOrName) + return { idOrName: this.idOrName, modelId: this.model }; + else return { spaceId: this.spaceId, modelId: this.model }; + } + + async listModels() { + const listModelParams = { + filters: "function_text_generation", + }; + const listModels = await this.completionWithRetry(() => + this.service.listFoundationModelSpecs(listModelParams) + ); + return listModels.result.resources?.map((item) => item.model_id); + } + + private async generateSingleMessage( + input: string, + options: this["ParsedCallOptions"], + stream: true + ): Promise>; + + private async generateSingleMessage( + input: string, + options: this["ParsedCallOptions"], + stream: false + ): Promise; + + private async generateSingleMessage( + input: string, + options: this["ParsedCallOptions"], + stream: boolean + ) { + const { + signal, + stop, + maxRetries, + maxConcurrency, + timeout, + ...requestOptions + } = options; + const tokenUsage = { generated_token_count: 0, input_token_count: 0 }; + const idOrName = options?.idOrName ?? this.idOrName; + const parameters = this.invocationParams(options); + if (stream) { + const textStream = idOrName + ? await this.service.deploymentGenerateTextStream({ + idOrName, + ...requestOptions, + parameters: { + ...parameters, + prompt_variables: { + input, + }, + }, + }) + : await this.service.generateTextStream({ + input, + parameters, + ...this.scopeId(), + ...requestOptions, + }); + return textStream as unknown as AsyncIterable; + } else { + const textGenerationPromise = idOrName + ? this.service.deploymentGenerateText({ + ...requestOptions, + idOrName, + parameters: { + ...parameters, + prompt_variables: { + input, + }, + }, + }) + : this.service.generateText({ + input, + parameters, + ...this.scopeId(), + ...requestOptions, + }); + + const textGeneration = await textGenerationPromise; + const singleGeneration: Generation[] = textGeneration.result.results.map( + (result) => { + tokenUsage.generated_token_count += result.generated_token_count + ? result.generated_token_count + : 0; + tokenUsage.input_token_count += result.input_token_count + ? result.input_token_count + : 0; + return { + text: result.generated_text, + generationInfo: { + stop_reason: result.stop_reason, + input_token_count: result.input_token_count, + generated_token_count: result.generated_token_count, + }, + }; + } + ); + return singleGeneration; + } + } + + async completionWithRetry( + callback: () => T, + options?: this["ParsedCallOptions"] + ) { + const caller = new AsyncCaller({ + maxConcurrency: options?.maxConcurrency || this.maxConcurrency, + maxRetries: this.maxRetries, + }); + const result = options + ? caller.callWithOptions( + { + signal: options.signal, + }, + async () => callback() + ) + : caller.call(async () => callback()); + + return result; + } + + async _generate( + prompts: string[], + options: this["ParsedCallOptions"], + _runManager?: CallbackManagerForLLMRun + ): Promise { + const tokenUsage: TokenUsage = { + generated_token_count: 0, + input_token_count: 0, + }; + if (this.streaming) { + const generations: Generation[][] = await Promise.all( + prompts.map(async (prompt, promptIdx) => { + if (options.signal?.aborted) { + throw new Error("AbortError"); + } + const callback = () => + this.generateSingleMessage(prompt, options, true); + + type ReturnMessage = ReturnType; + const stream = await this.completionWithRetry( + callback, + options + ); + + const responseChunk: ResponseChunk = { + id: 0, + event: "", + data: { + results: [], + }, + }; + const messages: ResponseChunk[] = []; + type ResponseChunkKeys = keyof ResponseChunk; + for await (const chunk of stream) { + if (chunk.length > 0) { + const index = chunk.indexOf(": "); + const [key, value] = [ + chunk.substring(0, index) as ResponseChunkKeys, + chunk.substring(index + 2), + ]; + if (key === "id") { + responseChunk[key] = Number(value); + } else if (key === "event") { + responseChunk[key] = String(value); + } else { + responseChunk[key] = JSON.parse(value); + } + } else if (chunk.length === 0) { + messages.push(JSON.parse(JSON.stringify(responseChunk))); + Object.assign(responseChunk, { id: 0, event: "", data: {} }); + } + } + + const geneartionsArray: GenerationInfo[] = []; + for (const message of messages) { + message.data.results.forEach((item, index) => { + const generationInfo: GenerationInfo = { + text: "", + stop_reason: "", + generated_token_count: 0, + input_token_count: 0, + }; + void _runManager?.handleLLMNewToken(item.generated_text ?? "", { + prompt: promptIdx, + completion: 1, + }); + geneartionsArray[index] ??= generationInfo; + geneartionsArray[index].generated_token_count = + item.generated_token_count; + geneartionsArray[index].input_token_count += + item.input_token_count; + geneartionsArray[index].stop_reason = item.stop_reason; + geneartionsArray[index].text += item.generated_text; + }); + } + return geneartionsArray.map((item) => { + const { text, ...rest } = item; + tokenUsage.generated_token_count += rest.generated_token_count; + tokenUsage.input_token_count += rest.input_token_count; + return { + text, + generationInfo: rest, + }; + }); + }) + ); + const result: LLMResult = { generations, llmOutput: { tokenUsage } }; + return result; + } else { + const generations: Generation[][] = await Promise.all( + prompts.map(async (prompt) => { + if (options.signal?.aborted) { + throw new Error("AbortError"); + } + + const callback = () => + this.generateSingleMessage(prompt, options, false); + type ReturnMessage = ReturnType; + + const response = await this.completionWithRetry( + callback, + options + ); + const [generated_token_count, input_token_count] = response.reduce( + (acc, curr) => { + let generated = 0; + let inputed = 0; + if (curr?.generationInfo?.generated_token_count) + generated = curr.generationInfo.generated_token_count + acc[0]; + if (curr?.generationInfo?.input_token_count) + inputed = curr.generationInfo.input_token_count + acc[1]; + return [generated, inputed]; + }, + [0, 0] + ); + tokenUsage.generated_token_count += generated_token_count; + tokenUsage.input_token_count += input_token_count; + return response; + }) + ); + + const result: LLMResult = { generations, llmOutput: { tokenUsage } }; + return result; + } + } + + async getNumTokens( + content: string, + options?: TextTokenizeParameters + ): Promise { + const params: TextTokenizationParams = { + ...this.scopeId(), + input: content, + parameters: options, + }; + const callback = () => this.service.tokenizeText(params); + type ReturnTokens = ReturnType; + + const response = await this.completionWithRetry(callback); + return response.result.result.token_count; + } + + async *_streamResponseChunks( + prompt: string, + options: this["ParsedCallOptions"], + runManager?: CallbackManagerForLLMRun + ): AsyncGenerator { + const callback = () => this.generateSingleMessage(prompt, options, true); + type ReturnStream = ReturnType; + const streamInferDeployedPrompt = + await this.completionWithRetry(callback); + const responseChunk: ResponseChunk = { + id: 0, + event: "", + data: { + results: [], + }, + }; + for await (const chunk of streamInferDeployedPrompt) { + if (options.signal?.aborted) { + throw new Error("AbortError"); + } + + type Keys = keyof typeof responseChunk; + if (chunk.length > 0) { + const index = chunk.indexOf(": "); + const [key, value] = [ + chunk.substring(0, index) as Keys, + chunk.substring(index + 2), + ]; + if (key === "id") { + responseChunk[key] = Number(value); + } else if (key === "event") { + responseChunk[key] = String(value); + } else { + responseChunk[key] = JSON.parse(value); + } + } else if ( + chunk.length === 0 && + responseChunk.data?.results?.length > 0 + ) { + for (const item of responseChunk.data.results) { + yield new GenerationChunk({ + text: item.generated_text, + generationInfo: { + stop_reason: item.stop_reason, + }, + }); + await runManager?.handleLLMNewToken(item.generated_text ?? ""); + } + Object.assign(responseChunk, { id: 0, event: "", data: {} }); + } + } + } + + _llmType() { + return "watsonx"; + } +} diff --git a/libs/langchain-community/src/llms/tests/ibm.int.test.ts b/libs/langchain-community/src/llms/tests/ibm.int.test.ts new file mode 100644 index 000000000000..236fd4950be8 --- /dev/null +++ b/libs/langchain-community/src/llms/tests/ibm.int.test.ts @@ -0,0 +1,391 @@ +/* eslint-disable no-process-env */ +import { CallbackManager } from "@langchain/core/callbacks/manager"; +import { LLMResult } from "@langchain/core/outputs"; +import { StringPromptValue } from "@langchain/core/prompt_values"; +import { TokenUsage } from "../../types/ibm.js"; +import { WatsonxLLM, WatsonxInputLLM } from "../ibm.js"; + +const originalBackground = process.env.LANGCHAIN_CALLBACKS_BACKGROUND; + +describe("Text generation", () => { + describe("Test invoke method", () => { + test("Correct value", async () => { + const watsonXInstance = new WatsonxLLM({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + }); + await watsonXInstance.invoke("Hello world?"); + }); + + test("Invalid projectId", async () => { + const watsonXInstance = new WatsonxLLM({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: "Test wrong value", + }); + await expect(watsonXInstance.invoke("Hello world?")).rejects.toThrow(); + }); + + test("Invalid credentials", async () => { + const watsonXInstance = new WatsonxLLM({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: "Test wrong value", + watsonxAIAuthType: "iam", + watsonxAIApikey: "WrongApiKey", + watsonxAIUrl: "https://wrong.wrong/", + }); + await expect(watsonXInstance.invoke("Hello world?")).rejects.toThrow(); + }); + + test("Wrong value", async () => { + const watsonXInstance = new WatsonxLLM({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + }); + // @ts-expect-error Intentionally passing wrong value + await watsonXInstance.invoke({}); + }); + + test("Stop", async () => { + const watsonXInstance = new WatsonxLLM({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + }); + await watsonXInstance.invoke("Hello, how are you?", { + stop: ["Hello"], + }); + }, 5000); + + test("Stop with timeout", async () => { + const watsonXInstance = new WatsonxLLM({ + version: "2024-05-31", + serviceUrl: "sdadasdas" as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + max_new_tokens: 5, + maxRetries: 3, + }); + + await expect(() => + watsonXInstance.invoke("Print hello world", { timeout: 10 }) + ).rejects.toThrowError("AbortError"); + }, 5000); + + test("Signal in call options", async () => { + const watsonXInstance = new WatsonxLLM({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + max_new_tokens: 5, + maxRetries: 3, + }); + const controllerNoAbortion = new AbortController(); + await expect( + watsonXInstance.invoke("Print hello world", { + signal: controllerNoAbortion.signal, + }) + ).resolves.toBeDefined(); + const controllerToAbort = new AbortController(); + await expect(async () => { + const ret = watsonXInstance.invoke("Print hello world", { + signal: controllerToAbort.signal, + }); + controllerToAbort.abort(); + return ret; + }).rejects.toThrowError("AbortError"); + }, 5000); + + test("Concurenccy", async () => { + const model = new WatsonxLLM({ + maxConcurrency: 1, + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + }); + await Promise.all([ + model.invoke("Print hello world"), + model.invoke("Print hello world"), + ]); + }); + + test("Token usage", async () => { + process.env.LANGCHAIN_CALLBACKS_BACKGROUND = "false"; + try { + const tokenUsage: TokenUsage = { + generated_token_count: 0, + input_token_count: 0, + }; + const model = new WatsonxLLM({ + maxConcurrency: 1, + version: "2024-05-31", + max_new_tokens: 1, + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + callbacks: CallbackManager.fromHandlers({ + async handleLLMEnd(output: LLMResult) { + const singleTokenUsage: TokenUsage | undefined = + output.llmOutput?.tokenUsage; + if (singleTokenUsage) { + tokenUsage.generated_token_count += + singleTokenUsage.generated_token_count; + tokenUsage.input_token_count += + singleTokenUsage.input_token_count; + } + }, + }), + }); + await model.invoke("Hello"); + expect(tokenUsage.generated_token_count).toBe(1); + expect(tokenUsage.input_token_count).toBe(1); + } finally { + process.env.LANGCHAIN_CALLBACKS_BACKGROUND = originalBackground; + } + }); + + test("Streaming mode", async () => { + let countedTokens = 0; + let streamedText = ""; + let usedTokens = 0; + const model = new WatsonxLLM({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + max_new_tokens: 5, + streaming: true, + + callbacks: CallbackManager.fromHandlers({ + async handleLLMEnd(output) { + usedTokens = output.llmOutput?.tokenUsage.generated_token_count; + }, + async handleLLMNewToken(token: string) { + countedTokens += 1; + streamedText += token; + }, + }), + }); + + const res = await model.invoke(" Print hello world?"); + expect(countedTokens).toBe(usedTokens); + expect(res).toBe(streamedText); + }); + }); + + describe("Test generate methods", () => { + test("Basic usage", async () => { + const model = new WatsonxLLM({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + max_new_tokens: 5, + }); + const res = await model.generate([ + "Print hello world!", + "Print hello universe!", + ]); + expect(res.generations.length).toBe(2); + }); + + test("Stop", async () => { + const model = new WatsonxLLM({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + max_new_tokens: 100, + }); + + const res = await model.generate( + ["Print hello world!", "Print hello world hello!"], + { + stop: ["Hello"], + } + ); + + expect( + res.generations + .map((generation) => generation.map((item) => item.text)) + .join("") + .indexOf("world") + ).toBe(-1); + }); + + test("Streaming mode with multiple prompts", async () => { + const nrNewTokens = [0, 0, 0]; + const completions = ["", "", ""]; + const model = new WatsonxLLM({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + max_new_tokens: 5, + streaming: true, + callbacks: CallbackManager.fromHandlers({ + async handleLLMNewToken(token: string, idx) { + nrNewTokens[idx.prompt] += 1; + completions[idx.prompt] += token; + }, + }), + }); + const res = await model.generate([ + "Print bye bye world!", + "Print bye bye world!", + "Print Hello IBM!", + ]); + res.generations.forEach((generation, index) => { + generation.forEach((g) => { + expect(g.generationInfo?.generated_token_count).toBe( + nrNewTokens[index] + ); + }); + }); + nrNewTokens.forEach((tokens) => expect(tokens > 0).toBe(true)); + expect(res.generations.length).toBe(3); + }); + + test("Prompt value", async () => { + const model = new WatsonxLLM({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + max_new_tokens: 5, + }); + const res = await model.generatePrompt([ + new StringPromptValue("Print hello world!"), + ]); + for (const generation of res.generations) { + expect(generation.length).toBe(1); + } + }); + }); + + describe("Test stream method", () => { + test("Basic usage", async () => { + let countedTokens = 0; + let streamedText = ""; + const model = new WatsonxLLM({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + max_new_tokens: 100, + callbacks: CallbackManager.fromHandlers({ + async handleLLMNewToken(token: string) { + countedTokens += 1; + streamedText += token; + }, + }), + }); + const stream = await model.stream("Print hello world."); + const chunks = []; + for await (const chunk of stream) { + chunks.push(chunk); + } + expect(chunks.length).toBeGreaterThan(1); + expect(chunks.join("")).toBe(streamedText); + }); + + test("Stop", async () => { + const model = new WatsonxLLM({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + max_new_tokens: 100, + }); + + const stream = await model.stream("Print hello world!", { + stop: ["Hello"], + }); + const chunks = []; + for await (const chunk of stream) { + chunks.push(chunk); + } + expect(chunks.join("").indexOf("world")).toBe(-1); + }); + + test("Timeout", async () => { + const model = new WatsonxLLM({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + max_new_tokens: 1000, + }); + await expect(async () => { + const stream = await model.stream( + "How is your day going? Be precise and tell me a lot about it/", + { + signal: AbortSignal.timeout(750), + } + ); + const chunks = []; + for await (const chunk of stream) { + chunks.push(chunk); + } + }).rejects.toThrowError(); + }); + + test("Signal in call options", async () => { + const model = new WatsonxLLM({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + max_new_tokens: 1000, + }); + const controller = new AbortController(); + await expect(async () => { + const stream = await model.stream( + "How is your day going? Be precise and tell me a lot about it", + { + signal: controller.signal, + } + ); + const chunks = []; + let i = 0; + for await (const chunk of stream) { + i += 1; + chunks.push(chunk); + if (i === 5) { + controller.abort(); + } + } + }).rejects.toThrowError(); + }); + }); + + describe("Test getNumToken method", () => { + test("Passing correct value", async () => { + const testProps: WatsonxInputLLM = { + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + }; + const instance = new WatsonxLLM({ + ...testProps, + }); + await expect( + instance.getNumTokens("Hello") + ).resolves.toBeGreaterThanOrEqual(0); + await expect( + instance.getNumTokens("Hello", { return_tokens: true }) + ).resolves.toBeGreaterThanOrEqual(0); + }); + + test("Passing wrong value", async () => { + const testProps: WatsonxInputLLM = { + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID, + maxRetries: 3, + }; + const instance = new WatsonxLLM({ + ...testProps, + }); + + // @ts-expect-error Intentionally passing wrong parameter + await expect(instance.getNumTokens(12)).rejects.toThrowError(); + await expect( + // @ts-expect-error Intentionally passing wrong parameter + instance.getNumTokens(12, { wrong: "Wrong" }) + ).rejects.toThrowError(); + }); + }); +}); diff --git a/libs/langchain-community/src/llms/tests/ibm.test.ts b/libs/langchain-community/src/llms/tests/ibm.test.ts new file mode 100644 index 000000000000..e0d6f3e4b521 --- /dev/null +++ b/libs/langchain-community/src/llms/tests/ibm.test.ts @@ -0,0 +1,207 @@ +/* eslint-disable no-process-env */ +import WatsonxAiMlVml_v1 from "@ibm-cloud/watsonx-ai/dist/watsonx-ai-ml/vml_v1.js"; +import { WatsonxLLM, WatsonxInputLLM } from "../ibm.js"; +import { authenticateAndSetInstance } from "../../utils/ibm.js"; +import { + WatsonxEmbeddings, + WatsonxEmbeddingsParams, +} from "../../embeddings/ibm.js"; + +const fakeAuthProp = { + watsonxAIAuthType: "iam", + watsonxAIApikey: "fake_key", +}; +export function getKey(key: K): K { + return key; +} +export const testProperties = ( + instance: WatsonxLLM | WatsonxEmbeddings, + testProps: WatsonxInputLLM, + notExTestProps?: { [key: string]: any } +) => { + const checkProperty = ( + testProps: T, + instance: T, + existing = true + ) => { + Object.keys(testProps).forEach((key) => { + const keys = getKey(key); + type Type = Pick; + + if (typeof testProps[key as keyof T] === "object") + checkProperty(testProps[key as keyof T], instance[key], existing); + else { + if (existing) + expect(instance[key as keyof T]).toBe(testProps[key as keyof T]); + else if (instance) expect(instance[key as keyof T]).toBeUndefined(); + } + }); + }; + checkProperty(testProps, instance); + if (notExTestProps) + checkProperty(notExTestProps, instance, false); +}; + +describe("LLM unit tests", () => { + describe("Positive tests", () => { + test("Test authentication function", () => { + const instance = authenticateAndSetInstance({ + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + ...fakeAuthProp, + }); + expect(instance).toBeInstanceOf(WatsonxAiMlVml_v1); + }); + + test("Test basic properties after init", async () => { + const testProps = { + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID || "testString", + }; + const instance = new WatsonxLLM({ ...testProps, ...fakeAuthProp }); + + testProperties(instance, testProps); + }); + + test("Test methods after init", () => { + const testProps: WatsonxInputLLM = { + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID || "testString", + }; + const instance = new WatsonxLLM({ + ...testProps, + ...fakeAuthProp, + }); + expect(instance.getNumTokens).toBeDefined(); + expect(instance._generate).toBeDefined(); + expect(instance._streamResponseChunks).toBeDefined(); + expect(instance.invocationParams).toBeDefined(); + }); + + test("Test properties after init", async () => { + const testProps = { + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID || "testString", + model: "ibm/granite-13b-chat-v2", + max_new_tokens: 100, + decoding_method: "sample", + length_penalty: { decay_factor: 1, start_index: 1 }, + min_new_tokens: 10, + random_seed: 1, + stop_sequences: ["hello"], + temperature: 0.1, + time_limit: 10000, + top_k: 1, + top_p: 1, + repetition_penalty: 1, + truncate_input_tokens: 1, + return_options: { + input_text: true, + generated_tokens: true, + input_tokens: true, + token_logprobs: true, + token_ranks: true, + + top_n_tokens: 2, + }, + include_stop_sequence: false, + maxRetries: 3, + maxConcurrency: 3, + }; + const instance = new WatsonxLLM({ ...testProps, ...fakeAuthProp }); + + testProperties(instance, testProps); + }); + }); + + describe("Negative tests", () => { + test("Missing id", async () => { + const testProps: WatsonxInputLLM = { + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + }; + expect( + () => + new WatsonxLLM({ + ...testProps, + ...fakeAuthProp, + }) + ).toThrowError(); + }); + + test("Missing other props", async () => { + // @ts-expect-error Intentionally passing not enough parameters + const testPropsProjectId: WatsonxInputLLM = { + projectId: process.env.WATSONX_AI_PROJECT_ID || "testString", + }; + + expect( + () => + new WatsonxLLM({ + ...testPropsProjectId, + ...fakeAuthProp, + }) + ).toThrowError(); + // @ts-expect-error Intentionally passing not enough parameters + const testPropsServiceUrl: WatsonxInputLLM = { + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + }; + expect( + () => + new WatsonxLLM({ + ...testPropsServiceUrl, + ...fakeAuthProp, + }) + ).toThrowError(); + const testPropsVersion = { + version: "2024-05-31", + }; + expect( + () => + new WatsonxLLM({ + // @ts-expect-error Intentionally passing wrong type of an object + testPropsVersion, + }) + ).toThrowError(); + }); + + test("Passing more than one id", async () => { + const testProps: WatsonxInputLLM = { + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID || "testString", + spaceId: process.env.WATSONX_AI_PROJECT_ID || "testString", + }; + expect( + () => + new WatsonxLLM({ + ...testProps, + ...fakeAuthProp, + }) + ).toThrowError(); + }); + + test("Not existing property passed", async () => { + const testProps = { + version: "2024-05-31", + serviceUrl: process.env.WATSONX_AI_SERVICE_URL as string, + projectId: process.env.WATSONX_AI_PROJECT_ID || "testString", + }; + const notExTestProps = { + notExisting: 12, + notExObj: { + notExProp: 12, + }, + }; + const instance = new WatsonxLLM({ + ...testProps, + ...notExTestProps, + ...fakeAuthProp, + }); + testProperties(instance, testProps, notExTestProps); + }); + }); +}); diff --git a/libs/langchain-community/src/llms/watsonx_ai.ts b/libs/langchain-community/src/llms/watsonx_ai.ts index 2da98bb86bd7..c92000d7b901 100644 --- a/libs/langchain-community/src/llms/watsonx_ai.ts +++ b/libs/langchain-community/src/llms/watsonx_ai.ts @@ -9,6 +9,8 @@ import { getEnvironmentVariable } from "@langchain/core/utils/env"; * The WatsonxAIParams interface defines the input parameters for * the WatsonxAI class. */ + +/** @deprecated Please use newer implementation @langchain/community/llms/ibm instead */ export interface WatsonxAIParams extends BaseLLMParams { /** * WatsonX AI Complete Endpoint. diff --git a/libs/langchain-community/src/load/import_constants.ts b/libs/langchain-community/src/load/import_constants.ts index 963f64864a04..65d40fc1f7d4 100644 --- a/libs/langchain-community/src/load/import_constants.ts +++ b/libs/langchain-community/src/load/import_constants.ts @@ -14,6 +14,7 @@ export const optionalImportEntrypoints: string[] = [ "langchain_community/embeddings/gradient_ai", "langchain_community/embeddings/hf", "langchain_community/embeddings/hf_transformers", + "langchain_community/embeddings/ibm", "langchain_community/embeddings/jina", "langchain_community/embeddings/llama_cpp", "langchain_community/embeddings/premai", @@ -27,6 +28,7 @@ export const optionalImportEntrypoints: string[] = [ "langchain_community/llms/cohere", "langchain_community/llms/gradient_ai", "langchain_community/llms/hf", + "langchain_community/llms/ibm", "langchain_community/llms/llama_cpp", "langchain_community/llms/portkey", "langchain_community/llms/raycast", @@ -82,6 +84,7 @@ export const optionalImportEntrypoints: string[] = [ "langchain_community/chat_models/arcjet", "langchain_community/chat_models/bedrock", "langchain_community/chat_models/bedrock/web", + "langchain_community/chat_models/ibm", "langchain_community/chat_models/iflytek_xinghuo", "langchain_community/chat_models/iflytek_xinghuo/web", "langchain_community/chat_models/llama_cpp", diff --git a/libs/langchain-community/src/load/import_type.ts b/libs/langchain-community/src/load/import_type.ts index 00f057aaa6a3..51536706142c 100644 --- a/libs/langchain-community/src/load/import_type.ts +++ b/libs/langchain-community/src/load/import_type.ts @@ -4,6 +4,7 @@ export interface OptionalImportMap {} export interface SecretMap { ALIBABA_API_KEY?: string; + AUTHENTICATOR?: string; AWS_ACCESS_KEY_ID?: string; AWS_SECRETE_ACCESS_KEY?: string; AWS_SECRET_ACCESS_KEY?: string; @@ -60,6 +61,12 @@ export interface SecretMap { VECTARA_API_KEY?: string; VECTARA_CORPUS_ID?: string; VECTARA_CUSTOMER_ID?: string; + WATSONX_AI_APIKEY?: string; + WATSONX_AI_AUTH_TYPE?: string; + WATSONX_AI_BEARER_TOKEN?: string; + WATSONX_AI_PASSWORD?: string; + WATSONX_AI_URL?: string; + WATSONX_AI_USERNAME?: string; WATSONX_PROJECT_ID?: string; WRITER_API_KEY?: string; WRITER_ORG_ID?: string; diff --git a/libs/langchain-community/src/types/ibm.ts b/libs/langchain-community/src/types/ibm.ts new file mode 100644 index 000000000000..cd11592ee48b --- /dev/null +++ b/libs/langchain-community/src/types/ibm.ts @@ -0,0 +1,45 @@ +export interface TokenUsage { + generated_token_count: number; + input_token_count: number; +} +export interface WatsonxAuth { + watsonxAIApikey?: string; + watsonxAIBearerToken?: string; + watsonxAIUsername?: string; + watsonxAIPassword?: string; + watsonxAIUrl?: string; + watsonxAIAuthType?: string; +} + +export interface WatsonxInit { + authenticator?: string; + serviceUrl: string; + version: string; +} + +export interface WatsonxParams extends WatsonxInit { + model?: string; + spaceId?: string; + projectId?: string; + idOrName?: string; + maxConcurrency?: number; + maxRetries?: number; +} + +export interface GenerationInfo { + text: string; + stop_reason: string | undefined; + generated_token_count: number; + input_token_count: number; +} + +export interface ResponseChunk { + id: number; + event: string; + data: { + results: (TokenUsage & { + stop_reason?: string; + generated_text: string; + })[]; + }; +} diff --git a/libs/langchain-community/src/utils/ibm.ts b/libs/langchain-community/src/utils/ibm.ts new file mode 100644 index 000000000000..acbb86f1a304 --- /dev/null +++ b/libs/langchain-community/src/utils/ibm.ts @@ -0,0 +1,193 @@ +import { WatsonXAI } from "@ibm-cloud/watsonx-ai"; +import { + IamAuthenticator, + BearerTokenAuthenticator, + CloudPakForDataAuthenticator, +} from "ibm-cloud-sdk-core"; +import { + JsonOutputKeyToolsParserParams, + JsonOutputToolsParser, +} from "@langchain/core/output_parsers/openai_tools"; +import { OutputParserException } from "@langchain/core/output_parsers"; +import { z } from "zod"; +import { ChatGeneration } from "@langchain/core/outputs"; +import { AIMessageChunk } from "@langchain/core/messages"; +import { ToolCall } from "@langchain/core/messages/tool"; +import { WatsonxAuth, WatsonxInit } from "../types/ibm.js"; + +export const authenticateAndSetInstance = ({ + watsonxAIApikey, + watsonxAIAuthType, + watsonxAIBearerToken, + watsonxAIUsername, + watsonxAIPassword, + watsonxAIUrl, + version, + serviceUrl, +}: WatsonxAuth & Omit): WatsonXAI | undefined => { + if (watsonxAIAuthType === "iam" && watsonxAIApikey) { + return WatsonXAI.newInstance({ + version, + serviceUrl, + authenticator: new IamAuthenticator({ + apikey: watsonxAIApikey, + }), + }); + } else if (watsonxAIAuthType === "bearertoken" && watsonxAIBearerToken) { + return WatsonXAI.newInstance({ + version, + serviceUrl, + authenticator: new BearerTokenAuthenticator({ + bearerToken: watsonxAIBearerToken, + }), + }); + } else if (watsonxAIAuthType === "cp4d" && watsonxAIUrl) { + if (watsonxAIUsername && watsonxAIPassword && watsonxAIApikey) + return WatsonXAI.newInstance({ + version, + serviceUrl, + authenticator: new CloudPakForDataAuthenticator({ + username: watsonxAIUsername, + password: watsonxAIPassword, + url: watsonxAIUrl, + apikey: watsonxAIApikey, + }), + }); + } else + return WatsonXAI.newInstance({ + version, + serviceUrl, + }); + return undefined; +}; + +// Mistral enforces a specific pattern for tool call IDs +// Thanks to Mistral for implementing this, I was unable to import which is why this is copied 1:1 +const TOOL_CALL_ID_PATTERN = /^[a-zA-Z0-9]{9}$/; + +export function _isValidMistralToolCallId(toolCallId: string): boolean { + return TOOL_CALL_ID_PATTERN.test(toolCallId); +} + +function _base62Encode(num: number): string { + let numCopy = num; + const base62 = + "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"; + if (numCopy === 0) return base62[0]; + const arr: string[] = []; + const base = base62.length; + while (numCopy) { + arr.push(base62[numCopy % base]); + numCopy = Math.floor(numCopy / base); + } + return arr.reverse().join(""); +} + +function _simpleHash(str: string): number { + let hash = 0; + for (let i = 0; i < str.length; i += 1) { + const char = str.charCodeAt(i); + hash = (hash << 5) - hash + char; + hash &= hash; // Convert to 32-bit integer + } + return Math.abs(hash); +} + +export function _convertToolCallIdToMistralCompatible( + toolCallId: string +): string { + if (_isValidMistralToolCallId(toolCallId)) { + return toolCallId; + } else { + const hash = _simpleHash(toolCallId); + const base62Str = _base62Encode(hash); + if (base62Str.length >= 9) { + return base62Str.slice(0, 9); + } else { + return base62Str.padStart(9, "0"); + } + } +} + +interface WatsonxToolsOutputParserParams> + extends JsonOutputKeyToolsParserParams {} + +export class WatsonxToolsOutputParser< + T extends Record = Record +> extends JsonOutputToolsParser { + static lc_name() { + return "WatsonxToolsOutputParser"; + } + + lc_namespace = ["langchain", "watsonx", "output_parsers"]; + + returnId = false; + + keyName: string; + + returnSingle = false; + + zodSchema?: z.ZodType; + + latestCorrect?: ToolCall; + + constructor(params: WatsonxToolsOutputParserParams) { + super(params); + this.keyName = params.keyName; + this.returnSingle = params.returnSingle ?? this.returnSingle; + this.zodSchema = params.zodSchema; + } + + protected async _validateResult(result: unknown): Promise { + let parsedResult = result; + if (typeof result === "string") { + try { + parsedResult = JSON.parse(result); + } catch (e: any) { + throw new OutputParserException( + `Failed to parse. Text: "${JSON.stringify( + result, + null, + 2 + )}". Error: ${JSON.stringify(e.message)}`, + result + ); + } + } else { + parsedResult = result; + } + if (this.zodSchema === undefined) { + return parsedResult as T; + } + const zodParsedResult = await this.zodSchema.safeParseAsync(parsedResult); + if (zodParsedResult.success) { + return zodParsedResult.data; + } else { + throw new OutputParserException( + `Failed to parse. Text: "${JSON.stringify( + result, + null, + 2 + )}". Error: ${JSON.stringify(zodParsedResult.error.errors)}`, + JSON.stringify(result, null, 2) + ); + } + } + + async parsePartialResult(generations: ChatGeneration[]): Promise { + const tools = generations.flatMap((generation) => { + const message = generation.message as AIMessageChunk; + if (!Array.isArray(message.tool_calls)) { + return []; + } + const tool = message.tool_calls; + return tool; + }); + if (tools[0] === undefined) { + if (this.latestCorrect) tools.push(this.latestCorrect); + } + const [tool] = tools; + this.latestCorrect = tool; + return tool.args as T; + } +} diff --git a/yarn.lock b/yarn.lock index 8c9766673e28..9be00a097637 100644 --- a/yarn.lock +++ b/yarn.lock @@ -10495,6 +10495,17 @@ __metadata: languageName: node linkType: hard +"@ibm-cloud/watsonx-ai@npm:^1.1.0": + version: 1.1.0 + resolution: "@ibm-cloud/watsonx-ai@npm:1.1.0" + dependencies: + "@types/node": ^12.0.8 + extend: 3.0.2 + ibm-cloud-sdk-core: ^4.2.5 + checksum: 0151bb0abe2a7d1dbcd6f8367ea02dfc924f15bdcbe8ec58bb89c8e055fa35c399b2253d6be3b84292f96c9161e49bcd6d6f5e1df0f2cd9adf21d1f3c0bc24b4 + languageName: node + linkType: hard + "@inquirer/figures@npm:^1.0.3": version: 1.0.5 resolution: "@inquirer/figures@npm:1.0.5" @@ -11463,6 +11474,7 @@ __metadata: "@google-cloud/storage": ^7.7.0 "@gradientai/nodejs-sdk": ^1.2.0 "@huggingface/inference": ^2.6.4 + "@ibm-cloud/watsonx-ai": ^1.1.0 "@jest/globals": ^29.5.0 "@langchain/core": "workspace:*" "@langchain/openai": ">=0.2.0 <0.4.0" @@ -11560,6 +11572,7 @@ __metadata: hdb: 0.19.8 hnswlib-node: ^3.0.0 html-to-text: ^9.0.5 + ibm-cloud-sdk-core: ^5.0.2 ignore: ^5.2.0 interface-datastore: ^8.2.11 ioredis: ^5.3.2 @@ -11638,6 +11651,7 @@ __metadata: "@google-cloud/storage": ^6.10.1 || ^7.7.0 "@gradientai/nodejs-sdk": ^1.2.0 "@huggingface/inference": ^2.6.4 + "@ibm-cloud/watsonx-ai": "*" "@langchain/core": ">=0.2.21 <0.4.0" "@layerup/layerup-security": ^1.5.12 "@libsql/client": ^0.14.0 @@ -11695,6 +11709,7 @@ __metadata: googleapis: "*" hnswlib-node: ^3.0.0 html-to-text: ^9.0.5 + ibm-cloud-sdk-core: "*" ignore: ^5.2.0 interface-datastore: ^8.2.11 ioredis: ^5.3.2 @@ -18685,6 +18700,15 @@ __metadata: languageName: node linkType: hard +"@types/debug@npm:^4.1.12": + version: 4.1.12 + resolution: "@types/debug@npm:4.1.12" + dependencies: + "@types/ms": "*" + checksum: 47876a852de8240bfdaf7481357af2b88cb660d30c72e73789abf00c499d6bc7cd5e52f41c915d1b9cd8ec9fef5b05688d7b7aef17f7f272c2d04679508d1053 + languageName: node + linkType: hard + "@types/decamelize@npm:^1.2.0": version: 1.2.0 resolution: "@types/decamelize@npm:1.2.0" @@ -19097,6 +19121,13 @@ __metadata: languageName: node linkType: hard +"@types/ms@npm:*": + version: 0.7.34 + resolution: "@types/ms@npm:0.7.34" + checksum: f38d36e7b6edecd9badc9cf50474159e9da5fa6965a75186cceaf883278611b9df6669dc3a3cc122b7938d317b68a9e3d573d316fcb35d1be47ec9e468c6bd8a + languageName: node + linkType: hard + "@types/mustache@npm:^4": version: 4.2.5 resolution: "@types/mustache@npm:4.2.5" @@ -19164,6 +19195,13 @@ __metadata: languageName: node linkType: hard +"@types/node@npm:^12.0.8": + version: 12.20.55 + resolution: "@types/node@npm:12.20.55" + checksum: e4f86785f4092706e0d3b0edff8dca5a13b45627e4b36700acd8dfe6ad53db71928c8dee914d4276c7fd3b6ccd829aa919811c9eb708a2c8e4c6eb3701178c37 + languageName: node + linkType: hard + "@types/node@npm:^17.0.5": version: 17.0.45 resolution: "@types/node@npm:17.0.45" @@ -19219,6 +19257,13 @@ __metadata: languageName: node linkType: hard +"@types/node@npm:~10.14.19": + version: 10.14.22 + resolution: "@types/node@npm:10.14.22" + checksum: 5dc12f9f284afe195584bfa553b3bd46828f0f568e1a349dcd7e6357d81fa82f8b3cd454f375b7478ad8e9b93a6e3341f102960ead4619829e5e82ea2bd8d204 + languageName: node + linkType: hard + "@types/offscreencanvas@npm:~2019.3.0": version: 2019.3.0 resolution: "@types/offscreencanvas@npm:2019.3.0" @@ -21432,6 +21477,17 @@ __metadata: languageName: node linkType: hard +"axios@npm:1.7.4": + version: 1.7.4 + resolution: "axios@npm:1.7.4" + dependencies: + follow-redirects: ^1.15.6 + form-data: ^4.0.0 + proxy-from-env: ^1.1.0 + checksum: 0c17039a9acfe6a566fca8431ba5c1b455c83d30ea6157fec68a6722878fcd30f3bd32d172f6bee0c51fe75ca98e6414ddcd968a87b5606b573731629440bfaf + languageName: node + linkType: hard + "axios@npm:^0.25.0": version: 0.25.0 resolution: "axios@npm:0.25.0" @@ -21494,6 +21550,17 @@ __metadata: languageName: node linkType: hard +"axios@npm:^1.7.5": + version: 1.7.7 + resolution: "axios@npm:1.7.7" + dependencies: + follow-redirects: ^1.15.6 + form-data: ^4.0.0 + proxy-from-env: ^1.1.0 + checksum: 882d4fe0ec694a07c7f5c1f68205eb6dc5a62aecdb632cc7a4a3d0985188ce3030e0b277e1a8260ac3f194d314ae342117660a151fabffdc5081ca0b5a8b47fe + languageName: node + linkType: hard + "axobject-query@npm:^3.1.1, axobject-query@npm:^3.2.1": version: 3.2.1 resolution: "axobject-query@npm:3.2.1" @@ -22417,7 +22484,7 @@ __metadata: languageName: node linkType: hard -"camelcase@npm:6, camelcase@npm:^6.2.0": +"camelcase@npm:6, camelcase@npm:^6.2.0, camelcase@npm:^6.3.0": version: 6.3.0 resolution: "camelcase@npm:6.3.0" checksum: 8c96818a9076434998511251dcb2761a94817ea17dbdc37f47ac080bd088fc62c7369429a19e2178b993497132c8cbcf5cc1f44ba963e76782ba469c0474938d @@ -25162,6 +25229,13 @@ __metadata: languageName: node linkType: hard +"diff-sequences@npm:^27.5.1": + version: 27.5.1 + resolution: "diff-sequences@npm:27.5.1" + checksum: a00db5554c9da7da225db2d2638d85f8e41124eccbd56cbaefb3b276dcbb1c1c2ad851c32defe2055a54a4806f030656cbf6638105fd6ce97bb87b90b32a33ca + languageName: node + linkType: hard + "diff-sequences@npm:^29.4.3": version: 29.4.3 resolution: "diff-sequences@npm:29.4.3" @@ -27385,6 +27459,18 @@ __metadata: languageName: node linkType: hard +"expect@npm:^27.5.1": + version: 27.5.1 + resolution: "expect@npm:27.5.1" + dependencies: + "@jest/types": ^27.5.1 + jest-get-type: ^27.5.1 + jest-matcher-utils: ^27.5.1 + jest-message-util: ^27.5.1 + checksum: b2c66beb52de53ef1872165aace40224e722bca3c2274c54cfa74b6d617d55cf0ccdbf36783ccd64dbea501b280098ed33fd0b207d4f15bc03cd3c7a24364a6a + languageName: node + linkType: hard + "expect@npm:^29.0.0": version: 29.6.1 resolution: "expect@npm:29.6.1" @@ -27535,7 +27621,7 @@ __metadata: languageName: node linkType: hard -"extend@npm:^3.0.0, extend@npm:^3.0.2": +"extend@npm:3.0.2, extend@npm:^3.0.0, extend@npm:^3.0.2": version: 3.0.2 resolution: "extend@npm:3.0.2" checksum: a50a8309ca65ea5d426382ff09f33586527882cf532931cb08ca786ea3146c0553310bda688710ff61d7668eba9f96b923fe1420cdf56a2c3eaf30fcab87b515 @@ -27878,7 +27964,7 @@ __metadata: languageName: node linkType: hard -"file-type@npm:^16.5.4": +"file-type@npm:16.5.4, file-type@npm:^16.5.4": version: 16.5.4 resolution: "file-type@npm:16.5.4" dependencies: @@ -30043,6 +30129,53 @@ __metadata: languageName: node linkType: hard +"ibm-cloud-sdk-core@npm:^4.2.5": + version: 4.3.4 + resolution: "ibm-cloud-sdk-core@npm:4.3.4" + dependencies: + "@types/debug": ^4.1.12 + "@types/node": ~10.14.19 + "@types/tough-cookie": ^4.0.0 + axios: ^1.7.5 + camelcase: ^6.3.0 + debug: ^4.3.4 + dotenv: ^16.4.5 + expect: ^27.5.1 + extend: 3.0.2 + file-type: 16.5.4 + form-data: 4.0.0 + isstream: 0.1.2 + jsonwebtoken: ^9.0.2 + mime-types: 2.1.35 + retry-axios: ^2.6.0 + tough-cookie: ^4.1.3 + checksum: 27d6bd692cde66766a7cea36e75d53a6a089e2b2b726cf86108ab48f9d452bb6d6a01324d2160e3bb54df7750240129bae989934ab2fd80c0950ecdb5bfc07b3 + languageName: node + linkType: hard + +"ibm-cloud-sdk-core@npm:^5.0.2": + version: 5.0.2 + resolution: "ibm-cloud-sdk-core@npm:5.0.2" + dependencies: + "@types/debug": ^4.1.12 + "@types/node": ~10.14.19 + "@types/tough-cookie": ^4.0.0 + axios: 1.7.4 + camelcase: ^6.3.0 + debug: ^4.3.4 + dotenv: ^16.4.5 + extend: 3.0.2 + file-type: 16.5.4 + form-data: 4.0.0 + isstream: 0.1.2 + jsonwebtoken: ^9.0.2 + mime-types: 2.1.35 + retry-axios: ^2.6.0 + tough-cookie: ^4.1.3 + checksum: fed92b851f816cbe94f4f28c6b45eed3b214f570897ed9936e5b7fd332b8c25c599f49d96866b9d936499b39a65e4d9db5a3191940b5a2489656e966e8fa6526 + languageName: node + linkType: hard + "iconv-lite@npm:0.4, iconv-lite@npm:0.4.24, iconv-lite@npm:^0.4.18, iconv-lite@npm:^0.4.24": version: 0.4.24 resolution: "iconv-lite@npm:0.4.24" @@ -31199,6 +31332,13 @@ __metadata: languageName: node linkType: hard +"isstream@npm:0.1.2": + version: 0.1.2 + resolution: "isstream@npm:0.1.2" + checksum: 1eb2fe63a729f7bdd8a559ab552c69055f4f48eb5c2f03724430587c6f450783c8f1cd936c1c952d0a927925180fcc892ebd5b174236cf1065d4bd5bdb37e963 + languageName: node + linkType: hard + "issue-parser@npm:6.0.0": version: 6.0.0 resolution: "issue-parser@npm:6.0.0" @@ -31534,6 +31674,18 @@ __metadata: languageName: node linkType: hard +"jest-diff@npm:^27.5.1": + version: 27.5.1 + resolution: "jest-diff@npm:27.5.1" + dependencies: + chalk: ^4.0.0 + diff-sequences: ^27.5.1 + jest-get-type: ^27.5.1 + pretty-format: ^27.5.1 + checksum: 8be27c1e1ee57b2bb2bef9c0b233c19621b4c43d53a3c26e2c00a4e805eb4ea11fe1694a06a9fb0e80ffdcfdc0d2b1cb0b85920b3f5c892327ecd1e7bd96b865 + languageName: node + linkType: hard + "jest-diff@npm:^29.5.0": version: 29.5.0 resolution: "jest-diff@npm:29.5.0" @@ -31620,6 +31772,13 @@ __metadata: languageName: node linkType: hard +"jest-get-type@npm:^27.5.1": + version: 27.5.1 + resolution: "jest-get-type@npm:27.5.1" + checksum: 63064ab70195c21007d897c1157bf88ff94a790824a10f8c890392e7d17eda9c3900513cb291ca1c8d5722cad79169764e9a1279f7c8a9c4cd6e9109ff04bbc0 + languageName: node + linkType: hard + "jest-get-type@npm:^29.4.3": version: 29.4.3 resolution: "jest-get-type@npm:29.4.3" @@ -31690,6 +31849,18 @@ __metadata: languageName: node linkType: hard +"jest-matcher-utils@npm:^27.5.1": + version: 27.5.1 + resolution: "jest-matcher-utils@npm:27.5.1" + dependencies: + chalk: ^4.0.0 + jest-diff: ^27.5.1 + jest-get-type: ^27.5.1 + pretty-format: ^27.5.1 + checksum: bb2135fc48889ff3fe73888f6cc7168ddab9de28b51b3148f820c89fdfd2effdcad005f18be67d0b9be80eda208ad47290f62f03d0a33f848db2dd0273c8217a + languageName: node + linkType: hard + "jest-matcher-utils@npm:^29.5.0": version: 29.5.0 resolution: "jest-matcher-utils@npm:29.5.0" @@ -31726,6 +31897,23 @@ __metadata: languageName: node linkType: hard +"jest-message-util@npm:^27.5.1": + version: 27.5.1 + resolution: "jest-message-util@npm:27.5.1" + dependencies: + "@babel/code-frame": ^7.12.13 + "@jest/types": ^27.5.1 + "@types/stack-utils": ^2.0.0 + chalk: ^4.0.0 + graceful-fs: ^4.2.9 + micromatch: ^4.0.4 + pretty-format: ^27.5.1 + slash: ^3.0.0 + stack-utils: ^2.0.3 + checksum: eb6d637d1411c71646de578c49826b6da8e33dd293e501967011de9d1916d53d845afbfb52a5b661ff1c495be7c13f751c48c7f30781fd94fbd64842e8195796 + languageName: node + linkType: hard + "jest-message-util@npm:^29.5.0": version: 29.5.0 resolution: "jest-message-util@npm:29.5.0" @@ -37201,6 +37389,17 @@ __metadata: languageName: node linkType: hard +"pretty-format@npm:^27.5.1": + version: 27.5.1 + resolution: "pretty-format@npm:27.5.1" + dependencies: + ansi-regex: ^5.0.1 + ansi-styles: ^5.0.0 + react-is: ^17.0.1 + checksum: cf610cffcb793885d16f184a62162f2dd0df31642d9a18edf4ca298e909a8fe80bdbf556d5c9573992c102ce8bf948691da91bf9739bee0ffb6e79c8a8a6e088 + languageName: node + linkType: hard + "pretty-format@npm:^29.0.0, pretty-format@npm:^29.6.1": version: 29.6.1 resolution: "pretty-format@npm:29.6.1" @@ -37872,6 +38071,13 @@ __metadata: languageName: node linkType: hard +"react-is@npm:^17.0.1": + version: 17.0.2 + resolution: "react-is@npm:17.0.2" + checksum: 9d6d111d8990dc98bc5402c1266a808b0459b5d54830bbea24c12d908b536df7883f268a7868cfaedde3dd9d4e0d574db456f84d2e6df9c4526f99bb4b5344d8 + languageName: node + linkType: hard + "react-is@npm:^18.0.0": version: 18.2.0 resolution: "react-is@npm:18.2.0" @@ -38724,6 +38930,15 @@ __metadata: languageName: node linkType: hard +"retry-axios@npm:^2.6.0": + version: 2.6.0 + resolution: "retry-axios@npm:2.6.0" + peerDependencies: + axios: "*" + checksum: cf7e63d89f00ead2633e60f00b504ec10217db8165327879b6feb0fa787fffe687d06ee145b2f43d2b4ea8916d42c951d34ee32ee1ea47c9d0b602d4963bd7f9 + languageName: node + linkType: hard + "retry-request@npm:^7.0.0": version: 7.0.2 resolution: "retry-request@npm:7.0.2"