Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

community[patch]: anthropic add tool call support new tools api #5640

Conversation

LordMsz
Copy link
Contributor

@LordMsz LordMsz commented Jun 2, 2024

This is a proposal to add support for latest tools call API for Anthropic models on Amazon Bedrock in LangChain / Community.
I originally developed my own solution, but then realized it basically matches solution in https://github.com/LordMsz/langchainjs/blob/main/libs/langchain-anthropic/src/chat_models.ts .
So I tried to sync as much as sensible as part of this change.

I'd like to ask for help getting this finalized, as I don't have experience contributing to this repo or LangChain in general.

I tested the code locally running following code:

import { config } from 'dotenv';
// import { BedrockChat } from "@langchain/community/chat_models/bedrock";
import { BedrockChat } from "./custom-models/index"; // this is my upadted version of BedrockChat that I want to propose as a PR
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { z } from "zod";
import { createToolCallingAgent, AgentExecutor } from "langchain/agents";
import { DynamicStructuredTool } from "@langchain/core/tools";

config(); // load .env file

if (!process?.env?.AWSAccessKeyId || !process?.env?.AWSSecretAccessKey) {
    console.error('AWSAccessKeyId or AWSSecretAccessKey not found in .env file');
    process.exit(1);
}

const addTool = new DynamicStructuredTool({
    name: "add",
    description: "Add two integers together.",
    schema: z.object({
        firstInt: z.number(),
        secondInt: z.number(),
    }),
    func: async ({ firstInt, secondInt }) => {
        console.log('------ Function called with: -----------', { firstInt, secondInt });
        return (firstInt + secondInt).toString();
    },
});

async function test() {
    const model = new BedrockChat({
        model: "anthropic.claude-3-sonnet-20240229-v1:0",
        temperature: 0.5,
        region: "us-east-1",
        credentials: {
            accessKeyId: process.env.AWSAccessKeyId as string,
            secretAccessKey: process.env.AWSSecretAccessKey as string,
        },
    });
    
    const prompt = ChatPromptTemplate.fromMessages([
        ["system", "You are a helpful assistant"],
        ["placeholder", "{chat_history}"],
        ["human", "{input}"],
        ["placeholder", "{agent_scratchpad}"],
    ])
    
    const agent = createToolCallingAgent({
        llm: model,
        tools: [addTool],
        prompt: prompt,
        streamRunnable: false // attempt to work around streaming issue with tools
    });
    
    const agentExecutor = new AgentExecutor({ 
        agent,
        tools: [addTool],
        // verbose: true,
        returnIntermediateSteps: true,
    });
    
    const r = await agentExecutor.invoke({ input: "Hi! Can you tell me what is 10 + 4?" });
    console.log(r);
    console.log(r.content);
}

test();

Fixes #5639 (issue)

Copy link

vercel bot commented Jun 2, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
langchainjs-api-refs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jun 7, 2024 2:17am
langchainjs-docs ✅ Ready (Inspect) Visit Preview Jun 7, 2024 2:17am

@dosubot dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. auto:improvement Medium size change to existing code to handle new use-cases labels Jun 2, 2024
@jakowenko
Copy link

jakowenko commented Jun 3, 2024

Thanks for putting this together! Make sure you format and lint the code as a large portion of the diff seems to be related to that.

@LordMsz LordMsz force-pushed the community-bedrock-anthropic-add-tool-call-support-new-tools-api-5639 branch from a11475e to c52ef97 Compare June 3, 2024 20:56
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:XXL This PR changes 1000+ lines, ignoring generated files. labels Jun 3, 2024
@LordMsz
Copy link
Contributor Author

LordMsz commented Jun 3, 2024

Apologies for the linting and formatting issue - fixed now.
I struggle to get the complete build and test of project working for some reason, but I tried all the changes with copy of the models. Any help with proper build highly appreciated!

@jacoblee93
Copy link
Collaborator

This is massive - thank you so much for this!

Will have a look tomorrow.

@jacoblee93
Copy link
Collaborator

I wonder if there's a way to reuse some of the existing logic we have in @langchain/anthropic - might have a short look to avoid duplication.

@jacoblee93 jacoblee93 changed the title #5639 community/bedrock anthropic add tool call support new tools api community[patch]: anthropic add tool call support new tools api Jun 4, 2024
@LordMsz
Copy link
Contributor Author

LordMsz commented Jun 4, 2024

Hi,
sure, I was also thinking that way - was surprised that big chunks of code are mostly a verbatim copy from @langchain/anthropic.
Ideally we would re-use it all and in Bedrock wrapper only authentication would be handled?

What is your suggestion? Should I expose the utilized methods in @langchain/anthropic, e.g. _mergeMessages. and import them here, or something else?

@jacoblee93
Copy link
Collaborator

Yes that's what I was thinking - if you're willing to take it on I think that'd be a good approach!

CC @efriis to weigh in as well.

@LordMsz
Copy link
Contributor Author

LordMsz commented Jun 5, 2024

Okay, working on it now.
I'm also trying to resolve my local build issue - seems I was missing Python 3.11 (required for node-gyp) and now I need C++ Workload in VS to add "msvs_version".
Is this documented anywhere as a per-requisite for development? Sorry if I missed it.

@@ -1 +1,2 @@
export * from "./chat_models.js";
export * from "./output_parsers.js";
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is biggest question for me - if it's inline with overall "strategy" that I can export this from Anthropic and use it in community? Any thoughts?
I only need single function from there, so that could be a minor copy-paste I assume.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think that's fine. Maybe we can prefix it with an underscore to show that it shouldn't be imported directly.

@LordMsz
Copy link
Contributor Author

LordMsz commented Jun 5, 2024

Ok, I got into a bit complex territory.

 Invalid package dependency graph: cyclic dependency detected:
  │     @langchain/community, @langchain/anthropic
  ╰─▶ cyclic dependency detected:
        @langchain/community, @langchain/anthropic

So community is used in Anthropic but I need it the other way around now.
When I simply copy-over the code into my test project, everything works code-wise.
I've cleaned up some linting errors along the way.

Any suggestions please? Any help appreciated!

Btw: I finally made local build work somewhat through dev-container; couldn't make it run on Windows at all.

@LordMsz
Copy link
Contributor Author

LordMsz commented Jun 5, 2024

I would propose to copy the required functions with TODO to refactor them to some common place both for Anthropic and Community?
I don't know the repo well enough (yet? :)) to be able to decide this on my own.

@jacoblee93
Copy link
Collaborator

Thanks for looking into this! I can resolve the build issue, it should just be dev dependencies I think.

@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Jun 7, 2024
@jacoblee93
Copy link
Collaborator

Thank you for the awesome start!

Will try to land tonight

@kamilcglr
Copy link

kamilcglr commented Jun 7, 2024

Hello, thank you all for the quick integration.

Is it normal for streaming not to work when using the createToolCallingAgent function? I see that commit contains the term streaming, but I don't understand if it's just about disabling calling tools in an async way or all events.

My use case is as follows: I am using the stream function and I want to send all the generations of the final response token by token to my client using websocket using the 'on_llm_stream' event.

for await (const event of eventStream) { ...

@jacoblee93
Copy link
Collaborator

Hey @kamilcglr, are you talking about streamEvents?

Proper streaming tool calls will take a bit more work - the delegation you see in that commit just invokes and yields the non-streaming result from Bedrock as a single chunk. You should still see it as an on_chat_model_stream event though.

@LordMsz LordMsz deleted the community-bedrock-anthropic-add-tool-call-support-new-tools-api-5639 branch June 7, 2024 20:34
@kamilcglr
Copy link

kamilcglr commented Jun 7, 2024

Hello @jacoblee93

Yes, I am talking about streamEvents, but I do not necessarily want to stream the tools, just the final answer.

Here is an example of what I am trying to do. I have a class with a run function that starts the process and sends event to my frontend application.
I used to use a similar class in Python with an SQL Agent and Bedrock with Claude 3. I managed to send all the tokens to my frontend one by one (with the on_chat_model_stream event).

class AgentService {
  private agent

  constructor() {}

  async initialize() {
    if (this.agent) return

    const llm = new BedrockChat({
      model: 'anthropic.claude-3-sonnet-20240229-v1:0',
      region: 'us-west-2',
      streaming: true,
      temperature: 0,
    })

    const tools = [
      ... some DynamicStructuredTools ...
    ]

    const agent = createToolCallingAgent({
      llm,
      tools,
      prompt: customPrompt,
      streamRunnable: true,
    })

    const agentExecutor = new AgentExecutor({
      agent,
      tools,
      handleParsingErrors: true,
      returnIntermediateSteps: true,
    }).withConfig({ runName: 'llm_agent' })

    const messageHistory = (id: string) =>
      new DynamoDBChatMessageHistory({
        tableName: 'SessionTable',
        partitionKey: 'SessionId',
        sessionId: id,
        config: {
          region: 'us-west-2',
        },
      })

    const runnableWithMessageHistory = new RunnableWithMessageHistory({
      runnable: agentExecutor,
      getMessageHistory: (sessionId: string) => messageHistory(sessionId),
      inputMessagesKey: 'input',
      historyMessagesKey: 'history',
    })

    this.agent = runnableWithMessageHistory
  }

  async run(input: string, sessionId: string) {
    // [This is where an event is sent to the client to say that the process has started]

    const eventStream = await this.agent.streamEvents(
      { input: input },
      {
        configurable: {
          sessionId: sessionId,
        },
        version: 'v1',
      }
    )

    for await (const item of eventStream) {
      if (item.event === 'on_llm_stream') {
        // Here I send "item.data.chunk.content" to the client as a completion
        // But it sends everything in a single event, whereas I expect something like: Hello,| how |can |I |help |you ? where ‘|’ is an event
      }

      if (item.event === 'on_chain_end') {
       // [End event is sent]
      }
    }
  }
}

anthonychu pushed a commit to anthonychu/langchainjs that referenced this pull request Jun 14, 2024
* docs: remove contextual search (langchain-ai#5378)

* google-genai[minor]: Add support for video/audio inputs (langchain-ai#5368)

* wip

* added audio/video

* genai update support media types

* cr

* chore: lint files

* cr

* all[patch]: Loosen core dependencies (langchain-ai#5367)

* all[patch]: Loosen core dependencies

* make community and lc depend on core rc

* resoultions

* community[patch]: Release 0.2.0-rc.0 (langchain-ai#5384)

* partnet packages[patch]: Release all (langchain-ai#5388)

* partnet packages[patch]: Release all

* bump other google packages

* docs: feedback link in banner (langchain-ai#5387)

* anthropic[patch]: Run formatting (langchain-ai#5390)

* docs[patch]: update api ref base url to contain v2 subdomain (langchain-ai#5392)

* Update debugging and streaming guide content (langchain-ai#5395)

* Fix link (langchain-ai#5396)

* docs[minor]: Delete 'old' docs folder & contents (langchain-ai#5394)

* Guide content fixes (langchain-ai#5400)

* Update default headers azure & Add Token Provider Check (langchain-ai#5379)

* Update Default Headers for Azure OpenAI Requests

* nit

* Update user agent string

* Add Token Provider check

* Update libs/langchain-openai/src/azure/llms.ts

Co-authored-by: Yohan Lasorsa <noda@free.fr>

* Update libs/langchain-openai/src/azure/embeddings.ts

Co-authored-by: Yohan Lasorsa <noda@free.fr>

* Update libs/langchain-openai/src/azure/chat_models.ts

Co-authored-by: Yohan Lasorsa <noda@free.fr>

* Format the code

---------

Co-authored-by: Yohan Lasorsa <noda@free.fr>

* openai[patch]: 0.0.31 (langchain-ai#5401) (langchain-ai#5402)

* docs[patch]: LangGraph Docs remove unused import from code examples (langchain-ai#5391)

Co-authored-by: Brace Sproul <braceasproul@gmail.com>

* core[major]: Update streamEvent signature to return IterableReadableStream (langchain-ai#5360)

* Update streamEvent signature to return IterableReadableStream

* Fix lint

* Update return type to be sync

* docs[patch]: Move installation doc to how tos (langchain-ai#5399)

* docs[patch]: Move instalation doc to how tos

* update sidebar

* cr

* fix[runOnDataset]: accept custom criteria in config factory

* docs[patch]: Add logprob docs, more updates (langchain-ai#5404)

* Add logprob docs, more updates

* More content updates

* docs[patch]: Add onlyWsa prop to chat model tabs, other nits (langchain-ai#5405)

* docs[patch]: Add onlyWsa prop to chat model tabs, other nits

* fix link

* chore: lint files

* cr

* docs[minor]: Show sidebar inside how to/tutorial (langchain-ai#5406)

* docs[minor]: Show sidebar inside how to/tutorial

* chore: lint files

* cr

* trailing slash

* docs: fix monorepo typo (langchain-ai#5411)

* docs[minor]: Match py how to index page (langchain-ai#5414)

* docs[minor]: Match py how to index page

* chore: lint files

* anthropic[minor]: Add tool_choice arg (langchain-ai#5416)

* anthropic[minor]: Add tool_choice arg

* chore: lint files

* release 0.1.19

* langchain[patch]: Add deprecation warnings to document loaders (langchain-ai#5419)

* langchain[patch]: Add deprecation warnings to document loaders

* chore: lint files

* docs[patch]: Broken links

* docs[patch]: Fix some broken links

* docs[patch]: Fix broken links (langchain-ai#5420)

* push

* cr

* make docusourus warn not throw on broken links

* chore: lint files

* Update text splitter docs (langchain-ai#5424)

* docs[minor]: Hide prev/next buttons on how to and tutorials (langchain-ai#5425)

* docs[minor]: Hide prev/next buttons on how to and tutorials

* chore: lint files

* chore: lint files

* docs[minor]: Log actual prompt when using prompt hub (langchain-ai#5423)

* docs[minor]: Log actual prompt when using prompt hub

* nit

* docs[minor]: version dropdown (langchain-ai#5422)

* docs: version dropdown

* Update docs/core_docs/docusaurus.config.js

* Update docs/core_docs/docusaurus.config.js

* docs[minor]: Add tools prompting docs (langchain-ai#5431)

* docs[minor]: Add tools prompting docs

* fix outputs

* docs[patch]: Update API ref url in docs to v02 (langchain-ai#5432)

* docs[patch]: Update retrieval and embeddings docs (langchain-ai#5429)

* Update retrieval and embeddings docs

* Remove silly rule

* Replace prerequisite links component (langchain-ai#5430)

* api_refs[minor]: Add version dropdown] (langchain-ai#5435)

* core[minor]: Unified model params for LS (langchain-ai#5427)

* core[minor]: Unified model params for LS

* add to openai, anthropic and together

* mistral

* cohere fireworks ollama

* fix tests

* remove underscore prefix

* fix metadata placement

* format

* fixes

* chore: lint files

* docs[patch]: Remove links to missing docs (langchain-ai#5438)

* docs[patch]: Remove links to missing docs

* fix title

* core[patch]: Move async generator consumption code into local storage context (langchain-ai#5439)

* Move async generator consumption code into local storage context

* Typing fix, adds test, remove unnecessary void

* lint n stuff

---------

Co-authored-by: bracesproul <braceasproul@gmail.com>

* core[patch]: Release 0.2.0 (next) (langchain-ai#5443)

* environment_tests[minor],langchain[minor]: Upgrade core dep to 0.2.0 (langchain-ai#5444)

* langchain[minor]: Upgrade core dep to 0.2.0

* update export tests to not use rc core version

* langchain[patch]: Release 0.2.0 (next) (langchain-ai#5445)

* community[minor]: Bump LC and core deps to 0.2.0 (langchain-ai#5446)

* community[patch]: Release 0.2.0 (next) (langchain-ai#5447)

* @langchain/google-genai [feature]: support Google genai API version and base URL (langchain-ai#5426)

* feat: support Google genai API version and base URL

* run lint n format

---------

Co-authored-by: Brace Sproul <braceasproul@gmail.com>

* all[minor]: Update partner packages core dep semver (langchain-ai#5448)

* partner packages[patch]: Release all (langchain-ai#5449)

* google-pkgs[patch]: Release Google packages (langchain-ai#5450)

* google-pkgs[patch]: Release Google packages

* chore: lint files

* partner-packages[patch]: Update core dep versions to contain single space (langchain-ai#5452)

* partner-packages[patch]: Update core dep versions to contain single space

* cr

* Fix styling (langchain-ai#5454)

* docs[major]: Fix broken links and turn on throw on broken link (langchain-ai#5455)

* docs[major]: Fix broken links and turn on throw on broken link

* chore: lint files

* Update QA use-case docs how tos (langchain-ai#5453)

* Update other howto use-case docs (langchain-ai#5457)

* docs[patch]: Update tools docs (langchain-ai#5458)

* Update tools docs

* Links

* core[patch]: Fix bindTools return type (langchain-ai#5459)

* Fix bindTools return type

* Format

* groq[patch]: Release 0.0.12 (langchain-ai#5461)

* anthropic[patch]: Release 0.1.21 (langchain-ai#5462)

* openai[patch]: Release 0.0.33 (langchain-ai#5463)

* google-common[patch]: Release 0.0.16 (langchain-ai#5464)

* mistralai[patch]: Remove default tool call value for translated messages (langchain-ai#5428)

* mistral[patch]: Remove default tool call value for translated messages

* Fix

* Fix

* Improve error handling

* mistralai[patch]: Release 0.0.22 (langchain-ai#5465)

* docs[patch]: Adds self query docs, LCEL cheatsheet (langchain-ai#5466)

* Adds self query docs, LCEL cheatsheet

* Bump deps

* Add dep

* Deno dep update

* docs[patch]: Contributor guide, polish (langchain-ai#5468)

* Contributor guide, polish

* Typo

* Fix build

* docs[patch]: Adds callback docs (langchain-ai#5469)

* Adds callback docs

* Updates

* Update banner

* Update copy

* Rename doc page (langchain-ai#5470)

* docs[patch]: Update tutorials (langchain-ai#5473)

* Update tutorials

* Fix LangSmith section:

* community[patch]: Neo4j metadata filtering (langchain-ai#5215)

* Implement addGraphDocuments method for neo4j graph~

* Add integration tests for new method and make them pass.

* Revert back to int skipped, because remote db

* Fix Serializable extension

* Make metadata optional

* Linting

* Format

* Update extensions and props passing

* Format

* Add metadata filtering

* Fix metadata test equality check. Format.

* Skip int test.

* Refactor IndexType from enum to plain type

* Move filter arg to params

* Format

* google-common[minor],google-genai[minor]: Standardize LS params (langchain-ai#5486)

* community[minor]: Adds support for Cohere Command-R via AWS Bedrock  (langchain-ai#5336)

* add-cohere-command-r-formatted-messages

* add integration test patterns

* fixed typo

* add comments

* fixed input system prompts check

* fixed cohere modelName validation

* fixed lint

* Small typo fix

* Add special message parsing for Cohere

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* docs[patch]: Update output parser docs (langchain-ai#5408)

* Update output parser docs

* Typo

* Remove bad link

* Fix broken links

* all[patch]: Ignore no unused vars eslint rule in test files (langchain-ai#5440)

* all[patch]: Ignore no unused vars eslint rule in test files

* chore: lint files

* google-genai[patch]: Fix api options which caused requests to hang (langchain-ai#5490)

* google-genai[patch]: Release 0.0.15 (langchain-ai#5491)

* docs[patch]: update Azure documentation and deprecate @langchain/azure-openai (langchain-ai#5477)

* docs: update Azure OpenAI chat documentation

* docs: update azure chat openai

* docs: update Azure OpenAI LLM docs

* docs: update docs for Azure OpenAI embeddings

* docs: update wording

* docs: fix example path

* docs: add deprecation notice

* docs: update Microsoft integrations docs

* chore: fix typo

* docs: fix install

* Format

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* Fix docs infinite redirect, add redirect (langchain-ai#5492)

* Replace ecosystem pages with real links (langchain-ai#5494)

* community[minor]: updated Browserbase loader (langchain-ai#5412)

* updated browserbase loader

* ran format

* Move to community

* Format

* Loosen peer dep

---------

Co-authored-by: Mish Ushakov <mishushakov@users.noreply.github.com>
Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* community[minor]: Add Google Routes Tool (langchain-ai#5329)

* Initial routes feature

* Update field masks, removed json mapping

* Fixed JSON return and better fieldMask

* working condition for all types of transport

* Removed departure and arrival time due to timezone issues

* Better prompt, better typescript, remove warning, fixed duplicated labels

* Working routes tool

* Add GoogleRoutesAPI interface and remove console.log statements

* Update Google Routes Tool documentation

* Updated example to include a example output

* Helper functions for better readability

* testing for google routes tool

* Changed tool to StructuredTool, input is now an object

* Updates tests for google routes tool

* Better description

* Removed some lines to make explanation simpler

* Add TollInfo interface and update filteredRoutes function in google_routes.ts

* Working version with other features

* Fix handling of past departure and arrival times in GoogleRoutesAPI

* Format

* Skip tests by default

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* community[minor]: Spider Document Loader (langchain-ai#5415)

* files

* format

* update yarn.lock

* new version of spider-client

* new spider-client version

* spider constructor

* correct doc

* change ref link

* commented out params in example

* updated spider-client version

* changed location of files

* change location of files

* changed file location

* updated correct imports

* Update spider.ts

---------

Co-authored-by: Brace Sproul <braceasproul@gmail.com>
Co-authored-by: Jacob Lee <jacoblee93@gmail.com>

* community[minor]: feat: Layerup Security integration (langchain-ai#4929)

* feat: Layerup Security integration

* nit: remove environment variable reference

* docs: Layerup Security docs

* fix: package.json, fix docs location

* feat: bump layerup-security version, support untrusted_input

* nit: LLM -> BaseLLM

* Update lock

* Build and lint fixes

* fix: bump layerup-security SDK version to 1.5.9

* Update lockfile

* feat: bump SDK to 1.5.10 to match OpenAI requirement

* Bump version

* fix: type errors

* Bump dep

* Rename test

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* docs[minor]: Unstructured MD loader doc (langchain-ai#5489)

* docs[minor]: Unstructured MD loader doc

* chore: lint files

* html loader

* update firecrawl loader

* fix firecrawl loader

* cr

* Add links to docs from how to index

* cr

* chore: lint files

* community[patch]: VoyageAI embedding with input_type parameter (langchain-ai#5493)

* added input_type

* input_type set to "none" by default , then choose between document or query

* Updated input_type for Voyage embeddings

* Update voyage.ts

* Update voyageai.mdx

* Update voyageai.mdx

* Update voyage.ts

* Update voyageai.mdx

¯\_(ツ)_/¯

* Update voyage.int.test.ts

With document to make sure it works

* Format

* Update voyageai.mdx

is it the space ? (sorry...)

* Format

* Fix types

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* Fix build artifacts (langchain-ai#5496)

* Temporary build fix (langchain-ai#5499)

* community[patch]: Release 0.2.1 (langchain-ai#5500)

* scripts[minor]: Fix extra newlines issue, switch all packages to use workspace dep (langchain-ai#5498)

* scripts[minor]: Fix extra newlines issue, switch all packages to use workspace dep

* cr

* docs[minor]: Add AI SDK docs (langchain-ai#5497)

* docs[minor]: Add AI SDK docs

* chore: lint files

* ai streaming tools doc

* chore: lint files

* drop docs, move links to tutorials section

* chore: lint files

* Add neo4j metadata filtering docs (langchain-ai#5502)

* langchain[patch]: Add entrypoints to deprecated omit import map (langchain-ai#5511)

* langchain[patch]: Add entrypoints to deprecated omit import map

* rm rf build artifacts

* langchain[patch]: Release 0.2.1 (langchain-ai#5512)

* google-genai[minor]: Add tool calling support (langchain-ai#5507)

* google-genai[minor]: Add tool calling support

* cr

* chore: lint files

* fixes

* docs and build error fixes

* dont set name in response

* add ls links

* cr

* Multi modal docs

* :cr

* fixup multi modal tool calls

* cr

* google-genai[patch]: Release 0.0.16 (langchain-ai#5513)

* community[minor]: Allow more forms of credentials for aws bedrock (langchain-ai#5514)

* community[minor]: Allow more forms of credentials for aws bedrock

* chore: lint files

* docs: move feedback into paginator from content

* core[patch]: Add support for messages in/messages out for RunnableWithMessageHistory (langchain-ai#5517)

* Add support for messages in/messages out for RunnableWithMessageHistory

* Add test

* docs[patch]: Fix lint (langchain-ai#5526)

* Fix lint

* Fix

* community[patch]: Release 0.2.2 (langchain-ai#5527)

* chore: Update `@google-ai/generativelanguage` module to 2.5.0 (langchain-ai#5523)

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Bump package (langchain-ai#5529)

* docs[minor]: Add generative UI docs (langchain-ai#5528)

* docs[minor]: Add generative UI docs

* chore: lint files

* broken link

* langchain[patch]: Readd dropped entrypoint (langchain-ai#5530)

* Fix memory entrypoint

* Update examples

* Release 0.2.2 (langchain-ai#5531)

* scripts[patch]: Add .js extension to build script (langchain-ai#5536)

* scripts[patch]: Add .js extension to build script

* cr

* yarn.lock

* commit it

* docs[minor]: Multi modal docs (langchain-ai#5537)

* docs[minor]: Multi modal docs

* cr

* scripts[major]: New build (langchain-ai#5538)

* scripts[major]: New build

* perfect build script and add to core!

* chore: lint files

* very nice much better

* yarn format && yarn lint:fix

* all build script updates

* standard clean scrop[t]

* add turbo to every repo

* add back test:ci

* remove concurrency limits in tests & ignore non testable packages

* core[minor]: Stream events v2 (langchain-ai#5539)

* Start stream events v2

* More progress

* Fixes

* Docs

* Docs

* Fix build

* community[minor]: Add Upstash Embeddings Support (langchain-ai#5150)

* add: metadata filtering for UpstashVectorStore

* fmt

* skip tests

* add: upstash embeddings support

* add: tests

* fmt

* Update upstash.mdx

* Update upstash.mdx

* use fake embeddings

* fix: replace UpstashEmbeddings parameter with FakeEmbeddings class

* Naming

---------

Co-authored-by: Jacob Lee <jacoblee93@gmail.com>

* community[patch]: langchain-ai#3369 Streaming support for Replicate models (langchain-ai#5365)

* langchain-community[patch]: langchain-ai#3369 Streaming support for Replicate models

* lock

* Add streaming test

---------

Co-authored-by: jeasonnow <guyf@seeyon.com>
Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* community[minor]: Add Moonshot chat models integration (langchain-ai#5239)

* Add Moonshot chat models

* fix: MOONSHOT_API_KEY needs to be capitalized

* feat: remove moonshotApiKey

* fix: remove encodeApiKey

* docs: complete parameter comment

* Format

* Lint

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* community[minor]: Integrate zep cloud components (langchain-ai#5542)

* feat: Add Zep Cloud Langchain components + corresponding docs

* chore: Run formatter

* chore: Throw an error on unsupported vector store methods

* Fix lint, update deprecated code

---------

Co-authored-by: paulpaliychuk <pavlo.paliychuk.ca@gmail.com>

* qdrant[minor]: Support maxMarginalRelevanceSearch() (langchain-ai#5467)

* community[minor]: Add UpstashRatelimitHandler (langchain-ai#5474)

* add upstash ratelimit callback

* add ratelimit package and imports

* improve error messages

* fix tests

* fix typo

* Polish, format, lint

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* community[minor]: add Neo4j Graph enhancedSchema option (langchain-ai#5413)

* Implement addGraphDocuments method for neo4j graph~

* Add integration tests for new method and make them pass.

* Revert back to int skipped, because remote db

* Fix Serializable extension

* Make metadata optional

* Linting

* Format

* Update extensions and props passing

* Format

* Add enhanced schema option.

* Format

* Update test

* Update test

* Format

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* community[patch]: add ?| (arrayContains) filter on metadata to PGVector search (langchain-ai#5381)

* add ?| (arrayContains) filter on metadata to PGVector search

* fix: align to style of IN filter

* Format

* Update build artifacts

* Format

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* community[patch]: Better errors for neo4j vector (langchain-ai#5501)

* Better errors for neo4j vector

* Lint

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* community[patch]: Default embeddingNodeProperty value (langchain-ai#5510)

* Default embeddingNodeProperty value

If this property is undefined when calling `Neo4jVectorStore.fromExistingGraph()`, the property is named `undefined`.

* Format

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* docs[patch]: update callbacks (langchain-ai#5515)

* x

* x

* x

* Format

* Update

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* community[patch]: include metadata returned by Amazon bedrock knowledge base (langchain-ai#5535)

* include metadata returned by Amazon bedrock knowledge base

* Update amazon_knowledge_base.ts

* Update dep

---------

Co-authored-by: Jacob Lee <jacoblee93@gmail.com>

* preserve ordinality in postgres when checking existence of documents (langchain-ai#5533)

* docs[minor]: LangGraph Migration Guide (langchain-ai#5487)

* [Docs] LangGraph Migration Guide

* fixup

* link

* Update

* Update and polish

* Add to how to index page

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* qdrant[patch]: Release 0.0.5 (langchain-ai#5543)

* community[patch]: Release 0.2.3 (langchain-ai#5544)

* core[patch]: Release 0.2.1 (langchain-ai#5545)

* pinecone[patch],weaviate[patch]: Bump weaviate and pinecone core deps (langchain-ai#5546)

* Bump Pinecone core deps

* Bump weaviate core dependency

* pinecone[patch]: Release 0.0.7 (langchain-ai#5547)

* weaviate[patch]: Release 0.0.4 (langchain-ai#5548)

* openai[patch]: fix langchain-ai#5520 (langchain-ai#5521)

* langchain-openai[patch]: fix langchain-ai#5520

* fix: don't rewrite config if they are already set in embedding and chat_model

* Format

---------

Co-authored-by: jeasonnow <guyf@seeyon.com>
Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* openai[patch]: Release 0.0.34 (langchain-ai#5562)

* core[patch]: Keep event stream for streamEvents v2 open until end of run (langchain-ai#5561)

* Keep event stream for streamEvents v2 open until end of run

* Style nit

* Lint

* Small fix

* core[patch]: Release 0.2.2 (langchain-ai#5563)

* docs[patch]: Missing how to index section for multi modal (langchain-ai#5575)

* docs[patch]: Missing how to index section for multi modal

* format

* api_refs[patch]: Fix link to v0.1 api refs (langchain-ai#5576)

* infra[minor]: Speed improvements for CI (langchain-ai#5580)

* infra[minor]: Speed improvements for CI

* cr

* infra[patch]: Improve yarn format (langchain-ai#5578)

* infra[patch]: Improve yarn format

* chore: lint files

* cr

* Fixed Typo on Build Simple LLM application page (langchain-ai#5566)

typo

* google-common [patch], google-* [tests]: Fix streaming false (langchain-ai#5571)

* Fix for issue langchain-ai#5475 - no tools_call in message.
Natively support non-streaming mode.

* Test for structuredOutput (see langchain-ai#5475).
Fixes for other tests.

* Fixes for tests.

* Formatting.

* fix: moonshot did not return usage cause error (langchain-ai#5551)

* google-common[patch]: Zod to Gemini params conversion when the schema contains arrays of items (langchain-ai#5553)

* Fix: [libs/langchain-google-common] already existing utils test

* Fix: [lib/langchain-google-common] Add failing test for arrays containing additional properties

* Fix: [lib/langchain-google-common] Fix failing test

* Update int test

* lint

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* google-common[patch]: Release 0.0.17 (langchain-ai#5588)

* google-*[patch]: Release 0.0.17 (langchain-ai#5589)

* langchain[patch]: Support for sqljs (langchain-ai#5560)

* Support for sqljs

Added support for sqljs, the browser transpile of sqlite

* Format

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* community[patch]: Upstash Vector Store Namespace Feature (langchain-ai#5557)

* feat: Upstash Vector Namespace feature

* add: Upstash Vector Namespace Tests

* docs: Upstash Vector namespace

* fmt

* infra[patch]: Set fail-fast strategy to false (langchain-ai#5590)

* langchain[patch],community[patch]: Loosen peer deps (langchain-ai#5583)

* langchain[patch],community[patch]: Loosen peer deps

* loosen more deps and cut community rc

* cut rc for langchain and community again

* drop unused google pkg from langchain

* Release 0.2.3 (langchain-ai#5591)

* community[patch]: Release 0.2.4 (langchain-ai#5592)

* infra[major]: Commit yarn cache (langchain-ai#5593)

* community[patch]: support stream for wenxin and zhipu chat (langchain-ai#5573)

* langchain-community[patch]: support stream for wenxin and zhipu chat model

* Format

* refactor: remove stream util

---------

Co-authored-by: jeasonnow <guyf@seeyon.com>

* community[patch]: add support for or in elastic vector query (langchain-ai#5568)

* langchain[patch]: add support for or in elastic vector query

Fixes langchain-ai#5558.

const filter = [
  { field: "genre", operator: "or", value: "Drama" },
  { field: "genre", operator: "or", value: "Action" },
];


const reteriver = vectorStore.asRetriever({
filter: filter
)
or 
ScoreThresholdRetriever.fromVectorStore(vectorStore, {
        minSimilarityScore: 0.5, // Finds results with at least this similarity score
        maxK: 1, // The maximum K value to use. Use it based to your chunk size to make sure you don't run out of tokens
        kIncrement: 1, // How much to increase K by each time. It'll fetch N results, then N + kIncrement, then N + kIncrement * 2, etc.
        filter: filter,
      });


After this change the filter will be passed to the elastic vector similaritySearchVectorWithScore function.

const relevantDocuments = await elasticVectorSearch.similaritySearchVectorWithScore(
  [/* your query vector */],
  10,
  filter
);

* Added type for should

* Format

* Lint

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* core[patch]: Support LANGSMITH_TRACING env var (langchain-ai#5587)

* [Core:Tracing] accept LANGSMITH_TRACING

* Rework check

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* core[patch]: Set global async local storage instance (langchain-ai#5601)

* Set global async local storage instance

* Rename

* Rename

* core[minor]: Make LLMs and chat models always stream when invoked within streamEvents (langchain-ai#5604)

* Make LLMs and chat models always stream when invoked within streamEvents

* Lint

* Lint

* core[patch]: Release 0.2.3 (langchain-ai#5606)

* docs[patch]: Update README and other links (langchain-ai#5605)

* Update README and other links

* nit

* Transparent handoff between @Traceable and LangChain

* Add LangSmith to langchain-core internals

* Fix CallbackManagerRunTree

* Use singletons

* Use singletons

* Resolve circular dependency

* Inherit more properties

* Remove comment, now that we have a singleton

* Use langsmith@0.1.28

* Fix base

* Add package

* Upstream changes from SDK

* Update to langsmith@0.1.30

* Update base.ts

* core[patch]: Release 0.2.4 (langchain-ai#5609)

* docs: redirect integration links to 0.2 (langchain-ai#5608)

* mistralai[minor]: Add llms entrypoint, update chat model integration (langchain-ai#5603)

* mistralai[minor]: Add llms entrypoint

* chore: lint files

* cr

* docs

* docs nit

* docs nit

* update mistral-large models to mistral-large-latest

* fix tools

* add codestral chat model tests

* chore: lint files

* cr

* mistralai[patch]: Release 0.0.23 (langchain-ai#5613)

* langchain[patch],core[patch]: Fix agent executor stream event behavior (langchain-ai#5614)

* Fix agent executor stream event behavior

* Clean up streamEvents implementation, small remote runnable fix

* Fix lint

* core[minor],openai[patch]: Add usage metadata to `AIMessage`/`Chunk` (langchain-ai#5586)

* core[minor],openai[patch]: Add usage metadata to base ai message

* chore: lint files

* Only set stream_options if the model is OpenAI

* add test confirming last finish reason is stop

* chore: lint files

* docs

* lint n format

* cr

* cr

* Remove Azure specific check

* Remove comment

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* core[patch]: Release 0.2.5 (langchain-ai#5615)

* Bump OpenAI deps (langchain-ai#5616)

* openai[minor]: Release 0.1.0 (langchain-ai#5617)

* community[patch],langchain[patch],groq[patch]: Bump deps (langchain-ai#5618)

* Bump deps

* Fix

* Release 0.2.4 (langchain-ai#5619)

* Release 0.2.4

* Update deps

* community[minor]: DeepInfra embeddings integration #1 (langchain-ai#5382)

* Init

* fix(type errors)

* feat(deepinfra embeddings)

* fix(default model)

* fix(deepinfra): axios is removed

* ref(deepinfra): remove redundant cast

* format(deepinfra)

* doc(deepinfra)

* doc(deepinfra)

* Update deepinfra.mdx

* Format

---------

Co-authored-by: Jacob Lee <jacoblee93@gmail.com>

* Fix bug in Neo4j new enhancedSchema flag (langchain-ai#5622)

* community[minor]: Add support for Tencent Hunyuan Chat Model and Embeddings (langchain-ai#5476)

* community: Add support for Tencent Hunyuan chat model and embeddings

Tencent provides its Hunyuan chat model and embeddings through Tencent cloud. This PR adds support for the [chat model](https://cloud.tencent.com/document/product/1729/105701) as well as [embedding](https://cloud.tencent.com/document/product/1729/102832).

* community[minor]: Add support for Tencent Hunyuan Chat Model and Embeddings

* refactor Chat Model streaming implementation with AsyncGenerator
* update stream example

* community[minor]: Add support for Tencent Hunyuan Chat Model and Embeddings

* make host configurable

* community[minor]: Add support for Tencent Hunyuan Chat Model and Embeddings

* support both nodejs and browser environment
* update documents and examples

* community[minor]: Add support for Tencent Hunyuan Chat Model and Embeddings

* make separate chat model and embedding entrypoints for web vs. node

* community[minor]: Add support for Tencent Hunyuan Chat Model and Embeddings

* rename docs sidebar name

* community[minor]: Add support for Tencent Hunyuan Chat Model and Embeddings

* format code

* Update build, lint, format

* Type export

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* Fix security header (langchain-ai#5625)

* community[patch]: overriding similaritySearchWithScore in neo4j vector store to enable filtering (langchain-ai#5599)

* overriding similaritySearchWithScore

* Format

* Fix build

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* LangGraph fix typo (langchain-ai#5626)

* Update import map (langchain-ai#5627)

* Remove yarn cache (langchain-ai#5628)

* community[patch]: Release 0.2.5 (langchain-ai#5629)

* 👥 Update LangChain people data (langchain-ai#5630)

Co-authored-by: github-actions <github-actions@github.com>

* Bump flaky better-sqlite3 dep (langchain-ai#5633)

* openai[patch],anthropic[patch]: Populate usage_metadata on invoke, add docs (langchain-ai#5632)

* Populate usage_metadata in more places

* Add usage metadata for Anthropic

* Remove unnecessary check

* Update artifacts

* Fix typo

* Fix multimodal tool call docs

* openai[patch]: Release 0.1.1 (langchain-ai#5634)

* anthropic[minor]: Release 0.2.0 (langchain-ai#5635)

* Switch to monthly shields (langchain-ai#5642)

* docs: fix package path (langchain-ai#5653)

* Update googlevertexai.mdx (langchain-ai#5638)

Fix import paths

* Fix JSDoc parameter description typo (langchain-ai#5644)

* docs[patch]: Update Unstructured Docker Image Reference in Documentation (langchain-ai#5651)

* Update unstructured.mdx

* Update document_loader_html.ipynb

* Update document_loader_markdown.ipynb

* docs[patch]: Add keywords for common queries (langchain-ai#5655)

* Add keywords for common queries

* Format

* standard-tests[major], openai[minor]: Init package & add standard tests to openai (langchain-ai#5612)

* standard-tests[major]: Init package

* fix build, cleanup

* base class and unit tests

* format n lint

* updates

* Added standard unit tests to oai and scripts, made getLsParams public

* yarn install

* always include api key

* set env var instead of constructor args

* added integration tests

* add int tests to openai

* add wso tests

* Add to root & turbo, added gh action

* usage metadata tests for streaming and invoke

* Added call options arg to integration tests

* tmp run standard test action in pr ci

* remove from pr ci

* test moar things, add readme

* chore: lint files

* token usage standard tests non streaming

* docs[patch]: Sitemap fixes (langchain-ai#5658)

* Sitemap fixes

* Update

* anthropic[minor]: Add standard chat model tests to anthropic (langchain-ai#5659)

* anthropic[minor]: Add standard chat model tests to anthropic

* format

* docs[patch]: Update quickstart tutorial (langchain-ai#5662)

* Update quickstart tutorial

* Broken link

* mongodb[minor]: add, implement delete method (langchain-ai#5559)

* feat: atlas search vector store delete method

Method delete was missing in MongoDBAtlasVectorSearch class and
it is needed if incremental indexing strategy is used

* refactor: use built-in function for array chunking

---------

Co-authored-by: Guille <guillermo.c.martinez@dcsl.com>

* mongodb[patch]: Release 0.0.4 (langchain-ai#5664)

* community[minor]: upgraded @mlc/web-llm dependency and updated it's ChatModel (langchain-ai#5637)

* chore(community/webllm): upgraded @mlc/web-llm dependency and updated it's ChatModel

* Format

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* langchain[minor]: add EnsembleRetriever (langchain-ai#5556)

* langchain[patch]: add support to merge retrievers

* Format

* Parallelize, lint, format, small fixes

* Add entrypoint

* Fix import

* Add docs and fix build artifacts

---------

Co-authored-by: jeasonnow <guyf@seeyon.com>
Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* community[minor]: Add support for bedrock guardrails and trace (langchain-ai#5631)

* add support for bedrock guardrails and trace

* updated bedrock tests for guardrails

* resolved guardrail and trace format errors, also resolved type errors for fetchFn

* updated chatbedrock test with guardrails

* Format and revert call option changes

* Lint

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* Release 0.2.5 (langchain-ai#5665)

* docs[patch]: Redirects for LangSmith and LangGraph (langchain-ai#5667)

* LangGraph docs link redirect

* LangSmith redirect

* community[patch]: Release 0.2.6 (langchain-ai#5668)

* partners[minor]: Add standard chat model tests to partner packages (langchain-ai#5660)

* partners[minor]: Add standard chat model tests to partner packages

* google genai

* yarn

* groq

* groq nit and mistral

* add to azure in chat openai

* chore: lint files

* drop azure openai pkg

* add generic constructor args to standard tests pkg

* implement cloudflare standard tests

* implement cohere standard tests

* google genai package standard tests

* groq

* allow for custom function call ids, fix mistral

* azure tests

* chore: lint files

* update standard tests gh action to run all pkgs

* chore: lint files

* revert workflow file rename

* fix workflow job naming issue

* add anthropic, fix api keys

* cache deps?

* fix build

* update standard tests

* cr

* fix

* remove dep on job which doesnt exist

* cr

* cr

* core[patch]: Fix formatting mustache image templates (langchain-ai#5666)

* core[patch]: Fix formatting mustache image templates

* unfocus test

* fix tests

* community[minor]: Add standard tests to community chat models (langchain-ai#5669)

* partners[minor]: Add standard chat model tests to partner packages

* google genai

* yarn

* groq

* groq nit and mistral

* add to azure in chat openai

* chore: lint files

* drop azure openai pkg

* add generic constructor args to standard tests pkg

* implement cloudflare standard tests

* implement cohere standard tests

* google genai package standard tests

* groq

* allow for custom function call ids, fix mistral

* azure tests

* chore: lint files

* update standard tests gh action to run all pkgs

* chore: lint files

* revert workflow file rename

* fix workflow job naming issue

* add anthropic, fix api keys

* cache deps?

* fix build

* update standard tests

* cr

* fix

* remove dep on job which doesnt exist

* cr

* cr

* community[minor]: Add standard tests to community chat models

* integration tests

* chore: lint files

* add bedrock test

* core[release]: Release 0.2.6 (langchain-ai#5670)

* docs[patch]: Adds heading keywords for search (langchain-ai#5678)

* Adds heading keywords for search

* Format

* Change neo4j verify connectivity method (langchain-ai#5679)

* partners[patch]: Release partner & community packages (langchain-ai#5688)

* openai[patch]: Release 0.1.2 (langchain-ai#5689)

* community[patch]: Deprecate Google PaLM integrations (langchain-ai#5690)

* community[patch]: Deprecate Google PaLM integrations

* chore: lint files

* docs[patch]: Adds PDF ingestion and QA tutorial (langchain-ai#5692)

* Adds PDF ingestion and QA tutorial

* Typo

* typo

* Bump dependency range for @langchain/textsplitters (langchain-ai#5693)

* textsplitters[patch]: Release 0.0.3 (langchain-ai#5695)

* docs[patch]: Add deprecation warnings to Google PaLM docs (langchain-ai#5696)

* community[patch]: Loosen conflicting peer dep (langchain-ai#5694)

* Loosen conflicting peer dep

* Bound better

* Deps

* Bump

* community[patch]: Release 0.2.8 (langchain-ai#5697)

* docs: remove duplicated heading section in rag.ipynb (langchain-ai#5674)

Co-authored-by: Brace Sproul <braceasproul@gmail.com>

* community[patch]: anthropic add tool call support new tools api (langchain-ai#5640)

* 5639 community/bedrock anthropic add tool call support new tools api

* Re-use most of parsing logic from @langchain/anthropic directly; export it there

* Fix build errors

* Updated docs, BedrockChat partial support for Tool Calling, Anthropic models

* Small updates

* Format

* Revert Anthropic

* Refactor to remove dep

* Delegate streaming tool calls to invoke for now

* Fix

* Bump core version

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* community[minor]: Fixed ChatWebLLM reload function and updated model name in example (langchain-ai#5671)

* [fix] Fix parameter order of reload and update model name in example

* Update example

---------

Co-authored-by: jacoblee93 <jacoblee93@gmail.com>

* community[patch]: Release 0.2.9 (langchain-ai#5698)

* docs[patch]: Add crosslinks, add LangGraph and LangSmith sections (langchain-ai#5702)

* Add crosslinks, add LangGraph and LangSmith sections

* Fix anchors

* docs: add azure dynamic sessions documentation

* Fix build and make it CJS compatible

* Update error message

* Add sessions to platform docs

* Formatting

* Add env var

* Add package to examples

* Fix agent example

* Add readme

* Fix bad merge

* Add missing package

* Fix formatting

* Fix langchain/core dependency

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Erick Friis <erick@langchain.dev>
Co-authored-by: Brace Sproul <braceasproul@gmail.com>
Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>
Co-authored-by: Jacob Lee <jacoblee93@gmail.com>
Co-authored-by: Sarangan Rajamanickam <sarajama@microsoft.com>
Co-authored-by: theItalianDev <giuseppedigiorno@yahoo.it>
Co-authored-by: Tat Dat Duong <david@duong.cz>
Co-authored-by: yoogle <dmltjs851@gmail.com>
Co-authored-by: sugarforever <william.1022@gmail.com>
Co-authored-by: Anej Gorkič <easwee@gmx.com>
Co-authored-by: m-hamaro <105864894+m-hamaro@users.noreply.github.com>
Co-authored-by: Mish Ushakov <10400064+mishushakov@users.noreply.github.com>
Co-authored-by: Mish Ushakov <mishushakov@users.noreply.github.com>
Co-authored-by: Luis Otavio Martins <luisotaviomgr1@gmail.com>
Co-authored-by: WilliamEspegren <131612909+WilliamEspegren@users.noreply.github.com>
Co-authored-by: Jamsheed Mistri <jmistri7@gmail.com>
Co-authored-by: Nicolas <8360521+nicolas-geysse@users.noreply.github.com>
Co-authored-by: Tomaz Bratanic <bratanic.tomaz@gmail.com>
Co-authored-by: oleg <me@olegivaniv.com>
Co-authored-by: Fahreddin Özcan <88107904+fahreddinozcan@users.noreply.github.com>
Co-authored-by: santree <santree122@gmail.com>
Co-authored-by: jeasonnow <guyf@seeyon.com>
Co-authored-by: Crazy Urus <crazyurus@vip.qq.com>
Co-authored-by: paulpaliychuk <pavlo.paliychuk.ca@gmail.com>
Co-authored-by: Anush <anushshetty90@gmail.com>
Co-authored-by: Cahid Arda Öz <cahidardaooz@gmail.com>
Co-authored-by: George Herbert <github@georgeherby.dev>
Co-authored-by: Adam Cowley <adam@adamcowley.co.uk>
Co-authored-by: Eugene Yurtsev <eugene@langchain.dev>
Co-authored-by: Mohamed Belhadj <medbelh@gmail.com>
Co-authored-by: Michael Watts <hello@michaelwatts.co>
Co-authored-by: William FH <13333726+hinthornw@users.noreply.github.com>
Co-authored-by: LukeCali-949 <76727497+LukeCali-949@users.noreply.github.com>
Co-authored-by: Allen Firstenberg <github@addventure.com>
Co-authored-by: Miguel Fernández <miguel@mff.io>
Co-authored-by: Darren Govoni <darren@cyberchain.ai>
Co-authored-by: Danish J <danish.jaffer@utexas.edu>
Co-authored-by: Oguz Vuruskaner <ovuruska@outlook.com>
Co-authored-by: TeCHiScy <741195+TeCHiScy@users.noreply.github.com>
Co-authored-by: AumeshMisra <38444310+AumeshMisra@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Jelf <okxiaoliang4@gmail.com>
Co-authored-by: Andrey <andrey-ne@mail.ru>
Co-authored-by: Oceansdeep7 <14208796@qq.com>
Co-authored-by: Sarfudheen JAINULABUDHEEN <52828445+sarfudheen@users.noreply.github.com>
Co-authored-by: Guillermo C. Martínez <telekosmos@ymail.com>
Co-authored-by: Guille <guillermo.c.martinez@dcsl.com>
Co-authored-by: Sharath <29162020+tnfssc@users.noreply.github.com>
Co-authored-by: Quinn <44334535+QuinnGT@users.noreply.github.com>
Co-authored-by: Evan <clcl6084@gmail.com>
Co-authored-by: LordMsz <33070601+LordMsz@users.noreply.github.com>
Co-authored-by: Kai Wang <kaiwdev@gmail.com>
@renovate renovate bot mentioned this pull request Jun 25, 2024
1 task
@Moe03
Copy link

Moe03 commented Jul 1, 2024

Hello @jacoblee93

Yes, I am talking about streamEvents, but I do not necessarily want to stream the tools, just the final answer.

Here is an example of what I am trying to do. I have a class with a run function that starts the process and sends event to my frontend application. I used to use a similar class in Python with an SQL Agent and Bedrock with Claude 3. I managed to send all the tokens to my frontend one by one (with the on_chat_model_stream event).

class AgentService {
  private agent

  constructor() {}

  async initialize() {
    if (this.agent) return

    const llm = new BedrockChat({
      model: 'anthropic.claude-3-sonnet-20240229-v1:0',
      region: 'us-west-2',
      streaming: true,
      temperature: 0,
    })

    const tools = [
      ... some DynamicStructuredTools ...
    ]

    const agent = createToolCallingAgent({
      llm,
      tools,
      prompt: customPrompt,
      streamRunnable: true,
    })

    const agentExecutor = new AgentExecutor({
      agent,
      tools,
      handleParsingErrors: true,
      returnIntermediateSteps: true,
    }).withConfig({ runName: 'llm_agent' })

    const messageHistory = (id: string) =>
      new DynamoDBChatMessageHistory({
        tableName: 'SessionTable',
        partitionKey: 'SessionId',
        sessionId: id,
        config: {
          region: 'us-west-2',
        },
      })

    const runnableWithMessageHistory = new RunnableWithMessageHistory({
      runnable: agentExecutor,
      getMessageHistory: (sessionId: string) => messageHistory(sessionId),
      inputMessagesKey: 'input',
      historyMessagesKey: 'history',
    })

    this.agent = runnableWithMessageHistory
  }

  async run(input: string, sessionId: string) {
    // [This is where an event is sent to the client to say that the process has started]

    const eventStream = await this.agent.streamEvents(
      { input: input },
      {
        configurable: {
          sessionId: sessionId,
        },
        version: 'v1',
      }
    )

    for await (const item of eventStream) {
      if (item.event === 'on_llm_stream') {
        // Here I send "item.data.chunk.content" to the client as a completion
        // But it sends everything in a single event, whereas I expect something like: Hello,| how |can |I |help |you ? where ‘|’ is an event
      }

      if (item.event === 'on_chain_end') {
       // [End event is sent]
      }
    }
  }
}

Would love to also stream the events properly since we have a setup for multiple llms, providers and its crucial the stream events is working across all these models, it currently doesn't work for claude-3.5-sonnet

@jacoblee93
Copy link
Collaborator

Thanks folks, we've implemented and merged in Python already. Will try to get to it this week, stay tuned!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto:improvement Medium size change to existing code to handle new use-cases size:XL This PR changes 500-999 lines, ignoring generated files.
Projects
None yet
6 participants