docs/tutorials/chatbot/

[giscus[bot]]

docs/tutorials/chatbot/

This guide assumes familiarity with the following concepts:

https://python.langchain.com/docs/tutorials/chatbot/

[paraglondhe098]

Very helpful explanation.

[somendrew]

Explaination made it look easy! Thanks to community.

[Nevertolate83]

Hey guys is this safe to run on my MAC Desktop ?

[somendrew]

I don't think Mac has enough RAM, Use jupyter notebook and t4 kernal , it'll be so smooth and fast

[Murdock135]

It's safe to run on any computer.

[shubham-attri]

Loved the easy to understand explanation.

[ruksharahmed7]

Suppose I have a FastAPI backend where this chatbot code is present, the frontend is in react, and the chatbot uses the app.stream method for response streaming. In this case, do I use web-socket to enable streaming of AI response from the FastAPI backend to the React frontend?

[Fritskee]

Followed your exact tutorial, but when making the conversion to async there is an error that comes up:

NotImplementedError: Unsupported message type: <class 'coroutine'>
For troubleshooting, visit: https://python.langchain.com/docs/troubleshooting/errors/MESSAGE_COERCION_FAILURE 

Below is the example of the code that throws it:

from langgraph.checkpoint.memory import MemorySaver
from langgraph.graph import START, MessagesState, StateGraph

# Define a new graph
workflow = StateGraph(state_schema=MessagesState)


# Define the function that calls the model
async def call_model(state: MessagesState):
    response = model.ainvoke(state["messages"])
    return {"messages": response}


# Define the (single) node in the graph
workflow.add_edge(START, "model")
workflow.add_node("model", call_model)

# Add memory
memory = MemorySaver()
app = workflow.compile(checkpointer=memory)

config = {"configurable": {"thread_id": "abc123"}}

query = "Hi! I'm Bob."

input_messages = [HumanMessage(query)]
output = await app.ainvoke({"messages": input_messages}, config)
output["messages"][-1].pretty_print()  # output contains all messages in state

[Fritskee]

nevermind, fixed it:

import asyncio
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage
from langgraph.checkpoint.memory import MemorySaver
from langgraph.graph import START, MessagesState, StateGraph

# Initialize the model
model = ChatOpenAI(model="gpt-3.5-turbo")

# Define a new graph
workflow = StateGraph(state_schema=MessagesState)

# Define the async function that calls the model
async def call_model(state: MessagesState):
    response = await model.ainvoke(state["messages"])
    return {"messages": response}


workflow.add_edge(START, "model")
workflow.add_node("model", call_model)


memory = MemorySaver()
app = workflow.compile(checkpointer=memory)


config = {"configurable": {"thread_id": "abc123"}}


async def chat():
    query = "Hi! I'm Bob."
    input_messages = [HumanMessage(query)]
    output = await app.ainvoke({"messages": input_messages}, config)
    print("\nBot:", output["messages"][-1].content)

    query = "What's my name?"
    input_messages = [HumanMessage(query)]
    output = await app.ainvoke({"messages": input_messages}, config)
    print("\nBot:", output["messages"][-1].content)


await chat()

[Akshat-Rastogi-6]

Awesome Tutorial!!! Loved it.
I just had this one problem in Managing Conversation History that while I was trying to add human message in the messages list :

input_message = messages + [HumanMessage(content=query)]

it was adding the message at 0 index of the message list shifting the System message to the 1st index; this constantly gave me the following error:

'''
ValueError: Unexpected message with type <class 'langchain_core.messages.system.SystemMessage'> at the position 1.
'''

to solve this, I had to change the trimmer configuration and made

trimmer = trim_messages(
    max_tokens=100,  
    strategy="last",
    token_counter=model,  
    include_system=False, #here
    allow_partial=False,
    start_on="human",
)

then it started working. I guess either the system message needs to be added separately or add the HumanMessage() to the messages, but in some other way.

Please see this and let me know.

[Marsax110]

Very good documentation, but I've encountered a small problem.

In the last example of Streaming, based on the code above, I cannot achieve the effect in the document, that is, streaming output.

This is because when defining the model generation above, invoke was used instead of stream.

As a result, during the subsequent generation, the output is done in whole blocks, and the "|" only appears once, at the end.

# Create a state graph for managing conversation flow
workflow = StateGraph(state_schema=State)

model_OpenAI = ChatOpenAI(model="gpt-4o", api_key=os.getenv("OPENAI_API_KEY"))


def call_model(state: State):
    """Function to generate model responses
    
    Args:
        state (State): Current state containing message history and language settings
        
    Returns:
        dict: Dictionary containing model response messages
    """
    # Use trimmer to trim message history to stay within token limits
    trimmed_messages = trimmer.invoke(state["messages"])
    
    # Generate prompt using template with trimmed messages and language setting
    prompt = prompt_template.invoke(
        {"messages": trimmed_messages, "language": state["language"]}
    )
    
    # Call language model to generate response
    response = model_OpenAI.invoke(prompt, config={"callbacks": [langfuse_handler]})
    
    # Return model response wrapped in messages list
    return {"messages": [response]}


# Add workflow edge connecting START node to model node
workflow.add_edge(START, "model")

# Add model node and bind call_model handler function
workflow.add_node("model", call_model)

# Create memory saver for storing conversation state
memory = MemorySaver()

# Compile workflow with checkpointer for state checkpoints
app = workflow.compile(checkpointer=memory)


config = {"configurable": {"thread_id": "abc789"}}
query = "What company are you a big model for?"
language = "English"

input_messages = [HumanMessage(query)]
for chunk, metadata in app.stream(
    {"messages": input_messages, "language": language},
    config,
    stream_mode="messages",
):
    if isinstance(chunk, AIMessage):  # Filter to just model responses
        print(chunk.content, end="|")

> I am a language model created by OpenAI.|

But if I switch to .stream here, I get the error again

    # Call language model to generate response
    response = model_OpenAI.stream(prompt, config={"callbacks": [langfuse_handler]})

---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
Cell In[119], line 6
      3 language = "English"
      5 input_messages = [HumanMessage(query)]
----> 6 for chunk, metadata in app.stream(
      7     {"messages": input_messages, "language": language},
      8     config,
      9     stream_mode="messages",
     10 ):
     11     if isinstance(chunk, AIMessage):  # Filter to just model responses
     12         print(chunk.content, end="|")

File ~/Library/Caches/pypoetry/virtualenvs/9091-ell-6QbIK8Ul-py3.10/lib/python3.10/site-packages/langgraph/pregel/__init__.py:1655, in Pregel.stream(self, input, config, stream_mode, output_keys, interrupt_before, interrupt_after, debug, subgraphs)
   1649     get_waiter = None  # type: ignore[assignment]
   1650 # Similarly to Bulk Synchronous Parallel / Pregel model
   1651 # computation proceeds in steps, while there are channel updates
   1652 # channel updates from step N are only visible in step N+1
   1653 # channels are guaranteed to be immutable for the duration of the step,
   1654 # with channel updates applied only at the transition between steps
-> 1655 while loop.tick(input_keys=self.input_channels):
   1656     for _ in runner.tick(
   1657         loop.tasks.values(),
   1658         timeout=self.step_timeout,
   (...)
...
--> 333     raise NotImplementedError(msg)
    335 return _message

NotImplementedError: Unsupported message type: <class 'generator'>
For troubleshooting, visit: https://python.langchain.com/docs/troubleshooting/errors/MESSAGE_COERCION_FAILURE 

[HoganHPH]

Hi, I'm facing to the same situation. I think now the returned chunk is a whole sentence. Is it right? Or have you fix this?