Can't make the chat to understand previous context

[deepak-hl]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

Below is my Code for Memory Implementation


UPSTASH_REDIS_REST_URL = "https://us1-*************"
UPSTASH_REDIS_REST_TOKEN ="**********************************"

history = UpstashRedisChatMessageHistory(
    url=UPSTASH_REDIS_REST_URL, token=UPSTASH_REDIS_REST_TOKEN, ttl=0, session_id="chat4"
)
def set_custom_prompt_condense():
    _template = """Given the following conversation and a follow up question, return the conversation history excerpt that includes any relevant context to the question if it exists and rephrase the follow up question to be a standalone question.
    Chat History:
    {chat_history}
    Follow Up Input: {question}
    Standalone question:"""
    CONDENSE_QUESTION_PROMPT = PromptTemplate.from_template(_template)
    return CONDENSE_QUESTION_PROMPT

def retreival_qa_chain(COLLECTION_NAME,prompt):
    embedding = OpenAIEmbeddings()
    CONDENSE_QUESTION_PROMPT = set_custom_prompt_condense()
    llm = ChatOpenAI(model="gpt-3.5-turbo-16k",temperature=0.1)  
    vector_store = PGVector(
            connection_string=CONNECTION_STRING,
            collection_name=COLLECTION_NAME,
            embedding_function=embedding
        )
    retriever = vector_store.as_retriever(search_kwargs={"k": 3})
    memory = ConversationBufferMemory(memory_key="chat_history",return_messages=True,chat_memory=history,output_key='answer',)
    chain = ConversationalRetrievalChain.from_llm(
            llm=llm,
            chain_type="stuff",
            retriever=retriever,
            return_source_documents=True,
            combine_docs_chain_kwargs={"prompt": prompt},
            condense_question_prompt=CONDENSE_QUESTION_PROMPT,
            memory=memory,
            # verbose=True,
        )
    
    return chain

Description

The problem is that, with the standard Langchain chain, the chat is only able to reason from the results of the vector store similarity search. This introduces us to the following problem:

If I send a message to the AI introducing my friend Bob and say he is 28 years old, the bot will reason from the input message, maybe consider some vector store context (if there is any) and give us a response.

Then, if I ask the bot how old Bob is, it will be unable to answer "28 years old," because the bot will try to search for Bob in the vector store, but Bob was only mentioned in the message history.

How can I fix the above problem, as even after conversation is stored in chat_history, its not able to answer the question I asked from previos conversation .

System Info

langchain==0.1.1
pgvector
upstash_redis

[dosubot[bot]]

Based on the code you've shared and the issues you're facing, it seems like the chatbot is not able to recall the context from previous conversations. This could be due to the way the ConversationalRetrievalChain is handling the conversation history.

In your current implementation, you're using UpstashRedisChatMessageHistory to store the chat history. However, it's important to note that the chat history is only used within the scope of a single call to the chain, and is not preserved for future calls. This could be the reason why the bot is unable to recall information from previous interactions.

To resolve this issue, you can modify your ConversationalRetrievalChain to store and retrieve the chat history across multiple calls. This can be done by creating a new class that inherits from ConversationalRetrievalChain and overrides the relevant methods to implement your desired behavior.

Here is a basic example of how you could do this:

class CustomConversationalRetrievalChain(ConversationalRetrievalChain):
    def _call(
        self,
        inputs: Dict[str, Any],
        run_manager: Optional[CallbackManagerForChainRun] = None,
    ) -> Dict[str, Any]:
        # Your custom logic here

In this new class, you can add your logic to validate if retrieval of new documents is necessary and decide whether these documents should replace the new context or be appended to the old context.

Additionally, if you want to modify how the context is stored or updated, you could also consider creating a new class that inherits from ConversationBufferMemory and overrides the relevant methods.

Please note that this is a high-level suggestion and you might need to adjust the code according to your specific requirements. You can refer to the ConversationalRetrievalChain and ConversationBufferMemory classes in the LangChain repository for more details.

I hope this helps! If you have any further questions or need more clarification, feel free to ask.

Sources

• Why does my agent sometimes cannnot access its past chat context even after having ConversationalBufferWindowMemory?
• Getting error while Implementing Memory in ConversationalRetreival Chain
• Context retrieval
• Conversation memory example - previous messages not remembered
• libs/langchain/langchain/agents/conversational_chat/base.py
• libs/langchain/langchain/chains/conversational_retrieval/base.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.