How to answer prompt with the previous questions context, using langchain.

[Opperessor]

from langchain.chat_models import ChatOpenAI
from langchain.document_loaders import TextLoader
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import FAISS
from langchain.chains import ConversationalRetrievalChain, ConversationChain
from langchain.memory import ConversationBufferMemory
import os
from PyPDF2 import PdfReader
from langchain.memory import ConversationBufferWindowMemory
from langchain.text_splitter import CharacterTextSplitter

os.environ["OPENAI_API_KEY"] = "---"


txt_file_path = 'test_file.pdf'
loader = PdfReader(txt_file_path)
raw_text = ''
for i, page in enumerate(loader.pages):
    content = page.extract_text()
    if content:
        raw_text += content
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
data = text_splitter.split_text(raw_text)

embeddings = OpenAIEmbeddings()
vectorstore = FAISS.from_texts(data, embedding=embeddings)

#my fine tuned model
llm = ChatOpenAI(temperature=0.7, model_name="ft:gpt-3.5-turbo-1106:xxx.XXXX")

memory = ConversationBufferWindowMemory(k=10, return_messages=True,memory_key='chat_history',)

conversation_chain = ConversationalRetrievalChain.from_llm(
        llm=llm,
        chain_type="stuff",
        retriever=vectorstore.as_retriever(),
        memory=memory,
        # verbose=True
        )

while True:
        print("--------------------------------")
        query = input("Enter the question (type 'exit to end): ")
        print("--------------------------------")
        if query.lower() == "exit":
                print("Exiting.")
                break
        result = conversation_chain({"question": query})
        print(result["answer"])

what I want is I wanna upload a document to my fine-tuned llm using langchain, and ask questions related to it. I have used ConversationBufferWindowMemory and ConversationalRetrievalChain, but was unable to do it. It answers only if the question is specific but not when its not specified one.

For eg: If the file has content about 2 people(say Barack Obama and bill gates), if the

first question is : who is bill gates.

model: [answers]

second question: where was he born?

model: [model answers - Barack Obama's DOB].

what I want is at second time it should answer based on the previous questions context.

I did set the k value to 10 in ConversationBufferWindowMemory still unable to do it.
appreciate if someone helps me.

[dosubot[bot]]

🤖

Hi @Opperessor,

I'm Dosu, a friendly bot here to help you out while we're waiting for a human maintainer. I'll assist you with bug resolutions, further questions, and even guide you to contribute. Let's crack this issue together!

Based on the issues and solutions I found in the LangChain repository, it seems like you might need to modify the PROMPT template to include previous questions and their answers in the context. This can be done by adding a new input variable, say previous_qa, to the input_variables list and including it in the prompt_template.

Here's how you can do it:

prompt_template = """Use the following pieces of context and previous questions and answers to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.

{context}

Previous Q&A: {previous_qa}

Question: {question}
Helpful Answer:"""
PROMPT = PromptTemplate(
    template=prompt_template, input_variables=["context", "previous_qa", "question"]
)

In this modified template, previous_qa should be a string that contains the previous questions and their answers formatted in a way that the model can understand. You would need to ensure that this information is passed to the PROMPT when it is used. You can refer to the LangChain code for more details.

Regarding the ConversationalRetrievalChain, it handles context from previous questions and answers by maintaining a chat history and using it to generate a new standalone question for retrieval. The question_generator chain is used to generate a new question for retrieval. This chain takes in the current question and any chat history and produces a new standalone question to be used later on. If there is a chat history, it is passed along with the current question to the question_generator chain to generate a new question. If there is no chat history, the current question is used as is. You can refer to the LangChain code for more details.

I hope this helps! If you have any more questions or need further clarification, feel free to ask.

Sources

• DOC: Passing Context to Conversational Chain
• ConversationalRetrievalChain did not look into chat histroy while making an answer
• libs/langchain/langchain/chains/retrieval_qa/prompt.py
• libs/langchain/langchain/chains/conversational_retrieval/base.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[Opperessor]

@dosubot still the problem has not solved

[dosubot[bot]]

🤖

I'm here to help! Could you please provide me with more details about the problem you're encountering with LangChain?

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.