Expanding OpenAi context

[BioStarr]

Hello, I made a model of openai using langchain that will communicate with my documents, however I have one question, I would like for the model to behave as such that if it does not find the question in documents to consult its general knowledge, what do I mean by this?

Lets say I have two models, one is chat model, another uses vector store and reads from files.

I ask basic chat model what is the main city of Germany and it answers Berlin.
I ask model that is based on documentation same question, it checks documentation sees it does not have information then it should act like basic chat model and find the answer using its own set of tools. But all i get is the answer is not in provided context.

This is the code i use

  vector_store = FAISS.from_texts(documents, embedding=embeddings)
  llm = ChatOpenAI(temperature=0.3, max_tokens=1000, model_name="gpt-4-1106-preview")
  qa = ConversationalRetrievalChain.from_llm(llm=llm,retriever=vector_store.as_retriever())

[eyurtsev]

You could try to create an agent that has access to several tools one of them is a RAG chain. Or you could try doing RAG first and it can't answer the question, then forward the question to an agent that has other tools but not the vectorstore.