Need help in Langchain Custom Agent with Local LLM (Open Source) to query pandas dataframe.

[ianuvrat]

Hi guys, after successfully trying out langchain pandas' agents with Open AI, my next aim is to use open source LLM to query database. So far I'm able to integrate Mistral 7B Instruct model with langchain , but I 'm not able to get final answer.

Initialize LLM

from langchain.llms import CTransformers
config = {'max_new_tokens': 1024, 'temperature': 0}
llm = CTransformers(model='TheBloke/Mistral-7B-Instruct-v0.1-GGUF',
                    model_file="mistral-7b-instruct-v0.1.Q4_K_M.gguf",
                    config=config)

Reading df

df = pd.read_csv("employee_data.csv") # load employee_data.csv as dataframe
python = PythonAstREPLTool(locals={"df": df}) # set access of python_repl tool to the dataframe

# create calculator tool
calculator = LLMMathChain.from_llm(llm=llm, verbose=True)

# create variables for f strings embedded in the prompts
user = 'Alexander Verdad' # set user
df_columns = df.columns.to_list() # print column names of df

Initializing tool (df as a tool)

tools = [
    Tool(
        name = "Employee Data",
        func=python.run,
        description = f"""
        Useful for when you need to answer questions about employee data stored in pandas dataframe 'df'.
        Run python pandas operations on 'df' to help you get the right answer.
        'df' has the following columns: {df_columns}

        <user>: How many Sick Leave do I have left?
        <assistant>: df[df['name'] == '{user}']['sick_leave']
        <assistant>: You have n sick leaves left.
        """
    ),
    Tool(
        name = "Calculator",
        func=calculator.run,
        description = f"""
        Useful when you need to do math operations or arithmetic.
        """
    )
]

prefix = f'You are friendly HR assistant. You are tasked to assist the current user: {user} on questions related to HR. You have access to the following tools: '
initialize the LLM agent

agent = initialize_agent(tools,
                         llm,
                         agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
                         verbose=True,
                        #  agent_kwargs=agent_kwargs
                         agent_kwargs = {"output_parser": parser,
                                      'prefix': prefix}
                         )

When I print the template of agent here is is what I get:
print(agent.agent.llm_chain.prompt.template)

You are friendly HR assistant. You are tasked to assist the current user: Alexander Verdad on questions related to HR. You have access to the following tools: 

Employee Data: 
        Useful for when you need to answer questions about employee data stored in pandas dataframe 'df'.
        Run python pandas operations on 'df' to help you get the right answer.
        'df' has the following columns: ['employee_id', 'name', 'position', 'organizational_unit', 'rank', 'hire_date', 'regularization_date', 'vacation_leave', 'sick_leave', 'basic_pay_in_php', 'employment_status', 'supervisor']

        <user>: How many Sick Leave do I have left?
        <assistant>: df[df['name'] == 'Alexander Verdad']['sick_leave']
        <assistant>: You have n sick leaves left.
        
Calculator: 
        Useful when you need to do math operations or arithmetic.
        

Use the following format:

Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [Employee Data, Calculator]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question

Begin!

Question: {input}
Thought:{agent_scratchpad}

Result
Now when run the agent ,

%%time
response = agent.run("how many vacation leaves are available for Alexander, give me the answer??")

This is what I get as answer:

I see Mistral do a modest attempt to query, but then stops there to fetch the result. Can someone please assist here. Here is my code. (I'm using Google Colab GPU free tier and quanitsed LLM model)

[dosubot[bot]]

🤖

Hi there,

Thanks for reaching out and for your interest in using LangChain with the Mistral 7B Instruct model. I see that you're trying to use the CTransformers class to initialize your LLM. However, based on the context provided, it seems like you might be missing a few steps in your process.

Firstly, the CTransformers class is initialized correctly in your code. However, to use it to query a pandas dataframe, you need to construct a prompt based on your dataframe and then use the generate method of the CTransformers instance to process the prompt. Here's an example:

from langchain.llms import CTransformers
import pandas as pd

# Initialize the dataframe
df = pd.DataFrame({
    'column1': ['data1', 'data2', 'data3'],
    'column2': ['data4', 'data5', 'data6']
})

# Initialize the CTransformers instance
llm = CTransformers(
    model='TheBloke/Mistral-7B-Instruct-v0.1-GGUF',
    model_file="mistral-7b-instruct-v0.1.Q4_K_M.gguf",
    config={'max_new_tokens': 1024, 'temperature': 0}
)

# Query the dataframe
for index, row in df.iterrows():
    prompt = f"Process the data: {row['column1']}, {row['column2']}"
    result = llm.generate(prompt)
    print(result)

In this example, the prompt is a string that you would need to construct based on the data in your dataframe. The llm.generate(prompt) line is where the model processes the prompt and returns the result.

Please note that the way you construct the prompt would depend on the specific task you want the model to perform.

Also, please note that the generate method is the correct method to use for generating responses from the model. The _call method is a private method (as indicated by the underscore prefix), which means it's not intended to be directly used outside of the class. It's generally recommended to use public methods for interacting with a class.

I hope this helps! If you have any other questions or run into any issues, feel free to ask.

Best,
Dosu

Sources

• libs/langchain/langchain/llms/openllm.py
• libs/langchain/langchain/llms/ctransformers.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[shivanshkaushikk]

Did you solve this? I'm stuck at a similar problem

[kariscourey]

Trying to recreate. What's your parser?

[akashAD98]

@ianuvrat have you solve the issue?

[ianuvrat]

Yes Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: Akash A Desai ***@***.***> Sent: Monday, January 22, 2024 5:57:14 PM To: langchain-ai/langchain ***@***.***> Cc: ianuvrat ***@***.***>; Mention ***@***.***> Subject: Re: [langchain-ai/langchain] Need help in Langchain Custom Agent with Local LLM (Open Source) to query pandas dataframe. (Discussion #11429) @ianuvrat<https://github.com/ianuvrat> have you solve the issue? — Reply to this email directly, view it on GitHub<#11429 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AQPPCDTCEW3ZHV3KIPEDEZDYPZLKFAVCNFSM6AAAAAA5UEULNGVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DEMBXHE4DS>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[akashAD98]

can you send the source code & how did you solved it?

[itsihsan]

Hi @ianuvrat, I am stuck on a similar issue. Can you please post the revised and refined working code with comments here?
thanks