PandasAI not working with LLM from HuggingFace via HuggingFaceTextGen method. Not able to Chat

[SimranAnand1]

Hi, I am facing issue: AttributeError: 'NoneType' object has no attribute 'type'

when I am trying to use a LLM other than BambooLLM. My LLM is pandasai_huggingface.huggingface_text_gen.HuggingFaceTextGen object

I have followed as given in the docs of PandasAI library:
https://docs.getpanda.ai/v3/large-language-models#huggingface-models

The repository has changed code significantly and I see that no LLms other than BambooLLM are present in llm folder now.. But earlier they were compatible. Please help me with a code so it would work for custom LLM (HuggingFacetextGen model). I have tried a lot to make it work but the LLM is always getting identified as None object only.

Thank you in advance! :)

[SimranAnand1]

Always getting this error as response:

raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

No response returned by chat function of LLM

[gventuri]

Hi @SimranAnand1, seems to be a bug about the latest version, let us investigate further.

@ArslanSaleem maybe you could check it out?

For the time being, I recommend using the 2.4, as we take care of the outstanding issue!

[SimranAnand1]

Hey @gventuri , thanks for letting me know.

Since HuggingFaceTextGen.call() returns a string (as mentioned in https://github.com/sinaptik-ai/pandas-ai/blob/f6673670ddde7c0e2531f0a103e5dd7d73330b7d/extensions/llms/huggingface/pandasai_huggingface/huggingface_text_gen.py) , I think we should not attempt res.json() inside the library.

The library should remove the res.json() line inside chat function of pandasai. Please let me know if HuggingFaceTextGen models work with PandasAI library or not. I even tried with PandasAI 2.4 library but the same error due to changes in Agent's base.py and chat method.

I found installations package from this page: https://github.com/sinaptik-ai/pandas-ai/tree/f6673670ddde7c0e2531f0a103e5dd7d73330b7d/extensions/llms/huggingface

Looking forward to the solution soon! Thanks :)

[SimranAnand1]

@ArslanSaleem Looking forward to a solution that could make PandasAI compatible to be used for HuggingFaceTextGen models

[SimranAnand1]

Hi Gabriel, thank you for your response. I tried with old version (2.4) as well but the git repository has new changes in the Agent's chat method and config due to which any LLM other than BamooLLM is not working with PandasAI. I would be grateful if you could please look into the same or share me guide to make HuggingFaceTextGen models work and be able to chat with few sample prompts. Thank you

…

On Sat, 1 Feb, 2025, 1:23 pm Gabriele Venturi, ***@***.***> wrote: Hi @SimranAnand1 <https://github.com/SimranAnand1>, seems to be a bug about the latest version, let us investigate further. @ArslanSaleem <https://github.com/ArslanSaleem> maybe you could check it out? For the time being, I recommend using the 2.4, as we take care of the outstanding issue! — Reply to this email directly, view it on GitHub <#1574 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQCQPGP6NJD5KSFDKPJLTND2NR4O5AVCNFSM6AAAAABWIABGCWVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMBSGQ2DOMQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[SimranAnand1]

I would suggest the llm class of PandasAI needs modification to incorporate other LLM instances such as HuggingFaceTextGen.