Data Size restrictions for data frame

[kesavaraju-anurag]

Hello Contributors and Developers of Pandas AI,

This is a really cool application of an LLM.

I'm a newbie and'd like to know about the restrictions of the Pandas data frame on which the LLM runs the prompt.
Please suggest!!

Also, it would be great if we can suggest ways to implement custom-trained LLMs.

Thank you in advance!!

[gventuri]

Hey @kesavaraju-anurag, thanks a lot for reaching out!
What do you mean by restrictions? If you mean the size of the dataset, it should (in theory) work with any datasets of any size, as we don't pass the dataset to the LLM!

[adrianogf]

Hi gventuri. I'm trying to understand what information is passed to LLM? Embeddings are generated? Is there any way to know how many tokens were used in the prompt?