You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently the Groq client in the src/llm/groq_client.py doesnt have any error handling , while such code might be okay for other APIs which have bigger token limit but Groq Llama3 70b has a rate limit of 6k tokens PER MINUTE, which get reached very easily by devika
and as soon as that limit is reached , error pops up in the devika backend console and the current task gets stopped.
How To Reproduce
Steps to reproduce the behavior (example):
Setup Devika with GROQ API
Select Llama3 70B as a model
Give a big problem that requires longer token limit (tbh any complex task will do)
Expected behavior
If the rate limit is reached for that minute , the agent should take a pause and then resume the task instead of abruptly stopping everything
Describe your issue
Currently the Groq client in the
src/llm/groq_client.py
doesnt have any error handling , while such code might be okay for other APIs which have bigger token limit but Groq Llama3 70b has a rate limit of 6k tokens PER MINUTE, which get reached very easily by devikaand as soon as that limit is reached , error pops up in the devika backend console and the current task gets stopped.
How To Reproduce
Steps to reproduce the behavior (example):
Expected behavior
If the rate limit is reached for that minute , the agent should take a pause and then resume the task instead of abruptly stopping everything
Screenshots and logs
Configuration
The text was updated successfully, but these errors were encountered: