You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Enhance the _call_pipeline method to support batched evaluation. Instead of processing the whole dataset at once, the method should allow to group dataset items into batches and process them sequentially.
Motivation
APIs like e.g. Azure OpenAI imposes rate limits on the number of requests a user or client can make within a specified period. Currently, _call_pipeline processes the whole dataset at once, which can quickly exceed these limits.
Additional context
No response
The text was updated successfully, but these errors were encountered:
Feature description
Enhance the
_call_pipeline
method to support batched evaluation. Instead of processing the whole dataset at once, the method should allow to group dataset items into batches and process them sequentially.Motivation
APIs like e.g. Azure OpenAI imposes rate limits on the number of requests a user or client can make within a specified period. Currently,
_call_pipeline
processes the whole dataset at once, which can quickly exceed these limits.Additional context
No response
The text was updated successfully, but these errors were encountered: