InvalidRequestError Please reduce the length of message

[piyush-at-sangti]

I was running the Example jupyter notebook at https://github.com/microsoft/autogen/blob/main/notebook/agentchat_auto_feedback_from_code_execution.ipynb

# followup of the previous question
user_proxy.send(
    recipient=assistant,
    message="""Plot a chart of their stock price change YTD and save to stock_price_ytd.png.""",
)

While running the above cell, I got below error.

{
	"name": "InvalidRequestError",
	"message": "This model's maximum context length is 8192 tokens. However, your messages resulted in 11433 tokens. Please reduce the length of the messages.",
	"stack": "---------------------------------------------------------------------------
InvalidRequestError                       Traceback (most recent call last)
/workspaces/autogen/notebook/agentchat_auto_feedback_from_code_execution.ipynb Cell 12 line 2
      <a href='vscode-notebook-cell://codespaces%2Bhumble-space-guide-x5rv4gvwj5r26j67/workspaces/autogen/notebook/agentchat_auto_feedback_from_code_execution.ipynb#X13sdnNjb2RlLXJlbW90ZQ%3D%3D?line=0'>1</a> # followup of the previous question
----> <a href='vscode-notebook-cell://codespaces%2Bhumble-space-guide-x5rv4gvwj5r26j67/workspaces/autogen/notebook/agentchat_auto_feedback_from_code_execution.ipynb#X13sdnNjb2RlLXJlbW90ZQ%3D%3D?line=1'>2</a> user_proxy.send(
      <a href='vscode-notebook-cell://codespaces%2Bhumble-space-guide-x5rv4gvwj5r26j67/workspaces/autogen/notebook/agentchat_auto_feedback_from_code_execution.ipynb#X13sdnNjb2RlLXJlbW90ZQ%3D%3D?line=2'>3</a>     recipient=assistant,
      <a href='vscode-notebook-cell://codespaces%2Bhumble-space-guide-x5rv4gvwj5r26j67/workspaces/autogen/notebook/agentchat_auto_feedback_from_code_execution.ipynb#X13sdnNjb2RlLXJlbW90ZQ%3D%3D?line=3'>4</a>     message=\"\"\"Plot a chart of their stock price change YTD and save to stock_price_ytd.png.\"\"\",
      <a href='vscode-notebook-cell://codespaces%2Bhumble-space-guide-x5rv4gvwj5r26j67/workspaces/autogen/notebook/agentchat_auto_feedback_from_code_execution.ipynb#X13sdnNjb2RlLXJlbW90ZQ%3D%3D?line=4'>5</a> )

File /workspaces/autogen/autogen/agentchat/conversable_agent.py:334, in ConversableAgent.send(self, message, recipient, request_reply, silent)
    332 valid = self._append_oai_message(message, \"assistant\", recipient)
    333 if valid:
--> 334     recipient.receive(message, self, request_reply, silent)
    335 else:
    336     raise ValueError(
    337         \"Message can't be converted into a valid ChatCompletion message. Either content or function_call must be provided.\"
    338     )

File /workspaces/autogen/autogen/agentchat/conversable_agent.py:462, in ConversableAgent.receive(self, message, sender, request_reply, silent)
    460 if request_reply is False or request_reply is None and self.reply_at_receive[sender] is False:
    461     return
--> 462 reply = self.generate_reply(messages=self.chat_messages[sender], sender=sender)
    463 if reply is not None:
    464     self.send(reply, sender, silent=silent)

File /workspaces/autogen/autogen/agentchat/conversable_agent.py:781, in ConversableAgent.generate_reply(self, messages, sender, exclude)
    779     continue
    780 if self._match_trigger(reply_func_tuple[\"trigger\"], sender):
--> 781     final, reply = reply_func(self, messages=messages, sender=sender, config=reply_func_tuple[\"config\"])
    782     if final:
    783         return reply

File /workspaces/autogen/autogen/agentchat/conversable_agent.py:606, in ConversableAgent.generate_oai_reply(self, messages, sender, config)
    603     messages = self._oai_messages[sender]
    605 # TODO: #1143 handle token limit exceeded error
--> 606 response = oai.ChatCompletion.create(
    607     context=messages[-1].pop(\"context\", None), messages=self._oai_system_message + messages, **llm_config
    608 )
    609 return True, oai.ChatCompletion.extract_text_or_function_call(response)[0]

File /workspaces/autogen/autogen/oai/completion.py:799, in Completion.create(cls, context, use_cache, config_list, filter_func, raise_on_ratelimit_or_timeout, allow_format_str_template, **config)
    797     base_config[\"max_retry_period\"] = 0
    798 try:
--> 799     response = cls.create(
    800         context,
    801         use_cache,
    802         raise_on_ratelimit_or_timeout=i < last or raise_on_ratelimit_or_timeout,
    803         **base_config,
    804     )
    805     if response == -1:
    806         return response

File /workspaces/autogen/autogen/oai/completion.py:830, in Completion.create(cls, context, use_cache, config_list, filter_func, raise_on_ratelimit_or_timeout, allow_format_str_template, **config)
    828 with diskcache.Cache(cls.cache_path) as cls._cache:
    829     cls.set_cache(seed)
--> 830     return cls._get_response(params, raise_on_ratelimit_or_timeout=raise_on_ratelimit_or_timeout)

File /workspaces/autogen/autogen/oai/completion.py:220, in Completion._get_response(cls, config, raise_on_ratelimit_or_timeout, use_cache)
    218         response = openai_completion.create(**config)
    219     else:
--> 220         response = openai_completion.create(request_timeout=request_timeout, **config)
    221 except (
    222     ServiceUnavailableError,
    223     APIConnectionError,
    224 ):
    225     # transient error
    226     logger.info(f\"retrying in {retry_wait_time} seconds...\", exc_info=1)

File ~/.local/lib/python3.10/site-packages/openai/api_resources/chat_completion.py:25, in ChatCompletion.create(cls, *args, **kwargs)
     23 while True:
     24     try:
---> 25         return super().create(*args, **kwargs)
     26     except TryAgain as e:
     27         if timeout is not None and time.time() > start + timeout:

File ~/.local/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py:155, in EngineAPIResource.create(cls, api_key, api_base, api_type, request_id, api_version, organization, **params)
    129 @classmethod
    130 def create(
    131     cls,
   (...)
    138     **params,
    139 ):
    140     (
    141         deployment_id,
    142         engine,
   (...)
    152         api_key, api_base, api_type, api_version, organization, **params
    153     )
--> 155     response, _, api_key = requestor.request(
    156         \"post\",
    157         url,
    158         params=params,
    159         headers=headers,
    160         stream=stream,
    161         request_id=request_id,
    162         request_timeout=request_timeout,
    163     )
    165     if stream:
    166         # must be an iterator
    167         assert not isinstance(response, OpenAIResponse)

File ~/.local/lib/python3.10/site-packages/openai/api_requestor.py:299, in APIRequestor.request(self, method, url, params, headers, files, stream, request_id, request_timeout)
    278 def request(
    279     self,
    280     method,
   (...)
    287     request_timeout: Optional[Union[float, Tuple[float, float]]] = None,
    288 ) -> Tuple[Union[OpenAIResponse, Iterator[OpenAIResponse]], bool, str]:
    289     result = self.request_raw(
    290         method.lower(),
    291         url,
   (...)
    297         request_timeout=request_timeout,
    298     )
--> 299     resp, got_stream = self._interpret_response(result, stream)
    300     return resp, got_stream, self.api_key

File ~/.local/lib/python3.10/site-packages/openai/api_requestor.py:710, in APIRequestor._interpret_response(self, result, stream)
    702     return (
    703         self._interpret_response_line(
    704             line, result.status_code, result.headers, stream=True
    705         )
    706         for line in parse_stream(result.iter_lines())
    707     ), True
    708 else:
    709     return (
--> 710         self._interpret_response_line(
    711             result.content.decode(\"utf-8\"),
    712             result.status_code,
    713             result.headers,
    714             stream=False,
    715         ),
    716         False,
    717     )

File ~/.local/lib/python3.10/site-packages/openai/api_requestor.py:775, in APIRequestor._interpret_response_line(self, rbody, rcode, rheaders, stream)
    773 stream_error = stream and \"error\" in resp.data
    774 if stream_error or not 200 <= rcode < 300:
--> 775     raise self.handle_error_response(
    776         rbody, rcode, resp.data, rheaders, stream_error=stream_error
    777     )
    778 return resp

InvalidRequestError: This model's maximum context length is 8192 tokens. However, your messages resulted in 11433 tokens. Please reduce the length of the messages."
}

Autogen should automatically handle the the context length of the message. Right?

Tasks

Preview Give feedback

No tasks being tracked yet.

[sonichi]

#156

[AllanSchergerGitHub]

@sonichi @piyush-at-sangti This is a symptom of a bigger change that is needed. Autogen should never stop working due to errors. Agents should be spun up to do any activity that could potentially cause an error of any kind. The agents are designed to read the error messages and figure out what to do next (rewrite code and try again; report problem to user; etc). If Autogen is failing it is because it is doing too much in it's core program and not via an agent.

Let me know your thoughts.

[sonichi]

Sure. That should be the goal. The progress in #156 is encouraging. I wish it can be accelerated with your help :)

[AllanSchergerGitHub]

I’d like to help! Is 156 just focused on context length? Or, is there a place that is more focused on overall architecture? Thank you. On Oct 21, 2023, at 9:48 AM, Chi Wang ***@***.***> wrote:  Sure. That should be the goal. The progress in #156<#156> is encouraging. I wish it can be accelerated with your help :) — Reply to this email directly, view it on GitHub<#238 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABS7E4XRGPCQBXCIX3365D3YAPODZAVCNFSM6AAAAAA6ADNED2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZTHAYTMMBVGI>. You are receiving this because you commented.Message ID: ***@***.***>

[thinkall]

We are closing this issue due to inactivity; please reopen if the problem persists.