Skip to content

Commit

Permalink
fix(assistant): summary now can output 2000 tokens (#2440)
Browse files Browse the repository at this point in the history
This pull request increases the token limit for the summary output to
2000 in order to accommodate larger documents.

<!--
ELLIPSIS_HIDDEN
-->


----

| <a href="https://ellipsis.dev" target="_blank"><img
src="https://avatars.githubusercontent.com/u/80834858?s=400&u=31e596315b0d8f7465b3ee670f25cea677299c96&v=4"
alt="Ellipsis" width="30px" height="30px"/></a> | 🚀 This PR
description was created by [Ellipsis](https://www.ellipsis.dev) for
commit 4412ecd. |
|--------|--------|

### Summary:
This PR increases the token limit for the summary output to 2000 in the
`SummaryAssistant` class to accommodate larger documents.

**Key points**:
- Increased `max_tokens` parameter in `ChatLiteLLM` instance to 2000 in
`process_assistant` function of `SummaryAssistant` class in
`/backend/modules/assistant/ito/summary.py` file.


----
Generated with ❤️ by [ellipsis.dev](https://www.ellipsis.dev)



<!--
ELLIPSIS_HIDDEN
-->
  • Loading branch information
StanGirard authored Apr 17, 2024
1 parent 3400b86 commit 1bd7e81
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion backend/modules/assistant/ito/summary.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ async def process_assistant(self):

data = loader.load()

llm = ChatLiteLLM(model="gpt-3.5-turbo")
llm = ChatLiteLLM(model="gpt-3.5-turbo", max_tokens=2000)

map_template = """The following is one document to summarize that has been split into multiple sections:
{docs}
Expand Down

0 comments on commit 1bd7e81

Please sign in to comment.