Pipeline Chat History

[proteusvacuum]

Description

This adds three different Chat History types to the pipelines:

• Node
• Named
• Global

Node history stores and includes the history for only that particular node. A Named history can be shared between nodes. Global history is the history for the full pipeline run (i.e. the input and output of the pipeline).

Global history uses the current ChatHistory model, and similarly will compress this history, using whatever LLM is associated with the node that is requesting that history to be inserted.
Note, if you have multiple LLMs with different allowed token lengths, I think you will need to set all of them to be the minimum of all the token lengths, otherwise you get errors.

For now, node and named history do not compress their history.

In follow up PRs I will work on ways of storing multiple compressed summaries so that the same history can be shared between nodes with different context lengths. At that time, I will also implement compression of the named and node history types.

User Impact

Allows actually chatting with a pipeline, rather than just sending a single message.

Demo

Screencast.from.2024-09-20.15-31-19.webm

Docs

[bderenzi]

Very cool. I think we also want no history, or perhaps it should be implemented as a max number of messages to keep (which could be zero). I think in the translation case you likely don't want much, if any, history as it's just x-shot prompting that unnecessarily gets expensive. I think you also wouldn't want to spend tokens on compression / summerization activities either in that case.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 99.12664% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
apps/chat/bots.py	0.00%	1 Missing ⚠️
apps/pipelines/nodes/nodes.py	98.00%	1 Missing ⚠️

Additional details and impacted files

📢 Thoughts on this report? Let us know!