Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add high-level looping function based on generateWithOptionalTools. #25

Open
jemc opened this issue May 10, 2024 · 0 comments
Open

Add high-level looping function based on generateWithOptionalTools. #25

jemc opened this issue May 10, 2024 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@jemc
Copy link
Collaborator

jemc commented May 10, 2024

One common pattern for LLM apps is to keep calling generateWithOptionalTools until the LLM has decided it's used enough tools to finally generate a natural language output.

This pattern is subject to a halting problem (the LLM may never decide it is satisfied, and may not be making progress toward a discernable goal - it may even lose track of the goal over time).

However the pattern is common enough (especially in quick prototypes) that it's worth supporting a simple way of doing it. Our current code sample for this is kinda ugly and verbose, so we'd like to provide a high-level looping function in Kurt that can accept:

  • the standard options for generating (prompt, extraMessages, etc)
  • a map of named tool parameter schemas
  • a map of named tool functions that take those data types as inputs
  • some additional settings, including:
    • a maximum number of iterations to allow before calling a final time with no tools (to force natural language output)
    • perhaps a way to remove tools that have already been called? (or been called a certain number of times?)
    • perhaps some control over if/how/what messages get added into the extraMessages history
    • whatever other options make sense

The output should get streamed just like we do now with KurtStream. However we need some way to have multiple consecutive streams, because we'll get multiple finished messages for each stream. Maybe an AsyncIterable<KurtStream> is the right return type.

@jemc jemc added the enhancement New feature or request label May 10, 2024
@jemc jemc self-assigned this May 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant