Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix incorrect Agent json function descriptions #2465

Closed
wants to merge 1 commit into from

Conversation

phazei
Copy link
Contributor

@phazei phazei commented Oct 13, 2024

Pull Request Type

  • ✨ feat
  • [ x] πŸ› fix
  • ♻️ refactor
  • πŸ’„ style
  • πŸ”¨ chore
  • πŸ“ docs

Relevant Issues

Resolves:

resolves #2432

What is in this change?

The example agent JSON query doesn't align with the description "All JSON responses should have two keys." "name" "arguments"

Previous agent example:

Query: "Save that to memory please."
JSON: {
    "action": "store",
    "content": "<insert summary of conversation until now>"
}

This PR fixes that:

Corrected agent example:

Query: "Save that to memory please."
JSON: {
  "name": "rag-memory",
  "arguments": {
    "action": "store",
    "content": "<insert summary of conversation until now>"
  }
}

Larger, smarter models can understand the inaccuracy in the description and correct for it, but smaller models follow the examples which are incorrect.

Developer Validations

  • I ran yarn lint from the root of the repo & committed changes
    yarn lint doesn't work and looks for directories that don't exist.

  • [ x] Relevant documentation has been updated
    none needed

  • [ x] I have tested my code functionality
    ran it locally and looked at generations, also created a test script not included in this pr.

  • [ x] Docker build succeeds locally

@phazei phazei changed the title Fix incorrect json API description. Fix incorrect Agent json function descriptions Oct 13, 2024
@timothycarambat
Copy link
Member

@phazei Great find, should have been done ages ago and can verify this dramatically improves smaller model calling for functions!

Have to make some minor edits so this is moved to #2513

(you still get credit, GH just makes new PRs when i make changes I still dont know why)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG]: Agent system messages isn't very clear.
2 participants