Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ai: Improve prompt of workspace agent #14159

Conversation

lucas-koehler
Copy link
Contributor

What it does

Fixes #14128

Extend the prompt of the Theia AI workspace agent to get better answers and get the agent to base the answers on the actual workspace contents instead of assuming general workspace and file contents.

The added guidelines are based on the prompt of the default chat agent (in packages/ai-chat/src/common/default-chat-agent.ts) and adapted to focus on the current workspace.

I also tried adding example approaches like the following. This did not significantly improve results for me but sometimes lead to the getFileContent function to be invoked with a plain README as file path. This leads to incorrect results. Thus, the examples were removed again.

Example approaches
## Example Approaches

Question: How do I build this?
Approach: Check the workspace files for files like package.json or a pom.xml or any other suitable configuration file.
Read the file contents and determine the answer based on this. Also check the workspace's README if present.

Question: What is this about?
Approach: Analyze the files in the workspace. Read the workspace's README file and other relevant files for additional information.

Question: Where is <some-concept> implemented?
Approach: Analyze the files in the workspace. Read file contents as needed to determine relevant code locations.

How to test

  • Open a workspace containing a software project
  • Ask questions about the project. Remember to start your question with @Workspace to force the agent. E.g.
    • @Workspace What is this project about?
    • @Workspace Which typescript / java version does the project build?
    • @Workspace How do I build this?
  • Verify that the answers make sense and that relevant files actually were read.

Review checklist

Reminder for reviewers

Fixes eclipse-theia#14128

Extend the prompt of the Theia AI workspace agent to get better answers
and get the agent to base the answers on the actual workspace contents
instead of assuming general workspace and file contents.
@lucas-koehler
Copy link
Contributor Author

@JonasHelming Here is the workspace prompt PR as discussed. I cannot directly assign you as a reviewer.

@JonasHelming JonasHelming merged commit 8ed5001 into eclipse-theia:master Sep 12, 2024
11 checks passed
@sgraband sgraband added this to the 1.54.0 milestone Sep 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

[Theia AI] Improve prompt of workspace agent (146)
3 participants