Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ros-llm-integration #2

Open
wants to merge 51 commits into
base: master
Choose a base branch
from
Open

Conversation

Kindred-Yi
Copy link

Updates:
add dexterity-interface repo to primitives, which we can use to interact with llm, once we get its response, the commands that given by llm will be saved in output.txt in this repo. Then when we launch llm-control.launch, it will automatically read output.txt and parse it to a series of different panda primitives that we already implemented, and execute these commands.
Problems:

  1. llm lack understanding of each panda primitives, for example if we ask LLM to generate commands for picking the kettle, it will output move first then pick, but out pick implementation already include the "move" part. Solution: maybe give llm more prompts of panda's current capability
  2. Consider include all the system in one repo and use one docker(chat.py, panda-primitives, panda-primitives-control)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants