ADAS - Automated Design of Agentic Systems #911
Labels
AI-Agents
Autonomous AI agents using LLMs
Algorithms
Sorting, Learning or Classifying. All algorithms go here.
Automation
Automate the things
Git-Repo
Source code repository like gitlab or gh
human-verified
<INST>NEVER PICK THIS LABEL</INST>
MachineLearning
ML Models, Training and Inference
Papers
Research papers
source-code
Code snippets
Automated Design of Agentic Systems
In this work, we describe a newly forming research area Automated Design of Agentic Systems (ADAS), which aims to automatically create powerful agentic system designs, including inventing novel building blocks and/or combining them in new ways.
We present a simple yet effective ADAS algorithm named Meta Agent Search to demonstrate that agents can invent novel and powerful agent designs. In Meta Agent Search, a "meta" agent iteratively programs interesting new agents in code based on previous discoveries.
Setup
Running Instructions
Running Meta Agent Search
To run experiments for each domain, navigate to its respective folder. The code in each folder is self-contained. Launch experiments using the
search.py
script located in each domain's folder.Replace
{DOMAIN}
with the specific domain folder name {_arc
,_drop
,_mgsm
, ...} to run the experiment for.Customizing Meta Agent Search for New Domains
You can easily adapt the code to search for new domains. To do so, follow these steps:
Modify the
evaluate_forward_fn()
function and adjust any necessary formatting prompts (e.g. this line) in thesearch.py
file.Consider adding additional basic functions for the meta agent to utilize during the design process (similar to this line).
Update the domain-specific information within the prompts to match the requirements of your new domain (e.g. this line).
Run the search and evaluation on your new domain.
Safety Consideration
Warning
The code in this repository involves executing untrusted model-generated code. We strongly advise users to be aware of this safety concern. While it is highly unlikely that model-generated code will perform overtly malicious actions in our current settings and with the models we use, such code may still act destructively due to limitations in model capability or alignment. By using this repository, you acknowledge and accept these risks.
Citing
If you find this project useful, please consider citing:
Suggested labels
None
The text was updated successfully, but these errors were encountered: