Home

DARPA-ASIST graph environment for reinforcement learning

Overview

This environment is a low-fidelity abstraction of the video game Minecraft, built using graphs and Python data structures. It is currently being used to run reinforcement learning (RL) experiments. We are trying to build models that can closely represent the observed behaviors and mental states of human agents when they play the search & rescue (SAR) scenario in Minecraft.

The overarching objective is to advance research in human-robot teamwork by simulating a cognitive skill called Theory of Mind (ToM). Our approach is to build computational models of ToM using reinforcement learning. Because Minecraft is a complex environment (i.e. graphics, many different kinds of objects), we propose the graph environment as a much simpler space for RL experimentation. After training policies in the graph, we aim to go back to Minecraft and hook those policies up to an AI-controlled player in Minecraft. Here our assumption is that the "best" policy that we train in the graph would translate over to Minecraft and be a "best" policy there too, without having to run RL experiments directly in Minecraft.

Project Malmo, developed at Microsoft Research, provides us with all the tools we need to eventually transition from a graph environment to a Minecraft environment. Additionally, Project Malmo provides tools within Minecraft that helps for user studies and for collecting data on the human decision-making process when they play the SAR scenario.

Read more about how Minecraft is used to build SAR scenarios and experiment with AI agents here.

Documentation

The code for this project lives in AsistEnv/asist_env. Install our required packages using pip:

pip install -r requirements.txt

See the following pages for more info on each aspect of the project:

1. Graph
2. Reinforcement learning
3. RL with Shiva
4. RL experiment log
5. User studies