User study: AI to Human teaching

Research objective

The purpose of this research is to find a medium to transfer knowledge from reinforcement learning (RL) agents to humans. This medium should enable the latter to perform comparably with explanations given by other humans.

Asumptions

This research is done assuming that the following statements are true:

1. Humans are able to transfer knowledge to others about any computable domain using either natural language or a whiteboard.
2. Each tested medium can be used to represent the policy of any RL agents (An inefficient way is valid).
3. Each tested medium can be understood by most humans with a short explanation of their functioning.

This research will not attempt to prove those statements as it will ony be useful if the research results are positive.

Hypotheses

This research will evaluate the following hypotheses:

1. Some mediums are better than others when it comes to transfering knowledge to humans.
2. Providing Hierarchical Behavior Explanations as Graphs (HBEG) is comparable to an other human explanation on the following evaluation metrics and domains.

Evaluation Metrics

1. Best performance on an episode of the task
2. Real time speed of performance increase of the task
3. Ability to reproduce the task without any given explanation after a short time

Knowledge Mediums

• Nothing (Human must learn by reinforcment itself)
• Step instruction (Output of the policy)
• Other human written instructions
• Other human graphical instructions
• Hierarchical Behavior Explanations as Graphs (HBEG)

Domains

• The Crafting environment for arbitrary hierarchical tasks.
• The MineCrafting environment for MineCraft hierarchical tasks without the complex 3D navigation.
• The Minigrid environment for classic 2D navigation.

Installation

1. git clone this repository.

2. Initialize submodules:

git submodule update --init --recursive

3. Install requirements

pip install -r requirements.txt

4. Install domains of interest

Crafting

pip install -e .\crafting

Minigrid

pip install -e .\minigrid

Quickstart

Crafting

Manual run

python -m crafting.examples.minecraft

HippoGym run

Enter the submodule:

cd hippo_gym

Install requirements:

pip install -r requirements.txt

Launch the local hosted server in dev mode:

python -m App dev

Go to local hosted frontend in a browser: App or Debug.

MiniGrid

KeyDoor

python -m minigrid.manual_control --env MiniGrid-DoorKey-8x8-v0 --agent_view