Home

Overview

The goal of this repository is to show how Reinforcement Learning (RL) can be used within Webots to train agents to solve a given task.

The code presented in this repository was adapted from a semester project at the Distributed Intelligent Systems and Algorithms Laboratory (DISAL). A summary of the project can be found here.

The part presented here is about Deep Reinforcement Learning, leveraging Policy Gradient to train a fully connected, feed forward neural network to solve a simple obstacle avoidance task on a continuous action space with a Khepera IV robot.

The project was entirely implemented in C++ and relies on libtorch, PyTorch's C++ API.

Reinforcement Learning

The algorithm used here is Vanilla Policy Gradient (VPG). The implementation was inspired from OpenAI's Spinning up website, which provides comprehensive introductory material to the field of Deep Reinforcement Learning as well as numerous references to relevant papers.

Task

In this project, a Khepera IV robot is trained to learn obstacle avoidance. A neural network is given the state of the robot, namely the reading of the Infrared Sensors surrounding the robot as well as the current motor speeds, and provides direct speed commands to control the robot.