RLtools: PX4 Module

⚠️ Note: This was tested with RLtools aecd4d9e6087abb4aa48e7a489b53d4061577032

This example aims at the Pixracer Pro which is a Pixhawk 6C compliant hardware implementation (have a look at ./px4_autopilot for the changes to the mro/pixracerpro board definition and compilation config).

Installation

The commands provided here assume that this repo is cloned as a submodule of RLtools (you can also have RLtools separately but need to adjust the paths in that case). If you haven't cloned RLtools already:

git clone https://github.com/rl-tools/rl-tools.git rl_tools
cd rl_tools
git submodule update --init --recursive embedded_platforms/px4

Docker Build Environment (Optional)

cd rl_tools/embedded_platforms/px4
docker run -it --rm -v $(cd .. && cd .. && pwd):/rl_tools -w /rl_tools/embedded_platforms/px4/px4_autopilot rltools/px4
make px4_fmu-v6c_default EXTERNAL_MODULES_LOCATION=$(cd .. && cd external_modules && pwd) RL_TOOLS_ROOT=$(cd ../../../ && pwd) -j8

You might have to make space (flash) for the module e.g. by disabling unnecessary modules in the px4config.default file of your board (e.g. ./px4_autopilot/boards/mro/pixracerpro/default.px4board):

CONFIG_DRIVERS_CAMERA_CAPTURE=n
CONFIG_DRIVERS_CAMERA_TRIGGER=n
CONFIG_COMMON_OPTICAL_FLOW=n
CONFIG_DRIVERS_UAVCAN=n
CONFIG_MODULES_CAMERA_FEEDBACK=n

Main CMakeList:

• Move the loading of external modules to after then "include(px4_add_library)"
• Change to c++17 in the main

Build with (after changing the path to wherever this repository is located)::

cd px4_autopilot
make mro_pixracerpro_default EXTERNAL_MODULES_LOCATION=$(cd .. && cd external_modules && pwd) RL_TOOLS_ROOT=$(cd ../../../ && pwd) -j8

Flash with:

make mro_pixracerpro_default upload

For other boards you can substitute mro_pixracerpro_default with the appropriate board name (e.g. px4_fmu-v6c_default) and conduct changes in the corresponding default.px4board file.

For the SITL simulation you can use the following form:

make px4_sitl gz_x500 EXTERNAL_MODULES_LOCATION=$(cd .. && cd external_modules && pwd) RL_TOOLS_ROOT=$(cd ../../../ && pwd)

HITL

1. Enable the simulation quadrotor (x) in QGroundControl
2. Disable the automatic connection to the Pixhawk in QGroundControl
3. Compile and update firmware make px4_fmu-v6c_default EXTERNAL_MODULES_LOCATION=$(cd .. && cd external_modules && pwd) RL_TOOLS_ROOT=$(cd ../../../ && pwd)
4. Compile gazebo sitl/hitl DONT_RUN=1 make px4_sitl_default gazebo-classic
5. Source gazebo stuff source Tools/simulation/gazebo-classic/setup_gazebo.bash $(pwd) $(pwd)/build/px4_sitl_default
6. Start gazebo (classic) gazebo Tools/simulation/gazebo-classic/sitl_gazebo-classic/worlds/hitl_iris.world
7. Start QGroundControl (connects over UDP to the forwarded MAVLINK)

Note: It is recommended to disable the landing detection (which might trigger in-flight when using the trained policy):

COM_DISARM_LAND=-1
COM_DISARM_PRFLT=-1

Note: It is necessary to disable the integral parts in the integrated controllers as they might accumulate in the background when the policy is active:

MC_***_I=0

To log high-frequency data required for sysid:

SDLOG_PROFILE=8

When switching to and from the policy all integral components of the default control stack should be disabled (otherwise they might saturate while the policy is in action and e.g. send maximum throttle when switching back)

Make sure that the PX4 IMU_GYRO_RATEMAX setting is matching the rate that is expected by the policy (400 Hz by default)

Note to activate the policy using a dead man's switch the aux passthrough should be activated in QGC.

Usage (Policy):

• The policy is triggered by the AUX1 channel, so you should map it in the QGroundControl to a trigger on you remote control. It is best to use a switch that automatically goes back, so you can test the policy for a split second (the controller will switch back to the original controller).
• You can test the switching (without a battery connected) by looking at the dmesg output in the QGroundControl console after pulling the trigger
• Make sure that you have a good position reference (e.g. GPS) because the policy will try to hover in place (where the trigger was activated)
• It is highly recommended to test your policies in the HITL simulation at first
• Make sure that the dynamics (parameters) of the simulation match the real drone!
• Flying is on your own risk!

Usage (Benchmark):

• Go into the QGroundControl console
• Start module: rl_tools_benchmark start
• Watch printed output: dmesg

Building the Docker Container

docker build -t rltools/px4 .

Reactivating

1. Ubuntu 24.04: Check out main to use the new ./Tools/setup/ubuntu.sh script.
2. git checkout v1.15.2
3. make px4_fmu-v6c_default to test if the compilation works
4. make px4_fmu-v6c_default EXTERNAL_MODULES_LOCATION=$(cd .. && cd external_modules && pwd) RL_TOOLS_ROOT=$(cd ../../../ && pwd) -j8

Software in the Loop (SITL)

GTK_PATH= make px4_sitl gz_x500 EXTERNAL_MODULES_LOCATION=$(cd .. && cd external_modules && pwd) RL_TOOLS_ROOT=$(cd ../../../ && pwd) -j8

Simulation in Hardware (SIH)

HITL but the dynamics simulation is running on the flight controller

• cd px4_autopilot

• Build and upload (mind to check the device path) make px4_fmu-v6c_default EXTERNAL_MODULES_LOCATION=$(cd .. && cd external_modules && pwd) RL_TOOLS_ROOT=$(cd ../../../ && pwd) -j8 upload SERIAL_PORT=/dev/ttyACM0

• Run mavproxy (because when using JMavSim as the relay the Mavlink console did not work): mavproxy.py --master=/dev/ttyACM0 --baud=921600 --out=udp:127.0.0.1:4560 --out=udp:127.0.0.1:14550

• Run JMavSim in simulation only mode (default UDP port: 4560): ./Tools/simulation/jmavsim/jmavsim_run.sh -q -u -o

• QGroundControl (set autoconnect to UDP only)

• Set the SIH_* dynamics parameters to match the model the policy was trained for (for the Crazyflie e.g. external_modules/src/modules/rl_tools_policy/crazyflie_sih.params)