This repository contains a GitHub Action that enables fully automated testing inside of a CARLA based simulation environment. The action leverages CARLOS, an open, modular and scalable simulation architecture, which is presented in our paper. Besides the action itself, this repository also contains detailed instructions on the usage and configuration.
Important
This repository is open-sourced and maintained by the Institute for Automotive Engineering (ika) at RWTH Aachen University.
Simulation, Containerization and DevOps for Automated Driving are some of many research topics within our Vehicle Intelligence & Automated Driving domain.
If you would like to learn more about how we can support your DevOps or automated driving efforts, feel free to reach out to us!
Timo Woopen - Manager Research Area Vehicle Intelligence & Automated Driving
+49 241 80 23549
timo.woopen@ika.rwth-aachen.de
CARLOS: An Open, Modular, and Scalable Simulation Framework for the Development and Testing of Software for C-ITS
Christian Geller, Benedikt Haas, Amarin Kloeker, Jona Hermens, Bastian Lampe, Lutz Eckstein Institute for Automotive Engineering (ika), RWTH Aachen University
Abstract – Future mobility systems and their components are increasingly defined by their software. The complexity of these cooperative intelligent transport systems (C-ITS) and the ever-changing requirements posed at the software require continual software updates. The dynamic nature of the system and the practically innumerable scenarios in which different software components work together necessitate efficient and automated development and testing procedures that use simulations as one core methodology. The availability of such simulation architectures is a common interest among many stakeholders, especially in the field of automated driving. That is why we propose CARLOS - an open, modular, and scalable simulation framework for the development and testing of software in C-ITS that leverages the rich CARLA and ROS ecosystems. We provide core building blocks for this framework and explain how it can be used and extended by the community. Its architecture builds upon modern microservice and DevOps principles such as containerization and continuous integration. In our paper, we motivate the architecture by describing important design principles and showcasing three major use cases - software prototyping, data-driven development, and automated testing. We make CARLOS and example implementations of the three use cases publicly available at https://github.com/ika-rwth-aachen/carlos.
The CARLOS CI Action provides a fully automated way to integrate simulation testing into your GitHub Actions workflows.
The action performs the following steps:
- (optional) Download Composefile as a template from a separate GitHub repository
- Render simulation environment Composefile by merging local (and optionally remote) templates
- Deploy simulation environment via Docker Compose
- Render Scenario Runner Composefile
- Deploy Scenario Runner via Docker Compose and execute scenario
- Tear down simulation environment
Important
Since this action utilizes resource-heavy tools like the CARLA simulator and requires the environment to persist between the action steps, it cannot be used with GitHub-hosted runners.
To use this action, a self-hosted runner is a necessity. For details on how to install and configure such a runner, please refer to the official documentation. Additionally, the machines hosting the runners need to fulfill the following requirements:
- Meets CARLA system requirements (e.g. 6-8 GB GPU)
- Working installation of Docker and Docker Compose (installation of Docker Desktop includes both)
- Docker has access to GPU on host via vendor-specific tools (Nvidia Container Toolkit or ROCm)
- User that started the runner (either directly or as a service) needs to be in the docker group (see official documentation for details)
- (Optional) If graphical output is required, the runner needs to have access to the display. For details, please refer to this explanation.
This section demonstrates some of the common usecases for this action. Besides an explanation on how to approach these usecases, a reference implementation is also provided for each.
Assume you want to run a single scenario in a simulation with graphical output, so that you could have visual feedback on what is happening during scenario execution. Achieving this in a simulation environment following the CARLOS framework is incredibly simple with this action. First, you need to prepare a simple Composefile to define your simulation environment:
my-composefile.yml
name: carlos-ci-action
services:
carla-simulator:
deploy: # Needed for container access to Nvidia GPU
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
privileged: True
environment:
DISPLAY: $DISPLAY # Needed for container access to display
volumes:
- /tmp/.X11-unix:/tmp/.X11-unix # Needed for container access to display
image: rwthika/carla-simulator:server
command: bash -ic './CarlaUE4.sh -nosound'The Composefile shown only specifies a single service that will be started as part of the simulation environment, the simulator itself. This specification can now be used when integrating this action:
Example Workflow
name: example-workflow
on: push
jobs:
simulation-scenario-test:
runs-on: [self-hosted] # Select self-hosted runner via tags
name: Run simulation test
steps:
- shell: bash
run: | # Allow containers access to display (UNSAFE FOR PRODUCTION!)
xhost +local:
- uses: ika-rwth-aachen/carlos-ci-action
with:
composefile-path: my-composefile.yml # Path to environment Composefile
scenario-folder-path: scenarios/ # Path to scenarios
scenario-file-name: town10.xosc # Filename of scenario
- shell: bash
run: | # Block access to display again
xhost -local:For this usecase, assume you want to execute multiple scenarios which are placed in a common folder. Additionally, the simulation should now have no graphical output to lower resource consumption and speed up the entire process. You can make a slight modification to the environment Composefile from before:
my-composefile.yml
name: carlos-ci-action
services:
carla-simulator:
deploy: # Needed for container access to Nvidia GPU
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
privileged: True
image: rwthika/carla-simulator:server
command: bash -ic './CarlaUE4.sh -nosound -RenderOffScreen' # New flag to avoid graphical outputJust like before, the Composefile includes just the simulator as its only service. To dynamically create CI jobs for each of the scenarios contained in a specified folder, you could use the custom generate-job-matrix action we provided in the CARLOS repository. A workflow using that action could look like this:
Example Workflow
name: example-workflow
on: push
jobs:
generate-scenario-job-matrix:
runs-on: [self-hosted] # Hosted runners are also possible here
name: Generate scenario job matrix
outputs:
matrix: ${{ steps.generate.outputs.matrix }}
steps:
- uses: actions/checkout@v4
- id: generate
uses: ./.github/actions/generate-job-matrix # Path to custom action depends on your setup
with:
starting-point: ./scenarios # Folder where recursive search starts
query-string: '*.xosc' # Pattern that dynamically generates a job for each hit
exclude-string: '*/catalogs/*' # Patterns that should not be considered for job creation
simulation-scenario-tests:
needs: generate-scenario-job-matrix
runs-on: [self-hosted]
name: Run ${{ matrix.filename }} # Dynamic name generation for each resulting job
strategy:
fail-fast: false # Run all scenarios even when one fails
matrix: ${{ fromJson(needs.generate-scenario-job-matrix.outputs.matrix) }}
steps:
- uses: ika-rwth-aachen/carlos-ci-action
with:
composefile-path: my-composefile.yml
scenario-folder-path: ${{ matrix.filedir }} # Individual values for each dynamic job
scenario-file-name: ${{ matrix.filename }}The last usecase utilizes the action's capability to use environment Composefiles stored in a separate repository and merge that with your custom Composefile. This could be used to extract commonly used parts to a simulation environment so that you would only need to include it and override or extend where needed. Assume you have stored the Composefile from before in a different repository and now want to include and modify it for your pipeline. You could write a Composefile like this:
my-modifications.yml
services:
carla-simulator: # Modify existing services
privileged: True
environment: # Add new configuration
EXPERIMENTAL_FEATURES: enabled
image: rwthika/carla-simulator:experimental # Override existing configuration
carla-ros-bridge: # Extend with additional services
image: rwthika/carla-ros-bridge
depends_on:
carla-simulator:
condition: service_healthy
volumes:
- ./launch/demo.launch.py:/demo.launch.py
command: bash -ic "ros2 launch /demo.launch.py"To access remote repositories and the Composefiles contained within, you need to make sure you are authorized to do so. One possibility is to set up a Deploy key in the remote repository which you then set as a Secret in your repository. The resulting workflow could look like this:
Example Workflow
name: example-workflow
on: push
jobs:
simulation-scenario-test:
runs-on: [self-hosted]
name: Run simulation test
steps:
- uses: ika-rwth-aachen/carlos-ci-action
with:
composefile-path: my-modifications.yml # Specify local file
remote-repository: BenediktHaas96/testing # Specify remote repository
remote-deploykey: ${{ secrets.SSH_PRIVKEY }} # Use secret for authentication
remote-composefile: remotefolder/my-composefile.yml
scenario-folder-path: scenarios/
scenario-file-name: town10.xosc-
composefile-path(default:${{ github.action_path }}/templates/carla-simulator.yml)Path to Composefile that will be used directly or via merge with a remote file for the deployment
-
remote-repositoryOptional GitHub repository containing Composefiles ( org1/repo1 => github.com/org1/repo1 )
-
remote-composefilePath to the Composefile from the remote repository which should be downloaded
-
remote-path(default:./rendercompose-remote/)Path relative to working directory where the remote files will be placed
-
remote-deploykeyDeploy key that will be used to access the remote repository
-
rendered-composefile(default:./simulation-environment.yml)Path where the rendered Composefile should be placed
-
sim-startup-delaysecs(default:10)Seconds to wait after starting environment and before scenario execution
-
scenario-runner-image(default:rwthika/carla-scenario-runner:latest)Docker image of the CARLA scenario runner
-
scenario-folder-path(required)Path to folder containing the scenario(s) and optional catalogs subfolder
-
scenario-file-name(required)Filename of scenario
-
carla-hostname(default:carla-simulator)Hostname of CARLA simulator that the scenario runner should connect to
-
docker-network(default:carlos-ci-action_default)Docker network that the scenario runner should attach to
We hope that our simulation framework CARLOS and this action can help your research. If this is the case, please cite it using the following metadata.
@inproceedings{CARLOS24,
author = {Geller, Christian and Haas, Benedikt and Kloeker, Amarin and Hermens, Jona and Lampe, Bastian and Eckstein, Lutz},
title = {{CARLOS: An Open, Modular, and Scalable Simulation Framework for the Development and Testing of Software for C-ITS}},
url = {http://arxiv.org/abs/2404.01836},
year = {2024}
}
This research is accomplished within the project AUTOtech.agil (FKZ 01IS22088A). We acknowledge the financial support for the project by the Federal Ministry of Education and Research of Germany (BMBF).