Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Containerization Pipeline and Ubuntu 22.04 Upgrade #1

Closed
wants to merge 31 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
193d1f9
Create initial github workflow
cgeller Dec 11, 2023
82c1301
Update docker.yml
cgeller Dec 11, 2023
6fe7791
Fix runs-on
cgeller Dec 11, 2023
118cda3
Update docker.yml
cgeller Dec 11, 2023
803ea9b
Add Dockerfile
cgeller Dec 11, 2023
ee15399
Add entrypoint.sh
cgeller Dec 11, 2023
e8ba544
Update Dockerfile
cgeller Dec 11, 2023
b2336ff
Update requirements.txt
cgeller Dec 11, 2023
a32b829
Update requirements.txt
cgeller Dec 12, 2023
6b8ab4a
Update Dockerfile
cgeller Dec 12, 2023
915e5fc
update carla version
cgeller Dec 12, 2023
858a56e
Add Python API extraction to Dockerfile
cgeller Dec 12, 2023
b1b0914
Fix requirements.txt
cgeller Dec 12, 2023
36e1abf
Update Dockerfile
cgeller Dec 12, 2023
cf5c7a0
Update Dockerfile
cgeller Dec 12, 2023
91c92dd
Update Dockerfile
cgeller Dec 12, 2023
3460faa
Update Dockerfile
cgeller Dec 12, 2023
25c962a
Update Dockerfile
cgeller Dec 12, 2023
db9f7b3
add release download action
cgeller Dec 18, 2023
673cc6f
Merge branch 'docker' of github.com:cgeller/scenario_runner into docker
cgeller Dec 18, 2023
37db813
update Dockerfile for release processing
cgeller Dec 18, 2023
af50942
Extract PythonAPI
cgeller Dec 19, 2023
0a20177
Update docker.yml
cgeller Dec 19, 2023
a862007
Update docker.yml
cgeller Dec 19, 2023
55ce918
move artifacts to parent folder
cgeller Dec 19, 2023
7da3f5c
no cache
cgeller Dec 19, 2023
570dada
Update docker.yml
cgeller Dec 19, 2023
942ccf6
Update Dockerfile
cgeller Dec 19, 2023
ee394e3
Update file
cgeller Dec 19, 2023
a660837
Update docker.yml
cgeller Dec 19, 2023
9fd5cb5
Update Dockerfile
cgeller Dec 19, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: Docker

on: [push, pull_request]

jobs:

clean-up:
name: Clean up
runs-on: self-hosted
steps:

- name: Clean up
run: rm -rf *

create-image:
name: Create docker image
runs-on: self-hosted
steps:
- uses: docker/login-action@v3
name: Login to Docker Hub
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}

- uses: robinraju/release-downloader@v1.8
name: Download python-api
with:
repository: carla-compose/carla
latest: true
fileName: PythonAPI.tar.gz
out-file-path: artifacts

- name: Extract PythonAPI
run: tar -xvzf artifacts/PythonAPI.tar.gz -C artifacts

- uses: docker/build-push-action@v5
name: Build and push
with:
push: true
file: docker/Dockerfile
tags: cgellerac/carla-scenario-runner
no-cache: true
2 changes: 1 addition & 1 deletion CARLA_VER
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
HOST = https://carla-releases.s3.eu-west-3.amazonaws.com/Linux
RELEASE=CARLA_0.9.13
RELEASE=CARLA_0.9.15
65 changes: 65 additions & 0 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
ARG UBUNTU_VERSION="22.04"


############ dependencies ############
FROM "ubuntu:${UBUNTU_VERSION}" as dependencies

USER root
SHELL ["/bin/bash", "-c"]

ARG CARLA_API_PATH=/opt/carla/PythonAPI
ARG SCENARIO_RUNNER_PATH=/opt/carla/scenario_runner

# Install essentials
RUN apt-get update && \
apt-get install -y \
libpng16-16 \
libtiff5 \
libjpeg8 \
build-essential \
wget \
git \
libxerces-c-dev \
python3-pip \
&& rm -rf /var/lib/apt/lists/*

RUN pip3 install --upgrade pip
RUN update-alternatives --install /usr/bin/python python /usr/bin/python3 10

# Install scenario_runner
COPY . $SCENARIO_RUNNER_PATH
RUN pip3 install -r $SCENARIO_RUNNER_PATH/requirements.txt

# Move over PythonAPI
RUN mv $SCENARIO_RUNNER_PATH/artifacts/PythonAPI ${CARLA_API_PATH}

# Recursively install PythonAPI requirements, keep version of first occurrence
RUN cat $(find ${CARLA_API_PATH} -type f -name "requirements.txt") > /tmp/requirements_raw.txt \
&& awk -F '==' '{print $1}' /tmp/requirements_raw.txt | awk '!visited[$1]++' > /tmp/requirements.txt \
&& pip3 install -r /tmp/requirements.txt

# Create script that adds API to pythonpath and make .bashrc source it
RUN echo "export PYTHONPATH=\$PYTHONPATH:$CARLA_API_PATH/carla/dist/$(ls $CARLA_API_PATH/carla/dist | grep py$(python --version | awk -F'[ .]' '{print $2"."$3}').)" >> /setup_carla_env.sh; \
echo "export PYTHONPATH=\$PYTHONPATH:$CARLA_API_PATH/carla/agents" >> /setup_carla_env.sh; \
echo "export PYTHONPATH=\$PYTHONPATH:$CARLA_API_PATH/carla" >> /setup_carla_env.sh; \
echo "export CARLA_API_PATH=$CARLA_API_PATH" >> /setup_carla_env.sh; \
echo "source /setup_carla_env.sh" >> ~/.bashrc

############ final ############
FROM dependencies as final

# Needed for (pygame based) scripts that have a GUI
ENV SDL_VIDEODRIVER=x11

USER root
SHELL ["/bin/bash", "-c"]

ARG WORKSPACE=${SCENARIO_RUNNER_PATH}
ENV WORKSPACE=${WORKSPACE}
WORKDIR ${WORKSPACE}

# Set entrypoint
COPY ./docker/entrypoint.sh /
ENTRYPOINT [ "/entrypoint.sh" ]

CMD [ "python3", "scenario_runner.py", "--help" ]
8 changes: 8 additions & 0 deletions docker/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/bash
set -e

# source CARLA environment setup script
source /setup_carla_env.sh

# run the actual command
exec "$@"
29 changes: 15 additions & 14 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
py-trees==0.8.3
numpy==1.18.4; python_version >= '3.0'
networkx==2.2
Shapely==1.7.1
psutil
xmlschema==1.0.18
ephem
tabulate
opencv-python==4.2.0.32
matplotlib
six
simple-watchdog-timer
antlr4-python3-runtime==4.10
graphviz
# Update all versions to be compatible with Ubuntu 22.04, Python 3.10.6, ROS Humble (at 2023-07-24)
py-trees==0.8.3 # newest version (2.2.3) is not compatible
numpy==1.21.5
networkx==3.1
shapely==2.0.1
psutil==5.9.0
xmlschema==2.3.1
ephem==4.1.4
tabulate==0.9.0
opencv-python==4.8.0.74
matplotlib==3.7.2
six==1.16.0
simple-watchdog-timer==0.1.1
antlr4-python3-runtime==4.13.1
graphviz==0.20.1
2 changes: 1 addition & 1 deletion scenario_runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
from srunner.scenarioconfigs.osc2_scenario_configuration import OSC2ScenarioConfiguration

# Version of scenario_runner
VERSION = '0.9.13'
VERSION = '0.9.15'


class ScenarioRunner(object):
Expand Down
Loading