Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM lab dockerfile & CI tests #52

Merged
merged 6 commits into from
Mar 29, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .github/workflows/optional-install-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,15 @@ jobs:
steps:
- uses: actions/checkout@v2

# dm-control
# dm-lab
- run: |
docker build -f bin/dm_lab.Dockerfile \
--build-arg PYTHON_VERSION='3.10' \
--tag shimmy-dm_lab-docker .
- name: Run dm-lab tests
run: docker run shimmy-dm_lab-docker pytest tests/test_dm_lab.py

# dm-control-multi-agent
- run: |
docker build -f bin/dm_control_multiagent.Dockerfile \
--build-arg PYTHON_VERSION='3.10' \
Expand Down
54 changes: 54 additions & 0 deletions bin/dm_lab.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# A Dockerfile that sets up a full shimmy install with test dependencies
ARG PYTHON_VERSION
FROM python:$PYTHON_VERSION

SHELL ["/bin/bash", "-o", "pipefail", "-c"]

# Install Shimmy requirements
RUN apt-get -y update \
&& apt-get install --no-install-recommends -y \
unzip \
libglu1-mesa-dev \
libgl1-mesa-dev \
libosmesa6-dev \
xvfb \
patchelf \
ffmpeg cmake \
&& apt-get autoremove -y \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

COPY . /usr/local/shimmy/
WORKDIR /usr/local/shimmy/

# Include Shimmy in Python path
ENV PYTHONPATH="$PYTHONPATH:/usr/local/shimmy/"

RUN pip install ".[dm-lab, testing]" --no-cache-dir

# Install DM lab requirements
RUN apt-get -y update \
&& apt-get install --no-install-recommends -y \
build-essential curl freeglut3 gettext git libffi-dev libglu1-mesa \
libglu1-mesa-dev libjpeg-dev liblua5.1-0-dev libosmesa6-dev \
libsdl2-dev lua5.1 pkg-config python-setuptools python3-dev \
software-properties-common unzip zip zlib1g-dev g++

# Install Bazel
RUN apt-get install -y apt-transport-https curl gnupg \
&& curl -fsSL https://bazel.build/bazel-release.pub.gpg | gpg --dearmor > bazel.gpg \
&& mv bazel.gpg /etc/apt/trusted.gpg.d/ \
&& echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" | tee /etc/apt/sources.list.d/bazel.list \
&& apt-get update && apt-get install -y bazel

# Build DM lab
RUN git clone https://github.com/deepmind/lab.git \
&& cd lab \
&& echo 'build --cxxopt=-std=c++17' > .bazelrc \
&& bazel build -c opt //python/pip_package:build_pip_package \
&& ./bazel-bin/python/pip_package/build_pip_package /tmp/dmlab_pkg \
&& pip3 install --force-reinstall /tmp/dmlab_pkg/deepmind_lab-*.whl \
&& cd .. \
&& rm -rf lab

ENTRYPOINT ["/usr/local/shimmy/bin/docker_entrypoint"]
44 changes: 44 additions & 0 deletions scripts/install_dm_lab.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
#!/bin/sh
set -eu

if [[ "$(uname -s)" == 'Linux' ]]; then
# Dependencies
apt-get update && apt-get install -y \
build-essential curl freeglut3 gettext git libffi-dev libglu1-mesa \
libglu1-mesa-dev libjpeg-dev liblua5.1-0-dev libosmesa6-dev \
libsdl2-dev lua5.1 pkg-config python-setuptools python3-dev \
software-properties-common unzip zip zlib1g-dev g++

# Bazel
apt-get install -y apt-transport-https curl gnupg
curl -fsSL https://bazel.build/bazel-release.pub.gpg | gpg --dearmor > bazel.gpg
mv bazel.gpg /etc/apt/trusted.gpg.d/
echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" | tee /etc/apt/sources.list.d/bazel.list
apt-get update && apt-get install -y bazel

elif [[ "$(uname -s)" == 'Darwin' ]]; then
which -s brew
if [[ $? != 0 ]] ; then
echo "Homebrew not installed, run: `ruby -e '$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)'`"
exit
else
brew install bazel
fi
else
echo "Install script does not support windows, melting pot and dmlab2d must be installed manually and are not natively supported"
exit 1
fi


pip3 install numpy

# Build
if [ ! -d "lab" ]; then
git clone https://github.com/deepmind/lab.git
cd lab
echo 'build --cxxopt=-std=c++17' > .bazelrc
bazel build -c opt //python/pip_package:build_pip_package
./bazel-bin/python/pip_package/build_pip_package /tmp/dmlab_pkg
pip3 install --force-reinstall /tmp/dmlab_pkg/deepmind_lab-*.whl
cd ..
rm -rf lab
1 change: 1 addition & 0 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ def get_version():
"h5py>=3.7.0",
"pettingzoo>=1.22.4",
],
"dm-lab": [],
"openspiel": ["open_spiel>=1.2", "pettingzoo>=1.22.4"],
"meltingpot": ["pettingzoo>=1.22.4"],
"bsuite": ["bsuite>=0.3.5"],
Expand Down