Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes #365 - Replace valgrind with sanitizers in Containerfile #366

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 42 additions & 24 deletions .github/scripts/compile.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,11 @@

set -euxo pipefail

WORKING=$(pwd)

wget ${PROTON_SOURCE_URL} -O qpid-proton.tar.gz
tar -zxf qpid-proton.tar.gz --one-top-level=qpid-proton-src --strip-components 1

do_patch () {
PATCH_DIR=$1
PATCH_SRC=$2
Expand All @@ -33,30 +38,43 @@ do_patch () {
fi
}

WORKING=`pwd`
wget ${PROTON_SOURCE_URL} -O qpid-proton.tar.gz
do_patch "patches/proton" qpid-proton-src

mkdir -p qpid-proton-src build staging proton_build proton_install
tar -zxf qpid-proton.tar.gz -C qpid-proton-src --strip-components 1
do_build () {
local suffix=${1}
local runtime_check=${2}

do_patch "patches/proton" qpid-proton-src
cmake -S "$WORKING/qpid-proton-src" -B "$WORKING/proton_build${suffix}" \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DRUNTIME_CHECK="${runtime_check}" \
-DENABLE_LINKTIME_OPTIMIZATION=ON \
-DCMAKE_POLICY_DEFAULT_CMP0069=NEW -DCMAKE_INTERPROCEDURAL_OPTIMIZATION=ON \
-DBUILD_TLS=ON -DSSL_IMPL=openssl -DBUILD_STATIC_LIBS=ON -DBUILD_BINDINGS=python -DSYSINSTALL_PYTHON=ON \
-DBUILD_EXAMPLES=OFF -DBUILD_TESTING=OFF \
-DCMAKE_INSTALL_PREFIX=/usr
cmake --build "proton_build${suffix}" --parallel "$(nproc)" --verbose
DESTDIR="$WORKING/proton_install${suffix}" cmake --install "proton_build${suffix}"

cmake -S "$WORKING/" -B "$WORKING/build${suffix}" \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DRUNTIME_CHECK="${runtime_check}" \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=ON \
-DProton_USE_STATIC_LIBS=ON -DUSE_LIBWEBSOCKETS=ON -DUSE_LIBNGHTTP2=ON \
-DProton_DIR="$WORKING/proton_install${suffix}/usr/lib64/cmake/Proton" \
-DBUILD_TESTING=OFF \
-DVERSION="${VERSION}" \
-DCMAKE_INSTALL_PREFIX=/usr
cmake --build "$WORKING/build${suffix}" --parallel "$(nproc)" --verbose
}

do_build "" OFF
do_build "_asan" asan
do_build "_tsan" tsan
Comment on lines +70 to +72
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The image build used to take 6 mins on GHA, this PR ran 14 minutes. Too bad. Each of these builds is different (different gcc options) so there is no space for any caching or such ways.


tar -z -C $WORKING/proton_install -cf /qpid-proton-image.tar.gz usr

cd proton_build
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DENABLE_LINKTIME_OPTIMIZATION=ON \
-DCMAKE_POLICY_DEFAULT_CMP0069=NEW -DCMAKE_INTERPROCEDURAL_OPTIMIZATION=ON \
-DBUILD_TLS=ON -DSSL_IMPL=openssl -DBUILD_STATIC_LIBS=ON -DBUILD_BINDINGS=python -DSYSINSTALL_PYTHON=ON \
-DBUILD_EXAMPLES=OFF -DBUILD_TESTING=OFF \
-DCMAKE_INSTALL_PREFIX=/usr $WORKING/qpid-proton-src/ \
&& VERBOSE=1 make DESTDIR=$WORKING/proton_install install \
&& tar -z -C $WORKING/proton_install -cf /qpid-proton-image.tar.gz usr \
&& VERBOSE=1 make install
cd $WORKING/build
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=ON \
-DProton_USE_STATIC_LIBS=ON -DUSE_LIBWEBSOCKETS=ON -DUSE_LIBNGHTTP2=ON \
-DBUILD_TESTING=OFF \
-DVERSION=${VERSION} \
-DCMAKE_INSTALL_PREFIX=/usr $WORKING/ \
&& VERBOSE=1 make DESTDIR=$WORKING/staging/ install \
&& tar -z -C $WORKING/staging/ -cf /skupper-router-image.tar.gz usr etc
DESTDIR=$WORKING/staging/ cmake --install $WORKING/build
cp "$WORKING/build_asan/router/skrouterd" "$WORKING/staging/usr/sbin/skrouterd_asan"
cp "$WORKING/build_tsan/router/skrouterd" "$WORKING/staging/usr/sbin/skrouterd_tsan"
cp --target-directory="$WORKING/staging/" "$WORKING/tests/tsan.supp" "$WORKING/build_asan/tests/lsan.supp"
tar -z -C $WORKING/staging/ -cf /skupper-router-image.tar.gz usr etc lsan.supp tsan.supp
23 changes: 14 additions & 9 deletions Containerfile
Original file line number Diff line number Diff line change
Expand Up @@ -19,19 +19,19 @@

FROM registry.access.redhat.com/ubi8/ubi:latest as builder

RUN dnf -y --setopt=tsflags=nodocs install \
RUN dnf -y --setopt=install_weak_deps=False --setopt=tsflags=nodocs install \
http://mirror.centos.org/centos/8-stream/BaseOS/x86_64/os/Packages/centos-gpg-keys-8-4.el8.noarch.rpm \
http://mirror.centos.org/centos/8-stream/BaseOS/x86_64/os/Packages/centos-stream-repos-8-4.el8.noarch.rpm \
&& dnf -y --setopt=tsflags=nodocs install epel-release \
&& dnf -y --setopt=install_weak_deps=False --setopt=tsflags=nodocs install epel-release \
&& dnf config-manager --set-enabled powertools \
&& dnf clean all

RUN dnf -y --setopt=tsflags=nodocs install \
RUN dnf -y --setopt=install_weak_deps=False --setopt=tsflags=nodocs install \
gcc gcc-c++ make cmake \
cyrus-sasl-devel openssl-devel libuuid-devel \
python3-devel swig \
libwebsockets-devel libnghttp2-devel \
wget patch findutils git valgrind \
wget patch findutils git libasan libubsan libtsan \
&& dnf clean all -y
WORKDIR /build
COPY . .
Expand All @@ -44,21 +44,21 @@ RUN .github/scripts/compile.sh

FROM registry.access.redhat.com/ubi8/ubi:latest

RUN dnf -y --setopt=tsflags=nodocs update \
&& dnf -y --setopt=tsflags=nodocs install \
RUN dnf -y --setopt=install_weak_deps=False --setopt=tsflags=nodocs update \
&& dnf -y --setopt=install_weak_deps=False --setopt=tsflags=nodocs install \
http://mirror.centos.org/centos/8-stream/BaseOS/x86_64/os/Packages/centos-gpg-keys-8-4.el8.noarch.rpm \
http://mirror.centos.org/centos/8-stream/BaseOS/x86_64/os/Packages/centos-stream-repos-8-4.el8.noarch.rpm \
&& dnf -y --setopt=tsflags=nodocs install epel-release \
&& dnf -y --setopt=install_weak_deps=False --setopt=tsflags=nodocs install epel-release \
&& dnf config-manager --set-enabled powertools \
&& dnf clean all

# gdb and valgrind are part of final image as they can be used as debug options for Skupper
RUN dnf -y --setopt=tsflags=nodocs install \
RUN dnf -y --setopt=install_weak_deps=False --setopt=tsflags=nodocs install \
glibc \
cyrus-sasl-lib cyrus-sasl-plain cyrus-sasl-gssapi libuuid openssl \
python3 \
libwebsockets libnghttp2 \
gdb valgrind \
gdb libasan libubsan libtsan \
gettext hostname iputils \
&& dnf clean all

Expand All @@ -74,5 +74,10 @@ ARG version=latest
ENV VERSION=${version}
ENV QDROUTERD_HOME=/home/skrouterd

ENV ASAN_OPTIONS="disable_coredump=0 detect_odr_violation=0 strict_string_checks=1 detect_stack_use_after_return=1 check_initialization_order=1 strict_init_order=1 detect_invalid_pointer_pairs=2"
ENV LSAN_OPTIONS="disable_coredump=0 suppressions=/lsan.supp"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lsan needs to use ptrace, which requires capabilities, for which I tend to run container with --privileged. Afaik this is no different from gdb, and therefore is not a problem.

one solution (to have asan without lsan) would be to define here detect_leaks=0.

ENV TSAN_OPTIONS="disable_coredump=0 history_size=4 second_deadlock_stack=1 suppressions=/tsan.supp"
ENV UBSAN_OPTIONS="disable_coredump=0 print_stacktrace=1 print_summary=1"

EXPOSE 5672 55672 5671
CMD ["/home/skrouterd/bin/launch.sh"]
8 changes: 6 additions & 2 deletions scripts/launch.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/sh
#!/bin/bash

#
# Licensed to the Apache Software Foundation (ASF) under one
Expand Down Expand Up @@ -33,7 +33,11 @@ fi
if [[ $QDROUTERD_DEBUG = "gdb" ]]; then
exec gdb -batch -ex "run" -ex "bt" --args skrouterd $ARGS
elif [[ $QDROUTERD_DEBUG = "valgrind" ]]; then
exec valgrind skrouterd $ARGS
exec skrouterd_asan $ARGS
Comment on lines 35 to +36
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

backward compatibility ftw!

elif [[ $QDROUTERD_DEBUG = "asan" ]]; then
exec skrouterd_asan $ARGS
elif [[ $QDROUTERD_DEBUG = "tsan" ]]; then
exec skrouterd_tsan $ARGS
else
exec skrouterd $ARGS
fi