Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

4040 docker openshift #4168

Merged
merged 26 commits into from
Oct 17, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
d287772
stub out Docker and OpenShift files #4040
pdurbin Sep 14, 2017
77b3f67
switch from Project to Template, add dev docs #4040
pdurbin Sep 15, 2017
4702e0a
use config from @danmcp #4040
pdurbin Sep 15, 2017
f41d753
add centos/postgresql-94-centos7 #4040
pdurbin Sep 18, 2017
c20dd39
add ndslabs/dataverse-solr image #4040
pdurbin Sep 18, 2017
3ab921c
add more handy `oc` commands to docs #4040
pdurbin Sep 19, 2017
2198367
expose postgres env vars needed by init-glassfish #4040
pdurbin Sep 19, 2017
6ec54c2
must have selector for expose route to work #4040
pdurbin Sep 19, 2017
e90f771
set dataverseAdmin password to "admin" #4040
pdurbin Sep 19, 2017
975d38e
explain how to run containers as root #4040
pdurbin Sep 19, 2017
0a44410
build and switch to iqss/dataverse-solr on Docker Hub #4040
pdurbin Sep 20, 2017
7c81b4e
switch from ndslabs to iqss Dataverse/Glassfish image #4040
pdurbin Sep 21, 2017
6e4ef45
use the right Docker tag for Dataverse/Glassfish #4040
pdurbin Sep 22, 2017
8040a5d
document dataverse.timerServer JVM option
pdurbin Sep 25, 2017
815e48a
document how to create Docker images #4040
pdurbin Sep 25, 2017
43d3844
add NDS Labs Workbench for testing Dataverse #4040 #4152
pdurbin Sep 25, 2017
ce949c9
Docker tag with no issue number (kick-the-tires) #4040
pdurbin Sep 25, 2017
b84526c
get Dataverse/Glassfish container running as non-root #4040
pdurbin Sep 27, 2017
ca6b6be
Dataverse is too fat to run in 1 GB of memory #4040
pdurbin Sep 27, 2017
ef81e53
Merge branch 'develop' into 4040-docker-openshift #4040
pdurbin Sep 27, 2017
e920e9f
cleanup #4040
pdurbin Sep 27, 2017
74767a6
remove cruft from botch conflict resolution #4040
pdurbin Sep 27, 2017
4dd759b
let build script push arbitrary tags to Docker Hub #4040
pdurbin Sep 29, 2017
8a48b4f
Documentation edits minishift #4040
matthew-a-dunlap Oct 11, 2017
25cfb30
doc fix: all containers run as non-root now #4040
pdurbin Oct 12, 2017
1e7fb53
File path clarity#4040
matthew-a-dunlap Oct 17, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# See `conf/docker` for Docker images
42 changes: 42 additions & 0 deletions conf/docker/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
#!/bin/sh
# Creates images and pushes them to Docker Hub.
# The "kick-the-tires" tag should be relatively stable. No breaking changes.
# Push to custom tags or tags based on branch names to iterate on the images.
if [ -z "$1" ]; then
echo "No argument supplied. Please specify \"branch\" or \"custom my-custom-tag\" for experiments or \"stable\" if your change won't break anything."
exit 1
fi

if [ "$1" == 'branch' ]; then
echo "We'll push a tag to the branch you're on."
GIT_BRANCH=$(git rev-parse --abbrev-ref HEAD)
TAG=$GIT_BRANCH
elif [ "$1" == 'stable' ]; then
echo "We'll push a tag to the most stable tag (which isn't saying much!)."
TAG=kick-the-tires
elif [ "$1" == 'custom' ]; then
if [ -z "$1" ]; then
echo "You must provide a custom tag as the second argument."
exit 1
else
echo "We'll push a custom tag."
TAG=$2
fi
else
echo "Unexpected argument: $1. Exiting. Run with no arguments for help."
exit 1
fi
echo Images will be pushed to Docker Hub with the tag \"$TAG\".
# Use "conf" directory as context so we can copy schema.xml into Solr image.
docker build -t iqss/dataverse-solr:$TAG -f solr/Dockerfile ../../conf
docker push iqss/dataverse-solr:$TAG
# TODO: Think about if we really need dataverse.war because it's in dvinstall.zip.
# FIXME: Automate the building of dataverse.war and dvinstall.zip. Think about https://github.com/IQSS/dataverse/issues/3974 and https://github.com/IQSS/dataverse/pull/3975
cp ../../target/dataverse*.war dataverse-glassfish/dataverse.war
cp ../../scripts/installer/dvinstall.zip dataverse-glassfish
cp ../../doc/sphinx-guides/source/_static/util/default.config dataverse-glassfish
cp ../../downloads/glassfish-4.1.zip dataverse-glassfish
cp ../../downloads/weld-osgi-bundle-2.2.10.Final-glassfish4.jar dataverse-glassfish
docker build -t iqss/dataverse-glassfish:$TAG dataverse-glassfish
# FIXME: Check the output of `docker build` and only push on success.
docker push iqss/dataverse-glassfish:$TAG
5 changes: 5 additions & 0 deletions conf/docker/dataverse-glassfish/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
glassfish-4.1.zip
weld-osgi-bundle-2.2.10.Final-glassfish4.jar
dvinstall.zip
dataverse.war
default.config
98 changes: 98 additions & 0 deletions conf/docker/dataverse-glassfish/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
FROM centos:7.2.1511
MAINTAINER Dataverse (support@dataverse.org)

COPY glassfish-4.1.zip /tmp
COPY weld-osgi-bundle-2.2.10.Final-glassfish4.jar /tmp
COPY default.config /tmp
COPY dvinstall.zip /tmp

# Install dependencies
#RUN yum install -y unzip
RUN yum install -y \
cronie \
git \
java-1.8.0-openjdk-devel \
nc \
perl \
postgresql \
sha1sum \
unzip \
wget

ENV GLASSFISH_DOWNLOAD_SHA1 d1a103d06682eb08722fbc9a93089211befaa080
ENV GLASSFISH_DIRECTORY "/usr/local/glassfish4"
ENV HOST_DNS_ADDRESS "localhost"
ENV POSTGRES_DB "dvndb"
ENV POSTGRES_USER "dvnapp"
ENV RSERVE_USER "rserve"
ENV RSERVE_PASSWORD "rserve"

#RUN ls /tmp
#
RUN find /tmp
#
#RUN exitEarly

# Install Glassfish 4.1

RUN cd /tmp \
&& unzip glassfish-4.1.zip \
&& mv glassfish4 /usr/local \
&& cd /usr/local/glassfish4/glassfish/modules \
&& rm weld-osgi-bundle.jar \
&& cp /tmp/weld-osgi-bundle-2.2.10.Final-glassfish4.jar . \
#FIXME: Patch Grizzly too!
&& echo "Done installing and patching Glassfish"

RUN chmod g=u /etc/passwd

RUN mkdir -p /home/glassfish
RUN chgrp -R 0 /home/glassfish && \
chmod -R g=u /home/glassfish

RUN mkdir -p /usr/local/glassfish4
RUN chgrp -R 0 /usr/local/glassfish4 && \
chmod -R g=u /usr/local/glassfish4


#RUN exitEarlyBeforeJq
RUN yum -y install epel-release
RUN yum install -y jq

# Install jq
#RUN cd /tmp \
# && wget https://github.com/stedolan/jq/releases/download/jq-1.5/jq-linux64 \
# && mv jq-linux64 /usr/local/bin \
# && chmod +x /usr/local/bin/jq-linux64 \
# && ln -s /usr/local/bin/jq-linux64 /usr/local/bin/jq

# Customized persistence xml to avoid database recreation
#RUN mkdir -p /tmp/WEB-INF/classes/META-INF/
#COPY WEB-INF/classes/META-INF/persistence.xml /tmp/WEB-INF/classes/META-INF/

# Install iRods iCommands
#RUN cd /tmp \
# && yum -y install epel-release \
# && yum -y install ftp://ftp.renci.org/pub/irods/releases/4.1.6/centos7/irods-icommands-4.1.6-centos7-x86_64.rpm

#COPY config-glassfish /root/dvinstall
#COPY restart-glassfish /root/dvinstall
#COPY config-dataverse /root/dvinstall

#RUN cd /root/dvinstall && ./config-dataverse

COPY ./entrypoint.sh /
#COPY ./ddl /root/dvinstall
#COPY ./init-postgres /root/dvinstall
#COPY ./init-glassfish /root/dvinstall
#COPY ./init-dataverse /root/dvinstall
#COPY ./setup-all.sh /root/dvinstall
#COPY ./setup-irods.sh /root/dvinstall
COPY ./Dockerfile /
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It shouldn't be necessary to copy the Dockerfile into the image.


VOLUME /usr/local/glassfish4/glassfish/domains/domain1/files

EXPOSE 8080

ENTRYPOINT ["/entrypoint.sh"]
CMD ["dataverse"]
135 changes: 135 additions & 0 deletions conf/docker/dataverse-glassfish/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
#!/bin/bash -x

# Entrypoint script for Dataverse web application. This script waits
# for dependent services (Rserve, Postgres, Solr) to start before
# initializing Glassfish.

echo "whoami before..."
whoami
if ! whoami &> /dev/null; then
if [ -w /etc/passwd ]; then
# Make `whoami` return the glassfish user. # See https://docs.openshift.org/3.6/creating_images/guidelines.html#openshift-origin-specific-guidelines
# Fancy bash magic from https://github.com/RHsyseng/container-rhel-examples/blob/1208dcd7d4f431fc6598184dba6341b9465f4197/starter-arbitrary-uid/bin/uid_entrypoint#L4
echo "${USER_NAME:-glassfish}:x:$(id -u):0:${USER_NAME:-glassfish} user:/home/glassfish:/bin/bash" >> /etc/passwd
fi
fi
echo "whoami after"
whoami

set -e

if [ "$1" = 'dataverse' ]; then

export GLASSFISH_DIRECTORY=/usr/local/glassfish4
export HOST_DNS_ADDRESS=localhost

TIMEOUT=30

if [ -n "$RSERVE_SERVICE_HOST" ]; then
RSERVE_HOST=$RSERVE_SERVICE_HOST
elif [ -n "$RSERVE_PORT_6311_TCP_ADDR" ]; then
RSERVE_HOST=$RSERVE_PORT_6311_TCP_ADDR
elif [ -z "$RSERVE_HOST" ]; then
RSERVE_HOST="localhost"
fi
export RSERVE_HOST

if [ -n "$RSERVE_SERVICE_PORT" ]; then
RSERVE_PORT=$RSERVE_SERVICE_PORT
elif [ -n "$RSERVE_PORT_6311_TCP_PORT" ]; then
RSERVE_PORT=$RSERVE_PORT_6311_TCP_PORT
elif [ -z "$RSERVE_PORT" ]; then
RSERVE_PORT="6311"
fi
export RSERVE_PORT

echo "Using Rserve at $RSERVE_HOST:$RSERVE_PORT"

if ncat $RSERVE_HOST $RSERVE_PORT -w $TIMEOUT --send-only < /dev/null > /dev/null 2>&1 ; then
echo Rserve running;
else
echo Optional service Rserve not running.
fi


# postgres
if [ -n "$POSTGRES_SERVICE_HOST" ]; then
POSTGRES_HOST=$POSTGRES_SERVICE_HOST
elif [ -n "$POSTGRES_PORT_5432_TCP_ADDR" ]; then
POSTGRES_HOST=$POSTGRES_PORT_5432_TCP_ADDR
elif [ -z "$POSTGRES_HOST" ]; then
POSTGRES_HOST="localhost"
fi
export POSTGRES_HOST

if [ -n "$POSTGRES_SERVICE_PORT" ]; then
POSTGRES_PORT=$POSTGRES_SERVICE_PORT
elif [ -n "$POSTGRES_PORT_5432_TCP_PORT" ]; then
POSTGRES_PORT=$POSTGRES_PORT_5432_TCP_PORT
else
POSTGRES_PORT=5432
fi
export POSTGRES_PORT

echo "Using Postgres at $POSTGRES_HOST:$POSTGRES_PORT"

if ncat $POSTGRES_HOST $POSTGRES_PORT -w $TIMEOUT --send-only < /dev/null > /dev/null 2>&1 ; then
echo Postgres running;
else
echo Required service Postgres not running. Have you started the required services?
exit 1
fi

# solr
if [ -n "$SOLR_SERVICE_HOST" ]; then
SOLR_HOST=$SOLR_SERVICE_HOST
elif [ -n "$SOLR_PORT_8983_TCP_ADDR" ]; then
SOLR_HOST=$SOLR_PORT_8983_TCP_ADDR
elif [ -z "$SOLR_HOST" ]; then
SOLR_HOST="localhost"
fi
export SOLR_HOST

if [ -n "$SOLR_SERVICE_PORT" ]; then
SOLR_PORT=$SOLR_SERVICE_PORT
elif [ -n "$SOLR_PORT_8983_TCP_PORT" ]; then
SOLR_PORT=$SOLR_PORT_8983_TCP_PORT
else
SOLR_PORT=8983
fi
export SOLR_PORT

echo "Using Solr at $SOLR_HOST:$SOLR_PORT"

if ncat $SOLR_HOST $SOLR_PORT -w $TIMEOUT --send-only < /dev/null > /dev/null 2>&1 ; then
echo Solr running;
else
echo Required service Solr not running. Have you started the required services?
exit 1
fi

GLASSFISH_INSTALL_DIR="/usr/local/glassfish4"
cd $GLASSFISH_INSTALL_DIR
cp /tmp/dvinstall.zip $GLASSFISH_INSTALL_DIR
unzip dvinstall.zip
cd dvinstall
echo Copying the non-interactive file into place
cp /tmp/default.config .
echo Looking at first few lines of default.config
head default.config
# non-interactive install
echo Running non-interactive install
#./install -y -f > install.out 2> install.err
./install -y -f

# if [ -n "$DVICAT_PORT_1247_TCP_PORT" ]; then
# ./setup-irods.sh
# fi

echo -e "\n\nDataverse started"

sleep infinity
else
exec "$@"
fi

3 changes: 3 additions & 0 deletions conf/docker/postgresql/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# PostgreSQL for Dataverse (but consider switching to the image from CentOS)
#
# See also conf/docker/dataverse-glassfish/Dockerfile
28 changes: 28 additions & 0 deletions conf/docker/solr/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
FROM centos:7.2.1511
MAINTAINER Dataverse (support@dataverse.org)

RUN yum install -y wget unzip perl git java-1.8.0-openjdk-devel postgresql.x86_64

# Install Solr 4.6.0
# The context of the build is the "conf" directory.
COPY solr/4.6.0/schema.xml /tmp

RUN cd /tmp && wget https://archive.apache.org/dist/lucene/solr/4.6.0/solr-4.6.0.tgz && \
tar xvzf solr-4.6.0.tgz && \
mv solr-4.6.0 /usr/local/ && \
cd /usr/local/solr-4.6.0/example/solr/collection1/conf/ && \
mv schema.xml schema.xml.backup && \
cp /tmp/schema.xml . && \
rm /tmp/solr-4.6.0.tgz

RUN ln -s /usr/local/solr-4.6.0/example/logs /var/log/solr

VOLUME /usr/local/solr-4.6.0/example/solr/collection1/data

EXPOSE 8983

COPY docker/solr/Dockerfile /Dockerfile
COPY docker/solr/entrypoint.sh /

ENTRYPOINT ["/entrypoint.sh"]
CMD ["solr"]
10 changes: 10 additions & 0 deletions conf/docker/solr/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/bin/bash

if [ "$1" = 'solr' ]; then
cd /usr/local/solr-4.6.0/example/
java -jar start.jar
elif [ "$1" = 'usage' ]; then
echo 'docker run -d iqss/dataverse-solr solr'
else
exec "$@"
fi
Loading