Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try running Buster #186

Merged
merged 26 commits into from
Mar 15, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ install:
- cp config/config.yml.example config/config.yml
- cp db/schema.rb.example db/schema.rb
- docker-compose build
- docker-compose run web sleep 10
- docker-compose run web bash -c "rake db:setup"
- docker-compose run web bash -c "rake db:migrate"
- docker-compose run web bash -l -c "sleep 10"
- docker-compose run web bash -l -c "rake db:setup"
- docker-compose run web bash -l -c "rake db:migrate"
- docker-compose run web bower install --allow-root

script:
- docker-compose run web bash -c "rake test"
- docker-compose run web bash -l -c "rake test"
72 changes: 19 additions & 53 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,77 +1,43 @@
# Dockerfile # Mapknitter
# https://github.com/publiclab/mapknitter/

FROM ruby:2.4.4-stretch
MAINTAINER Sebastian Silva "sebastian@fuentelibre.org"

FROM debian:buster
LABEL This image deploys Mapknitter!

# Set correct environment variables.
RUN mkdir -p /app
ENV HOME /root

# Install dependencies
RUN curl -sL https://deb.nodesource.com/setup_8.x | bash -
RUN apt-get update -qq && apt-get install -y \
bundler ruby-rmagick libfreeimage3 \
libfreeimage-dev ruby-dev curl \
libssl-dev zip nodejs \
build-essential \
python-dev \
python3-dev \
python-numpy \
python3-numpy \
libspatialite-dev \
sqlite3 \
libpq-dev \
libcurl4-gnutls-dev \
libproj-dev \
libxml2-dev \
libgeos-dev \
libnetcdf-dev \
libpoppler-dev \
libspatialite-dev \
libhdf4-alt-dev \
libhdf5-serial-dev \
bash-completion \
cmake
libfreeimage-dev zip nodejs gdal-bin \
curl g++ gcc autoconf automake bison \
libc6-dev libffi-dev libgdbm-dev \
libncurses5-dev libsqlite3-dev libtool \
libyaml-dev make pkg-config sqlite3 \
zlib1g-dev libgmp-dev libreadline-dev libssl-dev \
procps libmariadb-dev-compat libmariadb-dev git python-gdal \
imagemagick

# Ruby
RUN gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB && curl -sSL https://get.rvm.io | bash -s stable && usermod -a -G rvm root
RUN /bin/bash -l -c ". /etc/profile.d/rvm.sh && rvm install 2.4.4 && rvm use 2.4.4 --default"

RUN curl -sL https://deb.nodesource.com/setup_8.x | bash - && apt-get install -y npm
RUN npm install -g bower

# Install updated Gdal (taken from https://hub.docker.com/r/geographica/gdal2/dockerfile)
ENV ROOTDIR /usr/local/
ENV GDAL_VERSION 2.3.2
ENV OPENJPEG_VERSION 2.3.0

# Load assets
WORKDIR $ROOTDIR/

ADD http://download.osgeo.org/gdal/${GDAL_VERSION}/gdal-${GDAL_VERSION}.tar.gz $ROOTDIR/src/
ADD https://github.com/uclouvain/openjpeg/archive/v${OPENJPEG_VERSION}.tar.gz $ROOTDIR/src/openjpeg-${OPENJPEG_VERSION}.tar.gz

# Compile and install OpenJPEG
RUN cd src && tar -xvf openjpeg-${OPENJPEG_VERSION}.tar.gz && cd openjpeg-${OPENJPEG_VERSION}/ \
&& mkdir build && cd build \
&& cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$ROOTDIR \
&& make && make install && make clean \
&& cd $ROOTDIR && rm -Rf src/openjpeg*

# Compile and install GDAL
RUN cd src && tar -xvf gdal-${GDAL_VERSION}.tar.gz && cd gdal-${GDAL_VERSION} \
&& ./configure --with-python --with-spatialite --with-pg --with-curl --with-openjpeg \
&& make && make install && ldconfig \
&& apt-get update -y \
&& apt-get remove -y --purge build-essential \
&& cd $ROOTDIR && cd src/gdal-${GDAL_VERSION}/swig/python \
&& python3 setup.py build \
&& python3 setup.py install \
&& cd $ROOTDIR && rm -Rf src/gdal*

# Install bundle of gems
SHELL [ "/bin/bash", "-l", "-c" ]
WORKDIR /tmp
ADD Gemfile /tmp/Gemfile
ADD Gemfile.lock /tmp/Gemfile.lock
RUN bundle install

# HOTFIX Workaround for mysql2 gem incompatibility with libmariadb-dev
RUN sed -i "s/ LONG_PASSWORD |//g" /usr/local/rvm/gems/ruby-*/gems/mysql2-*/lib/mysql2/client.rb
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here be dragons, we're monkey patching mysql2 gem here to make it work with libmariadb-compat-dev in Debian buster


# Add the Rails app
WORKDIR /app
ADD . /app
Expand Down
2 changes: 1 addition & 1 deletion Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ gem "friendly_id"

# dependencies
group :dependencies do
gem 'mysql2', '~> 0.3.20'
gem 'mysql2', '< 0.4'
gem "geokit-rails", "1.1.4"
gem "image_science", "1.2.6"
gem "recaptcha", :require => "recaptcha/rails"
Expand Down
4 changes: 2 additions & 2 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ DEPENDENCIES
geokit-rails (= 1.1.4)
image_science (= 1.2.6)
jshintrb
mysql2 (~> 0.3.20)
mysql2 (< 0.4)
oa-openid (= 0.3.2)
open_id_authentication
paperclip (~> 4.2.2)
Expand All @@ -198,4 +198,4 @@ RUBY VERSION
ruby 2.4.4p296

BUNDLED WITH
1.16.2
1.16.6
6 changes: 3 additions & 3 deletions lib/exporter.rb
Original file line number Diff line number Diff line change
Expand Up @@ -286,9 +286,9 @@ def self.generate_tiles(key, slug, root)
# zips up tiles at root/public/tms/<slug>.zip;
def self.zip_tiles(slug)
rmzip = 'cd public/tms/ && rm '+slug+'.zip && cd ../../'
system(self.ulimit+rmzip)
zip = 'cd public/tms/ && zip -rq '+slug+'.zip '+slug+'/ && cd ../../'
system(self.ulimit+zip)
system(rmzip)
zip = 'cd public/tms/ && ' + self.ulimit + 'zip -rq '+slug+'.zip '+slug+'/ && cd ../../'
system(zip)
end

# generates a tileset at root/public/tms/<slug>/
Expand Down
3 changes: 3 additions & 0 deletions start.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/sh
sleep 5
bundle exec rails s -p 3000 -b '0.0.0.0'
11 changes: 6 additions & 5 deletions test/unit/exporter_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,6 @@
class ExporterTest < ActiveSupport::TestCase
test "isolated exporter lib" do

puts ">>>>>>>>>>>>>>>>>>>>>>>"
puts system('gdal2tiles.py --version')
puts ">>>>>>>>>>>>>>>>>>>>>>>"

# make a sample image
system('mkdir -p public/system/images/1/original')
system('cp test/fixtures/demo.png public/system/images/1/original/')
Expand Down Expand Up @@ -35,6 +31,8 @@ class ExporterTest < ActiveSupport::TestCase
assert origin
ordered = false

system('mkdir -p public/warps/saugus-landfill-incinerator')
system('mkdir -p public/tms/saugus-landfill-incinerator')
# these params could be compressed - warpable coords is part of origin; are coords and origin required?
assert Exporter.generate_composite_tiff(warpable_coords, origin, map.placed_warpables, map.slug, ordered)
assert Exporter.generate_tiles('', map.slug, Rails.root.to_s)
Expand All @@ -47,9 +45,12 @@ class ExporterTest < ActiveSupport::TestCase
# make a sample image
system('mkdir -p public/system/images/2/original/')
system('touch public/system/images/2/original/test.png')
system('mkdir -p public/warps/saugus-landfill-incinerator/')
system('mkdir -p public/warps/saugus-landfill-incinerator')
system('mkdir -p public/tms/saugus-landfill-incinerator')
system('touch public/warps/saugus-landfill-incinerator/folder')
assert File.exist?('public/warps/saugus-landfill-incinerator/folder')
system('mkdir -p public/warps/saugus-landfill-incinerator-working')
system('touch public/warps/saugus-landfill-incinerator/test.png')
assert Exporter.delete_temp_files(w.map.slug)
end
end
Expand Down