-
Notifications
You must be signed in to change notification settings - Fork 7
Setting up a Development Environment
There are a few initial runtimes/environments that need to be available if you are setting up a standalone environment. This guide assumes we are setting this up on an Ubuntu box, other platforms will work but you'll need to adjust the scripts below
- Java JDK (v1.6+) - Required for backend/admin systems and
migratorscript - Ruby (~v1.9.3) & Gem & Bundler - Required for middleman and
world-bank migratorscraprer - NodeJS (v0.8) & NPM - Required for Tilestream (map serving) Optional
The libraries/tools that need installed on top (or alongside) of these runtimes include,
- MongoDB (v2.4+) - Stores aggregated data for building the site, may be going awaya
- Tilestream - NodeJS based Map Tile Server for Country Projects page
- Middleman - Ruby program for static site generation
First things first we need to get the base OS into a good shape for doing our stuff. This means updating apt-get and adding new repositories
# update apt-get catalog
sudo add-apt-repository ppa:git-core/ppa
sudo apt-get update
# install baseline tools
sudo apt-get -y install curl build-essential openssl libssl-dev git-core unzip openjdk-6-jdk zlib1g-dev libopenssl-ruby1.9.1 libruby1.9.1 libreadline-dev libsqlite3-0 libsqlite3-dev python-software-properties git apache2 libapache2-mod-proxy-html
Adds the 10-gen mongo apt catalog and installs mongodb. Also locks the version.
# install mongodb
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10
echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | sudo tee /etc/apt/sources.list.d/10gen.list
sudo apt-get update
sudo apt-get -y install mongodb-10gen=2.4.3
sudo echo "mongodb-10gen hold" | sudo dpkg --set-selections
# install ruby1.9.3 and rbenv
cd
git clone git://github.com/sstephenson/rbenv.git .rbenv
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(rbenv init -)"' >> ~/.bashrc
exec $SHELL
mkdir -p ~/.rbenv/plugins
cd ~/.rbenv/plugins
git clone git://github.com/sstephenson/ruby-build.git
rbenv install 1.9.3-p194
rbenv rehash
rbenv global 1.9.3-p194
# install bundler
gem install bundler
rbenv rehash
# install nodejs & npm
sudo apt-add-repository ppa:chris-lea/node.js
sudo apt-get update
sudo apt-get -y install nodejs
# install tilestream
cd
git clone https://github.com/mapbox/tilestream.git
cd tilestream
npm install
Now we have the baseline set of tools and technologies installed we need to actually get the application setup
This uses https and at the time of writing is a private repo. This means you will be asked for a username and password for Github. Preferably you could change this and use the public key approach
# grab the source - you WILL be asked for a valid github username and password
cd
git clone https://github.com/DFID/aid-platform-beta.git
The solution uses a number of folders for storing data and logs so we create these
# make directories for app assets
mkdir ~/logs/
mkdir ~/data/
mkdir ~/www
Apache sits in front of our stuff when running and this sets it up
# setup apache2
sudo rm -rf /var/www
sudo ln -s ~/www /var/www
sudo a2enmod headers
sudo a2enmod proxy
sudo a2enmod proxy_html
sudo find /etc/apache2/sites-enabled/ -mindepth 1 -delete
# optional security config
# we can only provide basic config files for Apache
# if you do not have these files, you may want to set up your own apache security config
sudo cp -fr ~/aid-platform-beta/conf/aid-tracker /etc/apache2/sites-enabled/
sudo cp -fr ~/aid-platform-beta/conf/httpd.conf /etc/apache2/
sudo cp -fr ~/aid-platform-beta/conf/security /etc/apache2/conf.d/
sudo mkdir -p /usr/local/etc/httpd/
sudo cp -fr ~/aid-platform-beta/conf/users /usr/local/etc/httpd/
sudo apache2ctl -k restart
Currently the TileStream server resides on another box and is hard coded into the script tags of the site so when this changes you can use this scriptlet to run TileStream against the dfid.mbtiles file
# run tilestream with maps folder
~/tilestream/index.js start --tiles=/home/<USERNAME>/aid-platform-beta/src/tilestream/ >> ~/logs/tilestream.log 2>&1 &
You need to create a admin user so you can log in
cd ~/aid-platform-beta/scripts
./bcrypt WHATEVER_YOU_WANT_THE_PASSWORD_TO_BE
./createuser dfidadmin 'WHATEVER_BCRYPT_SPAT_OUT'
Make sure to use the single quotes in the last command to avoid token replacement.
You need to generate a deployment password to add to the environment variables
cd ~/aid-platform-beta/scripts
./bcrypt 'WHATEVER_YOU_WANT_THE_PASSWORD_TO_BE'
The application requires a number of system variables to tell it where to store data and deploy the site (optional). We set that up here.
sudo cat >> ~/.bashrc <<DELIM
export DFID_SECRET='your-secret-key'
export DFID_DATA_PATH=~/data/iati-xml
export DFID_ELASTICSEARCH_PATH=~/data/elasticsearch
export DFID_STATIC_FILE_PATH=~/www
export DFID_DEPLOYMENT_PASSWORD='WHATEVER_BCRYPT_SPAT_OUT'
#if using SMTP email
export DFID_SMTP_USER='YOUR_EMAIL_USER'
export DFID_SMTP_PASSWORD='YOUR_EMAIL_PASSWORD'
DELIM
source ~/.bashrc
The alpha accumulated a lot of "reference" data that is useful. The migrator runs and seeds the Mongo database with this data. It also scrapes the world bank API for initial Population, Income Level and Life Expectancy data.
# run the migrator to get initial cms data into mongo
cd ~/aid-platform-beta/src/migrator
./sbt run
# call world bank api to bring in base country data
cd worldbank
bundle install
bundle exec ruby api.rb
To run the admin app (data loads etc) we can do this.
# run the site
cd ~/aid-platform-beta/src/platform
./sbt start
To generate the site after data loads etc. run this (Deploy Site button in the admin site will not do anything in Dev mode - you need to run the commands below)
# run the site
cd ~/aid-platform-beta/src/platform/site
bundle install
bundle exec middleman build
If you just want to run the site in dev mode (no static generation - dynamic generation of pages for dev purposes) you can use
bundle exec middleman