Merge pull request #35 from getredash/dev_guide

Dev guide for Docker

_redirects

@@ -3,3 +3,4 @@
 /datasources/bigquery /data-sources/google-bigquery
 /datasources/redshift.html /data-sources/amazon-redshift
 /datasources/redshift /data-sources/amazon-redshift
+/help-onpremise/setup/setting-up-development-environment-using-vagrant.html /help-onpremise/dev/guide.html 301

onpremise/SUMMARY.md

@@ -15,7 +15,8 @@
 
 ## Developer Guide
 * [Guide](dev/guide.md)
-* [Setting up development environment (using Vagrant)](setup/setting-up-development-environment-using-vagrant.md)
+* [Docker Based Developer Installation Guide](dev/docker.md)
+* [Developer Installation Guide](dev/setup.md)
 
 ## How Redash Works
 * [Query Execution Model](how-rd-works/query-execution-model.md)

onpremise/dev/docker.md

@@ -0,0 +1,86 @@
+# Docker Based Developer Installation Guide
+
+## Installing Docker, Docker Compose and Node.js
+
+We will use Docker to run all the services needed for Redash, except for Node.js
+which we will run locally.
+
+1. [Install Docker and Docker Compose](https://docs.docker.com/engine/installation/).
+2. [Install Node.js](https://nodejs.org/en/download/) (v6 or later is recommended, can be installed with Homebrew on OS/X)
+
+## Setup
+
+### Create Docker Services
+Once you have the above setup, you need to create the Docker services:
+
+```bash
+docker-compose up
+```
+
+This will build the Docker images and fetch some prebuilt images and then start the services
+(Redash web server, worker, PostgreSQL and Redis). You can refer to the `docker-compose.yml`
+file to see the full configuration.
+
+### Install npm Packages
+
+```bash
+npm install
+```
+
+### Create Database
+
+```bash
+# Create tables
+docker-compose run --rm server create_db
+
+# Create database for tests
+docker-compose run --rm postgres psql -h postgres -U postgres -c "create database tests"
+```
+
+## Usage
+
+### Run webpack Dev Server
+
+Once all Docker services are running (can be started either by `docker-compose up` or
+`docker-compose start`), Redash is available at `http://localhost:5000/`.
+
+To work on the frontend code, you need to use the webpack dev server, which you start with:
+
+```bash
+npm run start
+```
+
+Now the dev server is available at `http://localhost:8080`. It rebuilds the frontend
+code when you change it and refreshes the browser. All the API calls are proxied to
+`localhost:5000` (the server running in Docker).
+
+### Restarting Celery Workers
+
+The Web server will restart on code changes, but the Celery workers won't. To restart
+the Celery workers run:
+
+```
+docker-compose restart worker
+```
+
+(or just stop `docker-compose up` and run it again)
+
+### Installing new Python packages (requirements.txt)
+
+If you pulled a new version with new packages or added some yourself, you will need to
+rebuild the `server` and `worker` images:
+
+```bash
+docker-compose rebuild worker
+docker-compose rebuild server
+```
+
+### Running Tests
+
+```bash
+docker-compose run --rm server tests
+```
+
+## Configuration
+
+TBD.

onpremise/dev/guide.md

@@ -16,95 +16,10 @@ the frontend we use ES6, Angular (1.5) and Webpack for bundling.
 >
 > Windows users: while it should be possible to run Redash on a Windows machine,
 > we don't know anyone who did this and lived to tell. We recommend using some
-> sort of a virtual machine in such case.
+> sort of a virtual machine or Docker in such case.
 
-## Installing Dependencies
+## Setup
 
-### PostgreSQL & Redis
-
-Refer to the documentation of Python, PostgreSQL, Redis and Node.js on how to install
-them in your environment. On macOS, you can use `brew` to install them. On Linux
-you can use your package manager, although need to make sure it installs recent
-enough versions.
-
-### Python Packages
-
-For development the minimum required packages to install are described in:
-
-* requirements.txt
-* requirements_dev.txt
-
-You install them with pip:
-
-```bash
-pip install -r requirements.txt -r requirements_dev.txt
-```
-
-(We recommend installing them in a virtualenv)
-
-### Node.js Packages
-
-Install all packages with:
-
-```bash
-npm install
-```
-
-## Configuration
-
-For development, in most cases the default configuration is enough. But if you need
-to adjust the database configuration, mail settings or any [other setting](../setup/settings-environment-variables.md),
-you do so with environment variables.
-
-To avoid having to export those variables manually, you can use a `.env` file and
-the `bin/run` helper script. By invoking any command with `bin/run` prefix, it will
-load your environment variables from the `.env` file and then run your command. For
-example:
-
-```bash
-bin/run ./manage.py check_settings
-```
-
-## Creating Database Tables
-
-TBD
-
-## Processes
-
-The main Redash processes you have to run:
-
-* Web server
-* Celery worker(s) & scheduler
-
-In development you will also run Webpack's dev server or watch utility.
-
-Our recommendation:
-
-* Web server: `bin/run ./manage.py runserver --debugger --reload`
-* Celery: `./bin/run celery worker --app=redash.worker --beat -Qscheduled_queries,queries,celery -c2`
-* Webpack dev server: `npm run start`
-
-This will result in a Flask web server listening on port `5000`, Webpack dev server
-on port `8080` and 2 Celery workers ready to run queries.
-
-To open the app in your web browser, use Webpack's dev server -- `localhost:8080`,
-which will auto reload and refresh whenever you make changes to the frontend code.
-
-## Running Tests
-
-Currently we currently have tests only for the backend code. To run them invoke:
-
-```bash
-make test
-```
-
-## Additional Resources
-
-* [How to create a new visualization](https://discuss.redash.io/t/how-to-create-new-visualization-types-in-redash/86)
-* [How to create a new query runner](https://discuss.redash.io/t/creating-a-new-query-runner-data-source-in-redash/347)
-
-## Getting Help
-
-* [Discussion Forum](https://discuss.redash.io/)
-* [Gitter (chat)](https://gitter.im/getredash/redash)
-* [Slack Community](http://slack.redash.io/)
+* [Docker Based Developer Installation Guide](./docker.md) (recommended for beginners)
+* [Developer Installation Guide](./setup.md) (recommended for experienced developers)
+* Using a remote server and installing locally only frontend dependencies (TBD)

onpremise/dev/setup.md

@@ -0,0 +1,81 @@
+# Developer Installation Guide
+
+## Installing Dependencies
+
+### PostgreSQL & Redis
+
+Refer to the documentation of Python, PostgreSQL, Redis and Node.js on how to install
+them in your environment. On macOS, you can use `brew` to install them. On Linux
+you can use your package manager, although need to make sure it installs recent
+enough versions.
+
+### Python Packages
+
+For development the minimum required packages to install are described in:
+
+* requirements.txt
+* requirements_dev.txt
+
+You install them with pip:
+
+```bash
+pip install -r requirements.txt -r requirements_dev.txt
+```
+
+(We recommend installing them in a virtualenv)
+
+### Node.js Packages
+
+Install all packages with:
+
+```bash
+npm install
+```
+
+## Configuration
+
+For development, in most cases the default configuration is enough. But if you need
+to adjust the database configuration, mail settings or any [other setting](../setup/settings-environment-variables.md),
+you do so with environment variables.
+
+To avoid having to export those variables manually, you can use a `.env` file and
+the `bin/run` helper script. By invoking any command with `bin/run` prefix, it will
+load your environment variables from the `.env` file and then run your command. For
+example:
+
+```bash
+bin/run ./manage.py check_settings
+```
+
+## Creating Database Tables
+
+TBD
+
+## Processes
+
+The main Redash processes you have to run:
+
+* Web server
+* Celery worker(s) & scheduler
+
+In development you will also run Webpack's dev server or watch utility.
+
+Our recommendation:
+
+* Web server: `bin/run ./manage.py runserver --debugger --reload`
+* Celery: `./bin/run celery worker --app=redash.worker --beat -Qscheduled_queries,queries,celery -c2`
+* Webpack dev server: `npm run start`
+
+This will result in a Flask web server listening on port `5000`, Webpack dev server
+on port `8080` and 2 Celery workers ready to run queries.
+
+To open the app in your web browser, use Webpack's dev server -- `localhost:8080`,
+which will auto reload and refresh whenever you make changes to the frontend code.
+
+## Running Tests
+
+Currently we currently have tests only for the backend code. To run them invoke:
+
+```bash
+make test
+```