diff --git a/_redirects b/_redirects index 2d3ac54e9..82ccca0f0 100644 --- a/_redirects +++ b/_redirects @@ -3,3 +3,4 @@ /datasources/bigquery /data-sources/google-bigquery /datasources/redshift.html /data-sources/amazon-redshift /datasources/redshift /data-sources/amazon-redshift +/help-onpremise/setup/setting-up-development-environment-using-vagrant.html /help-onpremise/dev/guide.html 301 diff --git a/onpremise/SUMMARY.md b/onpremise/SUMMARY.md index 1f4255cc7..a2191476f 100644 --- a/onpremise/SUMMARY.md +++ b/onpremise/SUMMARY.md @@ -15,7 +15,8 @@ ## Developer Guide * [Guide](dev/guide.md) -* [Setting up development environment (using Vagrant)](setup/setting-up-development-environment-using-vagrant.md) +* [Docker Based Developer Installation Guide](dev/docker.md) +* [Developer Installation Guide](dev/setup.md) ## How Redash Works * [Query Execution Model](how-rd-works/query-execution-model.md) diff --git a/onpremise/dev/docker.md b/onpremise/dev/docker.md new file mode 100644 index 000000000..41ce47208 --- /dev/null +++ b/onpremise/dev/docker.md @@ -0,0 +1,86 @@ +# Docker Based Developer Installation Guide + +## Installing Docker, Docker Compose and Node.js + +We will use Docker to run all the services needed for Redash, except for Node.js +which we will run locally. + +1. [Install Docker and Docker Compose](https://docs.docker.com/engine/installation/). +2. [Install Node.js](https://nodejs.org/en/download/) (v6 or later is recommended, can be installed with Homebrew on OS/X) + +## Setup + +### Create Docker Services +Once you have the above setup, you need to create the Docker services: + +```bash +docker-compose up +``` + +This will build the Docker images and fetch some prebuilt images and then start the services +(Redash web server, worker, PostgreSQL and Redis). You can refer to the `docker-compose.yml` +file to see the full configuration. + +### Install npm Packages + +```bash +npm install +``` + +### Create Database + +```bash +# Create tables +docker-compose run --rm server create_db + +# Create database for tests +docker-compose run --rm postgres psql -h postgres -U postgres -c "create database tests" +``` + +## Usage + +### Run webpack Dev Server + +Once all Docker services are running (can be started either by `docker-compose up` or +`docker-compose start`), Redash is available at `http://localhost:5000/`. + +To work on the frontend code, you need to use the webpack dev server, which you start with: + +```bash +npm run start +``` + +Now the dev server is available at `http://localhost:8080`. It rebuilds the frontend +code when you change it and refreshes the browser. All the API calls are proxied to +`localhost:5000` (the server running in Docker). + +### Restarting Celery Workers + +The Web server will restart on code changes, but the Celery workers won't. To restart +the Celery workers run: + +``` +docker-compose restart worker +``` + +(or just stop `docker-compose up` and run it again) + +### Installing new Python packages (requirements.txt) + +If you pulled a new version with new packages or added some yourself, you will need to +rebuild the `server` and `worker` images: + +```bash +docker-compose rebuild worker +docker-compose rebuild server +``` + +### Running Tests + +```bash +docker-compose run --rm server tests +``` + +## Configuration + +TBD. diff --git a/onpremise/dev/guide.md b/onpremise/dev/guide.md index 05eae2848..bacc54a6d 100644 --- a/onpremise/dev/guide.md +++ b/onpremise/dev/guide.md @@ -16,95 +16,10 @@ the frontend we use ES6, Angular (1.5) and Webpack for bundling. > > Windows users: while it should be possible to run Redash on a Windows machine, > we don't know anyone who did this and lived to tell. We recommend using some -> sort of a virtual machine in such case. +> sort of a virtual machine or Docker in such case. -## Installing Dependencies +## Setup -### PostgreSQL & Redis - -Refer to the documentation of Python, PostgreSQL, Redis and Node.js on how to install -them in your environment. On macOS, you can use `brew` to install them. On Linux -you can use your package manager, although need to make sure it installs recent -enough versions. - -### Python Packages - -For development the minimum required packages to install are described in: - -* requirements.txt -* requirements_dev.txt - -You install them with pip: - -```bash -pip install -r requirements.txt -r requirements_dev.txt -``` - -(We recommend installing them in a virtualenv) - -### Node.js Packages - -Install all packages with: - -```bash -npm install -``` - -## Configuration - -For development, in most cases the default configuration is enough. But if you need -to adjust the database configuration, mail settings or any [other setting](../setup/settings-environment-variables.md), -you do so with environment variables. - -To avoid having to export those variables manually, you can use a `.env` file and -the `bin/run` helper script. By invoking any command with `bin/run` prefix, it will -load your environment variables from the `.env` file and then run your command. For -example: - -```bash -bin/run ./manage.py check_settings -``` - -## Creating Database Tables - -TBD - -## Processes - -The main Redash processes you have to run: - -* Web server -* Celery worker(s) & scheduler - -In development you will also run Webpack's dev server or watch utility. - -Our recommendation: - -* Web server: `bin/run ./manage.py runserver --debugger --reload` -* Celery: `./bin/run celery worker --app=redash.worker --beat -Qscheduled_queries,queries,celery -c2` -* Webpack dev server: `npm run start` - -This will result in a Flask web server listening on port `5000`, Webpack dev server -on port `8080` and 2 Celery workers ready to run queries. - -To open the app in your web browser, use Webpack's dev server -- `localhost:8080`, -which will auto reload and refresh whenever you make changes to the frontend code. - -## Running Tests - -Currently we currently have tests only for the backend code. To run them invoke: - -```bash -make test -``` - -## Additional Resources - -* [How to create a new visualization](https://discuss.redash.io/t/how-to-create-new-visualization-types-in-redash/86) -* [How to create a new query runner](https://discuss.redash.io/t/creating-a-new-query-runner-data-source-in-redash/347) - -## Getting Help - -* [Discussion Forum](https://discuss.redash.io/) -* [Gitter (chat)](https://gitter.im/getredash/redash) -* [Slack Community](http://slack.redash.io/) +* [Docker Based Developer Installation Guide](./docker.md) (recommended for beginners) +* [Developer Installation Guide](./setup.md) (recommended for experienced developers) +* Using a remote server and installing locally only frontend dependencies (TBD) diff --git a/onpremise/dev/setup.md b/onpremise/dev/setup.md new file mode 100644 index 000000000..9f999b325 --- /dev/null +++ b/onpremise/dev/setup.md @@ -0,0 +1,81 @@ +# Developer Installation Guide + +## Installing Dependencies + +### PostgreSQL & Redis + +Refer to the documentation of Python, PostgreSQL, Redis and Node.js on how to install +them in your environment. On macOS, you can use `brew` to install them. On Linux +you can use your package manager, although need to make sure it installs recent +enough versions. + +### Python Packages + +For development the minimum required packages to install are described in: + +* requirements.txt +* requirements_dev.txt + +You install them with pip: + +```bash +pip install -r requirements.txt -r requirements_dev.txt +``` + +(We recommend installing them in a virtualenv) + +### Node.js Packages + +Install all packages with: + +```bash +npm install +``` + +## Configuration + +For development, in most cases the default configuration is enough. But if you need +to adjust the database configuration, mail settings or any [other setting](../setup/settings-environment-variables.md), +you do so with environment variables. + +To avoid having to export those variables manually, you can use a `.env` file and +the `bin/run` helper script. By invoking any command with `bin/run` prefix, it will +load your environment variables from the `.env` file and then run your command. For +example: + +```bash +bin/run ./manage.py check_settings +``` + +## Creating Database Tables + +TBD + +## Processes + +The main Redash processes you have to run: + +* Web server +* Celery worker(s) & scheduler + +In development you will also run Webpack's dev server or watch utility. + +Our recommendation: + +* Web server: `bin/run ./manage.py runserver --debugger --reload` +* Celery: `./bin/run celery worker --app=redash.worker --beat -Qscheduled_queries,queries,celery -c2` +* Webpack dev server: `npm run start` + +This will result in a Flask web server listening on port `5000`, Webpack dev server +on port `8080` and 2 Celery workers ready to run queries. + +To open the app in your web browser, use Webpack's dev server -- `localhost:8080`, +which will auto reload and refresh whenever you make changes to the frontend code. + +## Running Tests + +Currently we currently have tests only for the backend code. To run them invoke: + +```bash +make test +``` diff --git a/onpremise/setup/setting-up-development-environment-using-vagrant.md b/onpremise/setup/setting-up-development-environment-using-vagrant.md deleted file mode 100644 index a1b6a7843..000000000 --- a/onpremise/setup/setting-up-development-environment-using-vagrant.md +++ /dev/null @@ -1,11 +0,0 @@ -# Setting up Development Environment (using Vagrant) - -To simplify contribution there is a [Vagrant box](https://vagrantcloud.com/redash/boxes/dev) available with all the needed software to run Redash for development (use it only for development). - -To get started with this box: - -1. Make sure you have recent version of [Vagrant](https://www.vagrantup.com/) installed. -2. Clone the Redash repository: `git clone https://github.com/getredash/redash.git`. -3. Change dir into the repository (`cd redash`) -4. To execute tests, run `./bin/vagrant_ctl.sh test` -5. To run the app, run `./bin/vagrant_ctl.sh start`. This might take some time the first time you run it, as it downloads the Vagrant virtual box. Now the server should be available on your host on port 9001 and you can login with username admin and password admin.