vlcluster

Simple docker and nginx-based cluster designed for multi-env web applications.

vlcluster is currently beeing restructured

Getting Started

vlcluster consists of four different node types

• registry Manages the cluster; stores and delivers applications
• workers Run instances of applications
• gateways Create nginx-proxys to instances running in workers
• client Developers; can manage the cluster and push new applications

You can put multiple different node types on the same computer

Setting up the cluster

You can use windows, macOS or linux to manage your cluster and uplaod new application, but the registry, workers and gateways require linux with systemd (ubuntu, centos, fedora, ...)!

Creating the registry

Run the following commands to create our first registry. We'll call it mainnet. You can only have one registry per cluster - but you can have multiple clusters and switch between them! Make sure that your registry has a static ip address or a DNS entry, we'll need it later on to reference the node!

sudo npm install -g vlcluster
sudo vlcluster init registry -n "mainnet"

This will create the registry in ~/.vlcluster. The command will return a key. This key will be used later on to create the workers and clients

created registry!

private key: 8cfh8jVXPd...
Store this key safely!

We need to make sure that vlcluster runs, even after a reboot. So, let's install it as a systemd service!

sudo vlcluster daemon install -u root

Setting up our first worker

Now you can either switch to a new machine or install the worker and the registry on the same machine. Run the following commands to create a worker and assign it a endpoint address.

# install packages
sudo apt install docker.io # install docker (if you don't already have it)
sudo npm install -g vlcluster

# create worker
sudo vlcluster init worker -n "worker1" -h registry.example.com -k 8cfh8jVXPd4... -e worker1.example.com

We have to restart the daemon if your worker is on the same node as the registry.

sudo service vlcluster restart

If you're using a different node, install the vlcluster daemon with systemd

sudo vlcluster daemon install -u root

Creating a gateway

We want our requests to be able to get to our instances, so let's setup the gateway. You can do this on the same node as your worker or registry!

# install packages
sudo apt install nginx
sudo npm install -g vlcluster

# create gateway
sudo vlcluster init gateway --name "gateway1" --cluster-hostname registry.example.com --cluster-key 8cfh8jVXPd4... --endpoint-hostname gateway1.example.com

Restart the vlcluster daemon or install the daemon with systemd, like we did with the worker!

Connecting to the cluster

Let's get our first app published on our new mainnet cluster!

First, let's create a client. Open a new terminal on your computer and type

npm install -g vlcluster
vlcluster init

Now enter your email, the hostname of our registry (registry.example.com in the example) and paste the key (8cfh8jVXPd4... in our example).

You need to have docker installed on your system → Installation page

You don't need to run vlcluster as a daemon on your client.

Setting up applications

Clone vlcluster example and open a terminal in the project directory

Every vlcluster application requires a simple Dockerfile and a package.json with name and version

We'll deploy the vlcluster-example application to our registry, with the env productive.

vlcluster deploy -c mainnet -e productive

We want to reach the application, right? Let's route your domain name (example-application.example.com:80 in this example) to the vlcluster-example application - but only to instances running as productive.

vlcluster route domain -c mainnet -h example-application.example.com -p 80 -a vlcluster-example -e productive

example-application.example.com will automatically be routed to the latest version of vlcluster-example running as productive. A simple deploy is all you need now! vlcluster will install the new version, await application start up, update the proxies and then stop the old instances - zero downtime deploys!

Imagine you'd wanna run a test environnement, and have it available on test.example-application.example.com. All you'll have to do is just type in those commands:

vlcluster deploy -c mainnet -e test
vlcluster route domain -c mainnet -h test.example-application.example.com -p 80 -a vlcluster-example -e test

When making your own application, make sure to listen on the port defined in the env variable PORT. The port will automatically be assigned and routed.

Variables

Nobody wants hardcoded database connection strings! Thats why vlcluster allows you to create variables which will be set as ENV-variables in your applications

Just run:

# set variable for instances of all applications in all environnements
vlcluster var set -c mainnet -n DATABASE_HOST -v data.example.com -a -e

# set variable for instances of vlcluster-example in all environnements
vlcluster var set -c mainnet -n DATABASE_USER -v example -a vlcluster-example -e

# set variable for instances of vlcluster-example in test and productive environnements
vlcluster var set -c mainnet -n DATABASE_NAME -v example -a vlce_db_prod -e productive
vlcluster var set -c mainnet -n DATABASE_NAME -v example -a vlce_db_test -e test

The variables will be set after a deploy or a vlcluster instance restart

Command Reference

Setup

vlcluster init [client] [-e | --email <email>] [-h | --hostname <registry hostname>] [-k | --key <registry key>]
vlcluster init registry [-n | --name <registry name>]
vlcluster init worker [-h | --hostname <registry hostname>] [-k | --key <registry key>] [-n | --name <worker name>] [-e | --endpoint <worker endpoint>]
vlcluster init gateway [-h | --hostname <cluster hostname>] [-key | --key <cluster key>] [-n | --name <gateway name>] [-e | --endpoint <endpoint hostname>]

vlcluster daemon install [-u | --user <user>]

Building and Publishing

vlcluster build [[ -p | --project-path ] <project path> = "."]
vlcluster push [-c | --cluster <registry hostname>] [[ -a | --application ] <application>] [[ -v | --version ] <version>]
vlcluster upgrade [-c | --cluster <registry hostname>] [[ -a | --application ] <application>] [[ -v | --version ] <version>] [[ -e | --env ] <environnement>] [-i | --instances <instance count>]

vlcluster deploy [-c | --cluster <registry hostname>] [[ -e | --env ] <environnement>] [[ -p | --project-path ] <project path> = "."] [-i | --instances <instance count>]

Variables

vlcluster var set [-c | --cluster <registry hostname>] [[ -n | --name ] <name>] [[ -v | --value ] <value>] [-a | --application [ <application> ]] [-e | --env [ <environnement> ]]
vlcluster var read [-c | --cluster <registry hostname>] [[ -s | --source ] <path>] [[ -v | --value ] <value>] [-a | --application [ <application> ]] [-e | --env [ <environnement> ]]
vlcluster var list [-c | --cluster <registry hostname>] [-a | --application [ <application> ]] [-e | --env [ <environnement> ]]

Instance Management

vlcluster instance list [-c | --cluster <registry hostname>] [-a | --application [ <application> ]] [-e | --env [ <environnement> ]]
vlcluster instance restart [-c | --cluster <registry hostname>] [-a | --application [ <application> ]] [-e | --env [ <environnement> ]]

Routing

vlcluster route domain [-c | --cluster <registry hostname>] [-h | --host <host>] [-p | --port <port>] [-a | --application <application>] [-h | --host <host>] [-e | --env <env>]
vlcluster route websocket [-c | --cluster <registry hostname>] [-h | --host <host>] [-p | --port <port>] [-l | --location <location>]

SSL

SSL certificates are created by certbot which needs to be installed on the gateways! Creating the certificates requires the gateway to shutdown the nginx webserver temporarely!

vlcluster ssl enable [-c | --cluster <registry hostname>] [-h | --host <host>] [-p | --port <port>]

Utitlity

vlcluster system # shows cluster system info
vlcluster daemon # runs vlcluster daemon