Skip to content
This repository has been archived by the owner on Apr 29, 2020. It is now read-only.

Setting Up A Cluster

Jump to bottom
Stephen Jung edited this page Apr 17, 2015 · 13 revisions

Introduction

This document will explain how to set up a new p2 cluster capable of starting new services on any hosts you manage.

Before You Begin

p2 requires the following be available on your hosts to correctly run your applications.

  • The Runit process manager, which includes chpst. In the future we may make the process manager used configurable. You should have a runsvdir process watching the /var/service directory.
  • ServiceBuilder, a simple script that helps with runit service management.
  • Logrotate
  • You will need two pod manifests. One configured to launch a Consul agent and one set up to run the p2-preparer. More on how to set this up below.
  • a GPG keyring for the preparer. All manifests launched by the preparer have to be signed by a public key in this keyring. It should be somewhere on your filesystem.

Installing the p2 binaries

The binaries in github.com/square/p2/bin should be available for use. You can get a stable install by running

git clone https://github.com/square/p2.git
cd p2
godep go install ./bin/...

You can also run

go install github.com/square/p2/bin/...

but there is no guarantee that the dependencies of p2 will still be compatible.

This process will install a number of useful binaries. We will use several of them to get ourselves up and running.

One of the binaries for p2 is a bootstrapping dependency, so it has to be installed to the system path beforehand.

sudo mv $(which p2-exec) /usr/local/bin/

A Quick Primer on Pod Manifests

Pod manifests are YAML files that contain one or more launchable artifacts as well as their runtime configuration. The preparer deploys applications by looking in the intent store for pod manifests assigned to the host where it is running.

Writing a manifest isn't hard, but is required to complete setup. Read here for details on pods and pod manifests.

Bootstrapping your host with p2-bootstrap

On every host where you would like p2 to launch applications, you will need to run p2-bootstrap, which sets up both the Consul agent as well as the p2-preparer.

Although there are several ways to set up a host, we will assume that your host needs both a Consul agent to connect to the Consul server cluster as well as the preparer. To achieve this, you will need two pod manifests and two associated Hoist artifacts.

The Consul Agent

First lets build a simple Consul Hoist artifact. This process may seem onerous at first, but as these artifacts are central to using p2, more tools to simplify their creation will become available over time.

Create a directory called "p2-consul". When you're done, you will have a layout that looks like this:

╰── p2-consul
    ╰── bin
        ├── consul
        ╰── launch

Download your preferred flavor of the Consul binary from Consul.io and place it at p2-consul/bin/consul. Next we'll implement p2-consul/bin/launch, our hook into the execution of the Consul artifact.

#!/bin/bash
DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
exec $DIR/consul agent -server -bootstrap-expect 1 -data-dir /tmp/consul

This script launches a standalone Consul server agent. More complex topologies are possible (refer to the Consul documentation). At this point, make sure both consul and launch are executable. To wrap up the Consul Hoist artifact, simply use the tar command to bundle the folder up.

$ ARTIFACT=p2-consul_$RANDOM.tar.gz
$ tar cvzf $ARTIFACT -C p2-consul/ ./
$ mv $ARTIFACT /tmp/$ARTIFACT

Finally, lets write a simple pod manifest to launch this artifact. Open p2-consul.yaml and write a manifest containing the location of the artifact we just built.

id: consul
launchables:
  consul:
    launchable_type: hoist
    launchable_id: consul
    location: /tmp/p2-consul_12345.tar.gz

You should now clearsign the manifest with your PGP key: gpg --clearsign p2-consul.yaml. This generates another file, p2-consul.yaml.asc, which is the same manifest wrapped in an ASCII-armored PGP signature.

The Preparer Pod

Although we will need to create the same files as we did above for Consul, the preparer pod will be much simpler to set up. Because p2-preparer is already set up to accept configuration from the CONFIG_PATH, we can use a shortcut to create both the Hoist artifact and the pod manifest.

In the /tmp directory, run p2-bin2pod on your installed p2-preparer. p2-bin2pod comes with several options but you shouldn't need any of them for the purposes of this guide.

$ cd /tmp
$ p2-bin2pod $(which p2-preparer)
{
  "tar_path": "/tmp/p2-preparer_kmwhrovrzugqrjyxsrxkvjlzilljlqznwsjoxipv.tar.gz",
  "manifest_path": "/tmp/p2-preparer.yaml",
  "final_location": "/tmp/p2-preparer_kmwhrovrzugqrjyxsrxkvjlzilljlqznwsjoxipv.tar.gz"
}

The resultant JSON document shows the output of the utility. If you plan on building workflows around p2-bin2pod, you can pipe the JSON output to jq or other tools to streamline builds.

In the preparer manifest, set the location of your keyring:

...
config:
  preparer:
    keyring: /path/to/my/public/keyring.gpg

Then clearsign the generated preparer manifest: gpg --clearsign /tmp/p2-preparer.yaml.

Running p2-bootstrap

We now run the p2-bootstrap command to set up the host. The command will launch the Consul agent, followed by the preparer, and finally schedule the preparer on the target host. In the future, when you want to update the preparer, you can release a new preparer manifest just like you would any other application deployed with p2.

$ useradd consul # the consul pod does not run as root
$ p2-bootstrap --consul-pod /tmp/consul.yaml.asc --agent-pod /tmp/preparer.yaml.asc
2015/01/07 13:20:27 Starting bootstrap
2015/01/07 13:20:27 Installing and launching consul
{"Counter":0,"PID":3277,"level":"info","msg":"Successfully installed","pod":"intent","time":"2015-01-07T13:20:27-05:00"}
{"Counter":1,"PID":3277,"level":"info","msg":"Successfully launched","pod":"intent","time":"2015-01-07T13:20:28-05:00"}
2015/01/07 13:20:38 Registering base agent in consul
2015/01/07 13:20:38 Installing and launching base agent
{"Counter":2,"PID":3277,"level":"info","msg":"Successfully installed","pod":"p2-preparer","time":"2015-01-07T13:20:38-05:00"}
{"Counter":3,"PID":3277,"level":"info","msg":"Successfully launched","pod":"p2-preparer","time":"2015-01-07T13:20:38-05:00"}
2015/01/07 13:20:38 Bootstrapping complete

You're now ready to launch applications with p2!

Launching a Hello World Service

We're going to again use p2-bin2pod to make our lives easier. Write a simple shell script that launches a server.

$ echo '#!/bin/bash
exec python -m SimpleHTTPServer
' > simplehttp
$ sudo useradd simplehttp
$ chmod a+x simplehttp
$ p2-bin2pod simplehttp
{
  "tar_path": "/tmp/simplehttp_lkjsnkvoznyzwlruorzxzznulxskszupgkxgimqq.tar.gz",
  "manifest_path": "/tmp/simplehttp.yaml",
  "final_location": "/tmp/simplehttp_lkjsnkvoznyzwlruorzxzznulxskszupgkxgimqq.tar.gz"
}

Once the manifest is signed, it can be deployed like so:

$ p2-schedule /tmp/simplehttp.yaml

Looking at the preparer's standard out in the runit logs (under /var/service/p2-preparer_preparer/log/main/current, you should see the following line:

2015-01-07_18:41:45.11745 {"Counter":3,"PID":2494,"app":"preparer","level":"info","msg":"Successfully launched","pod":"/data/pods/simplehttp","time":"2015-01-07T18:41:45Z"}

To confirm that it's running, use a curl to the python http endpoint.

$ curl -I localhost:8000
HTTP/1.0 200 OK
Server: SimpleHTTP/0.6 Python/2.6.6
Date: Wed, 07 Jan 2015 18:42:17 GMT
Content-type: text/html; charset=ANSI_X3.4-1968
Content-Length: 272
Clone this wiki locally