Skip to content
This repository has been archived by the owner on Jan 14, 2020. It is now read-only.

Creating a Kubernetes Cluster

Jump to bottom
shoenisch edited this page Dec 24, 2016 · 21 revisions

Table of Contents

Introduction

Setting up a Kubernetes cluster for a web application demonstrates the power of Kubernetes as a service.

Deploying a Kubernetes cluster involves two main steps: creating resources for a tenant and spinning them up into a cluster with the photon cluster create command. On Photon Controller, the following primitives are the building blocks of clusters:

  • Images
  • Virtual machines
  • Disks
  • Resource tickets

Requirements

A Kubernetes cluster carries several requirements:

  • A static IP address to assign as the master IP address of the cluster.
  • At least one static IP address for an etcd node in the cluster. For high availability of etcd, you'll need three static IP addresses.
  • The Kubernetes image for Photon Controller.

The instructions assume that you are using a Linux workstation with the Photon Controller CLI connected to a deployment of Photon Controller on ESXi. You can also use a Mac or Windows workstation with the Photon Controller CLI installed, but you will have to adapt some of the commands to your operating system and environment.

For information about photon commands, subcommands, and options, see the help for the Photon Controller CLi on your Linux workstation; examples:

photon --help
photon resource-ticket create --help
photon cluster create -h

Obtaining, Uploading, and Enabling the Kubernetes Image

Download the Kubernetes disk image for Photon Controller from the following URL to the workstation on which you are running the Photon Controller command-line utility:

https://github.com/vmware/photon-controller/releases

The Kubernetes image is packaged as an OVA; it has a file name that looks like this, though the version and build numbers might be different:

kubernetes-1.4.3-pc-1.1.0-5de1cb7.ova

Upload the the Kubernetes image to Photon Controller by running the following commands, replacing the variables with the IP address of the load balancer for the Photon Controller management plane:

photon target set https://mgmt-ip-address:443
photon image create kubernetes-1.4.3-pc-1.1.0-5de1cb7.ova -n kube1 -i EAGER

The upload takes a few minutes. When it finishes, enable the image by obtaining its ID with the photon image list command and using it to replace the variable in the enable-cluster-type command:

photon image list
photon deployment enable-cluster-type -k KUBERNETES -i <Kubernetes_image_ID>

Creating a Tenant and a Project for the Cluster

A tenant is a unit of administrative control allocated a quota for projects, such as a Kubernetes cluster. The following sequence of commands creates a tenant, gives it a pool of resources, and creates a project with a ticket to use all the resources in the pool.

photon tenant create plato
photon resource-ticket create --tenant "plato" --name "plato-resources" --limits "vm.memory 100 GB, vm.cpu 100 COUNT, vm 100 COUNT, persistent-disk 100 COUNT, persistent-disk.capacity 200 GB" 
photon project create --tenant "plato" --name "plato-prjt" --resource-ticket "plato-resources" --limits "vm.memory 100 GB, vm.cpu 100 COUNT, vm 100 COUNT, persistent-disk 100 COUNT, persistent-disk.capacity 200 GB" 
photon tenant set "plato"
photon project set "plato-prjt"

Creating Resources for Use in the Cluster

The following command creates a flavor for a small VM that you'll use when you create the Kubernetes cluster.

photon flavor create --name cluster-small -k vm --cost "vm 1 COUNT, vm.cpu 1 COUNT, vm.memory 2 GB"

If you are setting up a Kubernetes cluster for production purposes in an environment without memory constraints, you can omit this command because Photon Controller includes the following default flavors for clusters:

  • cluster-master-vm. Photon Controller supplies this flavor as the default for a Kubernetes master node.
  • cluster-other-vm, for etcd nodes.
  • cluster-vm-disk, the default ephemeral disk for a Kubernetes cluster.

When you create a Kubernetes cluster, Photon Controller taps the default flavors unless you specify other flavors.

Creating Resources for Containerized Applications

The following sequence of commands provisions some resources for applications. You can skip these commands if you want, but the resources they create might come in handy later when you deploy applications. For more information, see Flavors.

photon -n flavor create --name "vm-basic" --kind "vm" --cost "vm 1 COUNT, vm.cpu 2 COUNT, vm.memory 2 GB" 
photon -n flavor create --name "disk-eph" --kind "ephemeral-disk" --cost "ephemeral-disk 1 COUNT"
photon -n flavor create --name "disk-persist" --kind "persistent-disk" --cost "persistent-disk 1 COUNT"

Setting Up a Network for Use with the Cluster

Finally, make a network for the cluster and set it as the default:

photon network create --name "vm-network" -portgroups "VM Network"

From the output of the photon network create command, note the network ID and then use it to set the default network in the following command:

photon network set-default <network_ID>

Creating the Kubernetes Cluster

You are now ready to create a Kubernetes cluster by running the following command. Replace the example IP addresses with those from your ESXi and network environments. The IP address for the master-ip option should contain the static IP address that you want to assign to the Kubernetes cluster. The etcd option should also contain a static IP address.

photon cluster create -n kube-socrates -k KUBERNETES --master-ip 203.0.113.208 --etcd1 203.0.113.209 --container-network 10.2.0.0/16 --dns 203.0.113.1 --gateway 203.0.113.253 --netmask 255.255.0.0 -c 1 --vm_flavor cluster-small

The cluster create command prompts you for several inputs. You can press Enter to accept the defaults and type 1 to for a worker node, or you can specify the options that you want.

The photon cluster create --help command shows descriptions of the options' values. Here's a truncated sample of its output:

   --name, -n 			Cluster name
   --type, -k 			Cluster type (accepted values are KUBERNETES, MESOS, or SWARM)
   --dns 			VM network DNS server IP address
   --gateway 			VM network gateway IP address
   --netmask 			VM network netmask
   --master-ip 			Kubernetes master IP address (required for Kubernetes clusters)
   --container-network 		CIDR representation of the container network, e.g. '10.2.0.0/16' (required for Kubernetes clusters)
   --etcd1 			Static IP address with which to create etcd node 1 (required for Kubernetes  clusters)

Checking the Cluster's Resources and Status

After provisioning the cluster and its resources, you can log in to the Photon Controller web interface to check them out:

http://<ip-address-of-load-balancer-or-mgt->

Take a few moments to click through the web UI. Examine the tenant who owns the project running Kubernetes:

A Tenant in Photon Platform

In the list of the tenant's projects, you can click a project to view information about it:

A Project in Photon Platform

And once you've reached the project, you can click Clusters and then click the name of a cluster to see its settings:

A Kubernetes Cluster

Opening the Kubernetes Dashboard

To open the Kubernetes web interface in a browser, go to the following URL:

http://<ip-address-of-kubernetes-master>:8080

Now you're ready to load and run an application in your Kubernetes cluster.

Deploying an nginx Web Server

Now that you've got a Kubernetes cluster up and running on Photon Controller, you can run a application as a service. This example deploys an nginx web server to demonstrate how to launch an application.

You'll need the Kubernetes command-line interface, kubectl, which you can quickly download through the Photon Platform web interface:

Install kubectl

After you download kubectl, install it by changing its mode bits so that it is executable and then moving it to /usr/local/bin. Here's an example:

cd ~/Downloads/
chmod +x kubectl 
sudo mv kubectl /usr/local/bin

Here's a quick trick to create a Kubernetes configuration file to simplify the commands that you will run:

kubectl config set-cluster default --server=http://<ip-address-of-kubernetes-master>:8080
kubectl config set-context default --cluster=default
kubectl config use-context default

Now you can quickly check your nodes' status, and you can omit the server option in the commands because the default is set:

kubectl get nodes
NAME            STATUS    AGE
198.51.36.6     Ready     25d
198.51.45.117   Ready     25d

You're now ready to deploy the nginx web server as a service. Copy the following block of code into a file named nginx.yml. The code block contains the configuration for running the nginx web server on Kubernetes.

By setting the type field to NodePort, this YAML file instructs the Kubernetes master to allocate a port from the default node port range of 30000 to 32767. Each node proxies the same port to your service. A node port exposes a node's IP address; for more information, see the Kubernetes documentation.

apiVersion: v1
kind: Service
metadata:
  name: nginx-demo-service
  labels:
    app: nginx-demo
spec:
  type: NodePort
  ports:
  - port: 80
    protocol: TCP
    name: http
  selector:
    app: nginx-demo
---
apiVersion: v1
kind: ReplicationController
metadata:
  name: nginx-demo
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: nginx-demo
    spec:
      containers:
      - name: nginx-demo
        image: nginx
        ports:
        - containerPort: 80

Run the following command to create the nginx service on the Kubernetes cluster; you might need to modify the path the YAML file if it's not in your current working directory:

kubectl create -f nginx.yml

service "nginx-demo-service" created
replicationcontroller "nginx-demo" created

Now view the services running on the cluster:

kubectl get svc
NAME                 CLUSTER-IP   EXTERNAL-IP   PORT(S)     AGE
kubernetes           10.0.0.1     <none>        443/TCP     3d
nginx-demo-service   10.0.0.202   <nodes>       80/TCP      5h

Finally, run the following command to obtain the NodePort on which the nginx web service is exposed as a service:

kubectl describe svc nginx-demo-service

Name:			nginx-demo-service
Namespace:		default
Labels:			app=nginx-demo
Selector:		app=nginx-demo
Type:			NodePort
IP:			10.0.0.202
Port:			http	80/TCP
NodePort:		http	30786/TCP
Endpoints:		10.2.100.7:80,10.2.100.8:80,10.2.100.9:80 + 3 more...
Session Affinity:	None

You can also see the nginx workload running in the Kubernetes web interface:

Kubernetes Workloads

That's it. Now you can connect to the nginx web server by launching a web browser and pointing it at the IP address of the Kubernetes master plus the NodePort on which the service is running, which is 30786 in this case:

http://203.0.113.208:30786

You should see the nginx welcome screen:

nginx Welcome Screen

Troubleshooting Cluster Creation

If the creation of the Kubernetes cluster failed, run the following command to clean up clusters that might be in an error state.

Keep in mind that this command deletes all your clusters, so you should run it only if you are trying to deploy your first one:

photon -n cluster list | awk '{print $1}' | xargs -n 1 photon -n cluster delete

Another troubleshooting method is to log into the VMs that Photon Controller creates for the Kubernetes master and the etcd nodes. The default password for the root account is changeme. You should change it the first time you log in.

After logging in, examine the system's status and logs as well as the Docker networking configuration.

To delete a cluster to free up space for a new cluster in an environment with memory constraints, run this command:

photon cluster delete <cluster_ID>

If you mistakenly deleted Photon's default flavors for Kubernetes clusters, you can restore them by running the following commands as the system administrator:

photon -n flavor create --name "cluster-master-vm" --kind "vm" --cost "vm 1 COUNT, vm.cpu 4 COUNT, vm.memory 4 GB"
photon -n flavor create --name "cluster-other-vm" --kind "vm" --cost "vm 1 COUNT, vm.cpu 1 COUNT, vm.memory 2 GB"
photon -n flavor create --name "cluster-vm-disk" --kind "ephemeral-disk" --cost "ephemeral-disk 1 COUNT"

Related

Deploying Tomcat on a Kubernetes cluster.

Home | Download Photon Controller | Quick Start Guide | VMware on GitHub

Clone this wiki locally