Skip to content

Commit

Permalink
Fixing doc for KF installation on existing clusters (#977)
Browse files Browse the repository at this point in the history
* Fixing doc for existing clusters

* Update authentication.md

* Update getting-started-k8s.md

* Update getting-started-k8s.md

* Address review comments

* Minor changes

* Address review comments

* Address review comments

* Minor title change

* Fix path
  • Loading branch information
johnugeorge authored and k8s-ci-robot committed Aug 6, 2019
1 parent 63b34cb commit 70bda37
Show file tree
Hide file tree
Showing 5 changed files with 160 additions and 70 deletions.
2 changes: 1 addition & 1 deletion content/docs/started/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ to suit your environment (cloud, on premises (on prem), or local):
follow the [ICP deployment guide](/docs/started/getting-started-icp/).
* If you have an existing Kubernetes cluster or want to use Kubeflow on prem,
follow the [guide to deploying Kubeflow on
Kubernetes](/docs/started/getting-started-k8s/).
Kubernetes](/docs/started/k8s/overview/).
* If you want to run Kubernetes locally in a virtual machine (VM), choose one of
the following options:

Expand Down
5 changes: 5 additions & 0 deletions content/docs/started/k8s/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
+++
title = "Kubeflow on Existing Kubernetes Clusters"
description = "Instructions for installing Kubeflow on your existing clusters"
weight = 2
+++
Original file line number Diff line number Diff line change
@@ -1,73 +1,18 @@
+++
title = "Kubeflow on Kubernetes"
description = "Instructions for installing Kubeflow on your existing Kubernetes cluster"
title = "Kubeflow Deployment with kfctl_existing_arrikto"
description = "Instructions for installing Kubeflow with kfctl_existing_arrikto config"
weight = 4
+++

Follow these instructions if you want to install Kubeflow on an existing Kubernetes cluster.
This deployment uses [Dex](https://github.com/dexidp/dex) and [Istio](https://istio.io/) for vendor-neutral authentication.

If you are using a Kubernetes distribution or Cloud Provider which has specific instructions for installing Kubeflow we recommend following those instructions. Those instructions do additional Cloud specific setup to create a really great Kubeflow experience.

For installing Kubeflow on an existing Kubernetes Cluster, you can use one of the following options:

* [Kubeflow for Existing Clusters by Arrikto](#Kubeflow-for-Existing-Clusters---by-Arrikto)

## Kubeflow for Existing Clusters - by Arrikto

This installation of Kubeflow is maintained by [Arrikto](https://www.arrikto.com/), is geared towards existing Kubernetes clusters and does not depend on any cloud-specific feature.

In this reference architecture, we use [Dex](https://github.com/dexidp/dex) and [Istio](https://istio.io/) for vendor-neutral authentication.
**Maintainer and supporter: Arrikto**

![platform existing architecture](https://i.imgur.com/OlaN73j.png)

### Prerequisites
- Kubernetes Cluster with LoadBalancer support.

If you don't have a Kubernetes Cluster, you can create a compliant Kubernetes Engine (GKE) on Google Cloud Platform cluster with the following script:

<details>

<summary>GKE Cluster Creation Script</summary>

```bash
#!/bin/bash

set -e
- Kubernetes Cluster with LoadBalancer support. Refer [Load Balancer](https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/) guide.

# This script uses the gcloud command.
# For more info, visit: https://cloud.google.com/sdk/gcloud/reference/container/

# Edit according to your preference
GCP_USER="$(gcloud config list account --format "value(core.account)")"
GCP_PROJECT="$(gcloud config list project --format "value(core.project)")"
GCP_ZONE="us-west1-b"

CLUSTER_VERSION="$(gcloud container get-server-config --format="value(validMasterVersions[0])")"
CLUSTER_NAME="kubeflow"

############################
# Create and setup cluster #
############################

gcloud container clusters create ${CLUSTER_NAME} \
--project ${GCP_PROJECT} \
--zone ${GCP_ZONE} \
--cluster-version ${CLUSTER_VERSION} \
--machine-type "n1-standard-8" --num-nodes "1" \
--image-type "UBUNTU" \
--disk-type "pd-ssd" --disk-size "50" \
--no-enable-cloud-logging --no-enable-cloud-monitoring \
--no-enable-ip-alias \
--enable-autoupgrade --enable-autorepair

echo "Getting credentials for newly created cluster..."
gcloud container clusters get-credentials ${CLUSTER_NAME} --zone=${GCP_ZONE}

echo "Setting up GKE RBAC..."
kubectl create clusterrolebinding cluster-admin-binding --clusterrole=cluster-admin --user=${GCP_USER}
```

</details>

### Deploy Kubeflow

Expand All @@ -80,14 +25,7 @@ Follow these steps to deploy Kubeflow:
```
tar -xvf kfctl_<release tag>_<platform>.tar.gz
```

Alternatively, you can build the `kfctl` binary yourself:

```
git clone https://github.com/kubeflow/kubeflow.git
cd kubeflow/bootstrap
make build-kfctl-container
```


1. Run the following commands to set up and deploy Kubeflow. The code below includes an optional command to add the binary `kfctl` to your path. If you don't add the binary to your path, you must use the full path to the `kfctl` binary each time you run it.

Expand Down Expand Up @@ -521,4 +459,4 @@ Your Kubeflow app directory contains the following files and directories:
UIs](/docs/other-guides/accessing-uis/), where you can manage various
aspects of your Kubeflow deployment.
* Run a [sample machine learning workflow](/docs/examples/resources/).
* Get started with [Kubeflow Pipelines](/docs/pipelines/pipelines-quickstart/)
* Get started with [Kubeflow Pipelines](/docs/pipelines/pipelines-quickstart/)
92 changes: 92 additions & 0 deletions content/docs/started/k8s/kfctl-k8s-istio.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
+++
title = "Kubeflow Deployment with kfctl_k8s_istio"
description = "Instructions for installing Kubeflow on your existing Kubernetes cluster using kfctl_k8s_istio config"
weight = 2
+++

This config creates a vanilla deployment of Kubeflow with all its core components without any external dependencies. The deployment can be customized based on your environment needs.

**Maintainer and supporter: Kubeflow community**

### Deploy Kubeflow

Follow these steps to deploy Kubeflow:

1. Download a `kfctl` release from the [Kubeflow releases page](https://github.com/kubeflow/kubeflow/releases/) and unpack it:

```
tar -xvf kfctl_<release tag>_<platform>.tar.gz
```

1. Run the following commands to set up and deploy Kubeflow. The code below includes an optional command to add the binary `kfctl` to your path. If you don't add the binary to your path, you must use the full path to the `kfctl` binary each time you run it.

```bash
# Add kfctl to PATH, to make the kfctl binary easier to use.
export PATH=$PATH:"<path to kfctl>"
export KFAPP="<your choice of application directory name>"
# Installs istio by default. Comment out istio components in the config file to skip istio installation. See https://github.com/kubeflow/kubeflow/pull/3663
export CONFIG="https://raw.githubusercontent.com/kubeflow/kubeflow/master/bootstrap/config/kfctl_k8s_istio.yaml"

kfctl init ${KFAPP} --config=${CONFIG} -V
cd ${KFAPP}
kfctl generate all -V
kfctl apply all -V
```

* **${KFAPP}** - the _name_ of a directory where you want Kubeflow
configurations to be stored. This directory is created when you run
`kfctl init`. If you want a custom deployment name, specify that name here.
The value of this variable becomes the name of your deployment.
The value of this variable cannot be greater than 25 characters. It must
contain just the directory name, not the full path to the directory.
The content of this directory is described in the next section.


Check the resources deployed in namespace `kubeflow`:

```
kubectl -n kubeflow get all

```

### Access Kubeflow Dashboard

Once Kubeflow is deployed, the Kubeflow Dashboard can be accessed via `istio-ingressgateway` service. If loadbalancer is not available in your environment, NodePort or Port forwarding can be used to access the Kubeflow Dashboard.Refer [Ingress Gateway guide](https://istio.io/docs/tasks/traffic-management/ingress/ingress-control/).

### Delete Kubeflow

Run the following commands to delete your deployment and reclaim all resources:

```bash
cd ${KFAPP}
# If you want to delete all the resources, run:
kfctl delete all -V
```

### Understanding the deployment process

The deployment process is controlled by 4 different commands:

* **init** - one time set up.
* **generate** - creates config files defining the various resources.
* **apply** - creates or updates the resources.
* **delete** - deletes the resources.

With the exception of `init`, all commands take an argument which describes the
set of resources to apply the command to; this argument can be one of the
following:

* **k8s** - all resources that run on Kubernetes.
* **all** - platform and Kubernetes resources.

#### App layout

Your Kubeflow app directory contains the following files and directories:

* **${KFAPP}/app.yaml** defines configurations related to your Kubeflow deployment.
* **${KFAPP}/kustomize**: contains the YAML manifests that will be deployed.

### Next steps

* Run a [sample machine learning workflow](/docs/examples/resources/).
* Get started with [Kubeflow Pipelines](/docs/pipelines/pipelines-quickstart/)
55 changes: 55 additions & 0 deletions content/docs/started/k8s/overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
+++
title = "Deploying Kubeflow on Existing Clusters"
description = "Instructions for installing Kubeflow on your existing Kubernetes cluster with list of supported options"
weight = 1
+++

Follow these instructions if you want to install Kubeflow on an existing Kubernetes cluster.

If you are using a Kubernetes distribution or Cloud Provider which has specific instructions for installing Kubeflow we recommend following those instructions. Those instructions do additional Cloud specific setup to create a really great Kubeflow experience.

The following table lists the options for installing Kubeflow on an existing Kubernetes Cluster and links to detailed instructions.

### Community maintained

This section includes vendor-neutral solutions governed by community consensus. Below deployment configs are maintained and supported by the community.

<div class="table-responsive">
<table class="table table-bordered">
<thead class="thead-light">
<tr>
<th>Deployment config</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>kfctl_k8s_istio.yaml</td>
<td> This config creates a vanilla deployment of Kubeflow with all its core components without any external dependencies. The deployment can be customized based on your environment needs. <br />Follow instructions: <a href="/docs/started/k8s/kfctl-k8s-istio/">Kubeflow Deployment with kfctl_k8s_istio</a></td>
</tr>
</tbody>
</table>
</div>

### Vendor maintained

This section includes the deployment solutions that are supported by specific vendors/providers.

<div class="table-responsive">
<table class="table table-bordered">
<thead class="thead-light">
<tr>
<th>Deployment config</th>
<th>Description</th>
<th>Maintainer/Supporter</th>
</tr>
</thead>
<tbody>
<tr>
<td>existing_arrikto.yaml</td>
<td> This deployment uses Dex and Istio for vendor-neutral authentication. <br />Follow instructions: <a href="/docs/started/k8s/kfctl-existing-arrikto/">Kubeflow Deployment with existing_arrikto</a></td>
<td><a href="https://www.arrikto.com/">Arrikto</a></td>
</tr>
</tbody>
</table>
</div>

0 comments on commit 70bda37

Please sign in to comment.