Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates to KFP End-to-end tutorial (Issue #860) #890

Closed
wants to merge 17 commits into from
Closed
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 19 additions & 23 deletions content/docs/pipelines/tutorials/pipelines-tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,7 @@ Google Cloud Platform (GCP) is a suite of cloud computing services running
on Google infrastructure. The services include compute power, data storage,
data analytics, and machine learning.

The [Cloud SDK][cloud-sdk] is a set of tools that you can use to interact with
GCP from the command line, including the `gcloud` command and others.
The [Cloud Shell][cloud-shell] is a browser interface that provides command-line access to cloud resources that you can use to interact with GCP, including the `gcloud` command and others.

[Kubernetes Engine][kubernetes-engine] (GKE) is a managed service on GCP where
you can deploy containerized applications. You describe the resources that your
Expand Down Expand Up @@ -72,28 +71,20 @@ Follow these steps to set up your GCP environment:
1. Select or create a project on the [GCP Console][gcp-console].
1. Make sure that billing is enabled for your project. See the guide to
[modifying a project's billing settings][billing-guide].
1. Install the [Cloud SDK][cloud-sdk].
1. Grant users access to kubeflow with the `Owner` and `IAP-secured Web App User` roles:
OfficePop marked this conversation as resolved.
Show resolved Hide resolved

```
gcloud projects add-iam-policy-binding <PROJECT>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove this script block. I was wrong, project owner cannot be granted from the command line. They need to use Cloud Console.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I adjusted this, removing the 'Owner' part but leaving the IAP role. Or do they all need to be removed?

--member user:<EMAIL> --role <ROLE>
```
For more details, see [Granting, changing, and revoking access to resources][granting-changing-revoking-access].
OfficePop marked this conversation as resolved.
Show resolved Hide resolved

Notes:

* As you work through this tutorial, your project uses billable components of
GCP. To minimise costs, follow the instructions to
[clean up your GCP resources](#cleanup) when you've finished with them.
* This guide assumes you want to manage your GCP environment on your own server
rather than in the [Cloud Shell][cloud-shell] environment. If you choose to
use the Cloud Shell, some of the components are pre-installed in your shell.

### (Optional) Install kubectl

`kubectl` is the command-line tool for Kubernetes. `kubectl` is useful if you
want to interact with your Kubeflow cluster locally. If you decide not to use
`kubectl`, you can skip the steps later in the tutorial that use the command.

Run the following Cloud SDK command to install `kubectl`:

```
gcloud components install kubectl
```
* This guide assumes you want to manage your GCP environment in the [Cloud Shell][cloud-shell] environment rather than your own server. If you choose to use a local server instead, some of the components (such as Cloud SDK) must be manually installed.

### Set up some handy environment variables

Expand Down Expand Up @@ -131,7 +122,7 @@ Set up the following environment variables for use throughout the tutorial:
```
export DEPLOYMENT_NAME=kubeflow
```

### Deploy Kubeflow

Deploy Kubeflow on GCP:
Expand All @@ -146,7 +137,7 @@ Deploy Kubeflow on GCP:
credentials**, use the same value for the `<deployment_name>` as you used
when setting up the `DEPLOYMENT_NAME` environment variable earlier in this
tutorial.
* Choose **Kubeflow version v0.4.1** or later.
* Choose **Kubeflow version {{% kubeflow-current-version %}}** or later.

The following screenshot shows the Kubeflow deployment UI with hints about
the value for each input field:
Expand All @@ -158,7 +149,7 @@ Deploy Kubeflow on GCP:
1. (Optional) If you want to examine your cluster while waiting for the UI to
be available, you can use `kubectl` to connect to your cluster:

* Connect your local `kubectl` session to the cluster:
* Connect your cloud shell session to the cluster:
OfficePop marked this conversation as resolved.
Show resolved Hide resolved

```
gcloud container clusters get-credentials \
Expand Down Expand Up @@ -264,7 +255,7 @@ If you don't have a Python 3 environment set up, install
environment, run the following commands:

```bash
apt-get update; apt-get install -y wget bzip2
apt-get update;
OfficePop marked this conversation as resolved.
Show resolved Hide resolved
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh
```
Expand Down Expand Up @@ -400,6 +391,10 @@ UI.
classification label from 0 to 9. Each bar represents
the probability that the image matches the respective label.

Notes:

* You can find your trained model data in the bucket path you entered in step 5 of this procedure.

## Understanding the pipeline definition code

The pipeline is defined in the Python file `mnist_pipeline.py` which you
Expand Down Expand Up @@ -610,4 +605,5 @@ SDK](/docs/pipelines/sdk/sdk-overview/).
[gsutil-mb]: https://cloud.google.com/storage/docs/gsutil/commands/mb
[gsutil-acl-ch]: https://cloud.google.com/storage/docs/gsutil/commands/acl#ch


[granting-changing-revoking-access]: https://cloud.google.com/iam/docs/granting-changing-revoking-access#quick_updates_using_the_gcloud_command-line_tool
[release-page]:https://github.com/kubeflow/kubeflow/releases
1 change: 1 addition & 0 deletions layouts/shortcodes/kubeflow-current-version.html
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
0.5.1