Skip to content

Commit

Permalink
Support blueprints in autodeployment.
Browse files Browse the repository at this point in the history
* Create a blueprint reconciler to autodeploy and reconcile
blueprints.

  * The reconciler decides whether we need to deploy a new blueprint
    and if it does it creates a Tekton PipelineRun to deploy Kubeflow.

* Here are some differences in how we are deploying blueprints vs. kfctl
  deployments

  * We are using Tekton PipelineRuns as opposed to K8s jobs to do the
    deployment

  * We no longer use deployments.yaml to describe the group of deployments.
    Instead we just create a PipelineRun.yaml and that provides all
    the information the reconciler needs e.g. the branch to watch
    for changes.

* Update the flask app to provide information about blueprints.
  * Include a link to the Tekton dashboard showing the PipelineRun
    that deployed Kubeflow.

* Define a Pipeline to deploy Kubeflow so we don't have to inline the
  spec in the PipelienRun.

* Remove Dockerfile.skaffold; we can use skaffold auto-sync in developer mode. Add a column in the webserver to redirect to the Tekton dashboard for the PipelineRun that deployed it.

* GoogleCloudPlatform/kubeflow-distribution#5 Setup autodeploy for gcp blueprints.
  • Loading branch information
Jeremy Lewi committed May 9, 2020
1 parent d778258 commit 0d18921
Show file tree
Hide file tree
Showing 29 changed files with 1,347 additions and 171 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
apiVersion: tekton.dev/v1alpha1
kind: Pipeline
metadata:
name: deploy-gcp-blueprint
namespace: auto-deploy
spec:
params:
- default: kf-vbp-{uid}
description: The name for the Kubeflow deployment
name: name
type: string
- default: kf-ci-management
description: The name of the management cluster.
name: management-cluster-name
type: string
- default: kubeflow-ci-deployment
description: The project to deploy into
name: project
type: string
resources:
- name: testing-repo
type: git
- name: blueprint-repo
type: git
tasks:
- name: deploy-gcp
params:
- name: name
value: $(params.name)
- name: project
value: $(params.project)
- name: management-cluster-name
value: $(params.management-cluster-name)
resources:
inputs:
- name: blueprint-repo
resource: blueprint-repo
- name: testing-repo
resource: testing-repo
taskRef:
kind: namespaced
name: deploy-gcp-blueprint
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,11 @@ spec:
- description: The GitHub repo containing the blueprint
name: blueprint-repo
type: git
stepTemplate:
env:
- name: KUBECONFIG
value: /workspace/kubeconfig
image: gcr.io/kubeflow-ci/test-worker-py3@sha256:b679ce5d7edbcc373fd7d28c57454f4f22ae987f200f601252b6dcca1fd8823b
steps:
- command:
- /workspace/$(inputs.resources.blueprint-repo.name)/kubeflow/hack/create_context.sh
Expand All @@ -45,11 +50,8 @@ spec:
value: $(inputs.params.management-location)
- name: NAME
value: $(inputs.params.management-cluster-name)
- name: KUBECONFIG
value: /workspace/kubeconfig
- name: NAMESPACE
value: $(inputs.params.project)
image: gcr.io/kubeflow-ci/test-worker-py3@sha256:b679ce5d7edbcc373fd7d28c57454f4f22ae987f200f601252b6dcca1fd8823b
name: get-credential
- command:
- python
Expand All @@ -59,11 +61,19 @@ spec:
- --name=$(inputs.params.name)
- --blueprint-dir=/workspace/$(inputs.resources.blueprint-repo.name)/kubeflow
- --management-context=$(inputs.params.management-cluster-name)
- --labels-file=/etc/podinfo/labels
env:
- name: KUBECONFIG
value: /workspace/kubeconfig
- name: PYTHONPATH
value: /workspace/$(inputs.resources.testing-repo.name)/py
image: gcr.io/kubeflow-ci/test-worker-py3@sha256:b679ce5d7edbcc373fd7d28c57454f4f22ae987f200f601252b6dcca1fd8823b
name: deploy-gcp
volumeMounts:
- mountPath: /etc/podinfo
name: podinfo
workingDir: /workspace/$(inputs.resources.blueprint-repo.name)/kubeflow
volumes:
- downwardAPI:
items:
- fieldRef:
fieldPath: metadata.labels
path: labels
name: podinfo
Loading

0 comments on commit 0d18921

Please sign in to comment.