Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only have one set of examples and test that they are valid #109

Merged
merged 4 commits into from
Oct 9, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,8 +144,11 @@ as inputs and outputs of a `TaskRun`.

* `Resources` is created directly in a pipeline configuration and bound
to `TaskRun` as an input and/or output source.
* The (optional) `passedConstraint` key on an `input source` defines a set of previous task names.
* When the `passedConstraint` key is specified on an input source, only the version of
* The (optional) `passedConstraints` key on an `input source` defines a set of previous task names.
* When the `passedConstraints` key is specified on an input source, only the version of
the resource that passed through the defined list of tasks is used.
* The `passedConstraint` allows for `Tasks` to fan in and fan out, and ordering can be expressed explicitly
* The `passedConstraints` allows for `Tasks` to fan in and fan out, and ordering can be expressed explicitly
using this key since a task needing a resource from a another task would have to run after.

See [docs/pipeline-resources.md](./docs/pipeline-resources.md) for more detail about tyeps
of `Resources`.
4 changes: 2 additions & 2 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@
This directory contains examples of [the Pipeline CRDs](../README.md) in action.

To deploy them to your cluster (after
[installing the CRDs and running the controller](../DEVELOPMENT.md#installing-andrunning)):
[installing the CRDs and running the controller](../DEVELOPMENT.md#getting-started)):

```bash
kubectl apply -f examples/pipelines
kubectl apply -f examples/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use ko here ? (like on some other knative projects ?)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vdemeester we could but I'm not sure we need to since there's nothing to build - even ko under the hood I think still calls out to kubectl (i think?) - but maybe there are some other benefits I'm missing?

kubectl apply -f examples/pipelines
kubectl apply -f examples/invocations
```

Expand Down
8 changes: 4 additions & 4 deletions examples/deploy_tasks.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,13 @@ spec:
value: string
- name: image
value: string
cluster:
- name: clusterName
clusters:
- name: targetCluster
buildSpec:
steps:
- name: deploy
image: kubernetes-helm
args: ['deploy', '--path=${pathToHelmChart}', '--set image=${image}', '${helmArgs}']
args: ['install', '--kube-context=${targetCluster}', '--set image=${image}', '${helmArgs}', '${pathToHelmChart}']

---
apiVersion: pipeline.knative.dev/v1alpha1
Expand All @@ -39,7 +39,7 @@ spec:
value: string
- name: pathToFiles
value: string
cluster:
clusters:
- name: targetCluster
buildSpec:
steps:
Expand Down
24 changes: 3 additions & 21 deletions examples/invocations/kritis-pipeline-run.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,29 +14,11 @@ status:
taskRuns:
- taskRef:
name: unit-test-kritis-12321312984
nextTasks:
- taskRef:
name: push-kritis-12321312984
prevTasks: []
- taskRef:
name: push-kritis-12321312984
nextTasks: []
prevTasks:
- taskRef:
name: unit-test-kritis-12321312984
conditions:
- type: Started
status: "True"
lastTransitionTime: "2018-10-04T12:25:39Z"
reason: manualTrigger
message: "Pipeline has been triggered manually"
- type: Completed
status: "True"
lastTransitionTime: "2018-10-04T13:25:39Z"
reason: done
message: "Pipeline execution has finished"
- type: Successful
- type: Succeeded
status: "False"
lastTransitionTime: "2018-10-04T13:25:39Z"
lastTransitionTime: "2018-10-04T12:25:39Z"
reason: taskFailure
message: "TaskRun `build-push-kritis-12321312984` had non-zero exit code"
message: "TaskRun `build-push-kritis-12321312984` had non-zero exit code"
41 changes: 9 additions & 32 deletions examples/invocations/run-kritis-test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,49 +9,26 @@ spec:
name: make
trigger:
triggerRef:
type: manual
type: PipelineRun
name: kritis-pipeline-run-12321312984
pipelineParamsRef:
name: pipelineparams-sample
inputs:
sources:
- name: 'kritis'
type: 'github'
url: 'github.com/grafeas/kritis'
branch: 'featureX'
commit: 'HEAD'
resourcesVersion:
- resourceRef:
name: kritis-resources-git
version: 4da79b91e8e37ea441cfe4972565e2184c1a127f
params:
- name: 'makeTarget'
type: 'string'
value: 'test'
results:
runs:
name: 'runsBucket'
type: 'gcs'
url: 'gcs://somebucket/results/runs'
logs:
name: 'logBucket'
type: 'gcs'
url: 'gcs://somebucket/results/logs'
tests:
name: 'testBucket'
type: 'gcs'
url: 'gcs://somebucket/results/tests'
status:
steps:
- name: make
logsURL: 'gcs://somebucket/results/tests/unit-test-kritis-feature-x/make'
exitCode: 0
conditions:
- type: Started
- type: Succeeded
status: "True"
lastTransitionTime: "2018-10-04T12:25:39Z"
reason: pipelineRun
message: "Task has been triggered by a Pipeline run"
- type: Completed
status: "True"
lastTransitionTime: "2018-10-04T13:25:39Z"
reason: done
message: "Pipeline execution has finished"
- type: Successful
status: "True"
lastTransitionTime: "2018-10-04T13:25:39Z"
reason: zeroExitCode
message: "All steps completed with an exit code of 0"
3 changes: 0 additions & 3 deletions examples/pipelineparams.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,11 @@ spec:
name: 'runsBucket'
type: 'gcs'
url: 'gcs://somebucket/results/runs'
token: 'todo'
logs:
name: 'logBucket'
type: 'gcs'
url: 'gcs://somebucket/results/logs'
token: 'todo'
tests:
name: 'testBucket'
type: 'gcs'
url: 'gcs://somebucket/results/tests'
token: 'todo'
18 changes: 9 additions & 9 deletions examples/pipelines/guestbook.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -45,21 +45,21 @@ spec:
key: workspace
resourceRef:
name: guestbook-resources-git
passedConstraint:
passedConstraints:
- build-guestbook
- build-redis
- name: workspace
key: redis-docker
resourceRef:
name: guestbook-resources-redis-docker
passedConstraint:
passedConstraints:
- build-push
params:
- name: pathToFiles
value: guestbook/all-in-one/guestbook-all-in-one.yaml
clusterBindings:
- inputName: test
key: test
- inputName: clusterName
key: testCluster
- name: int-test-osx # 3.a Run Integration tests for osx
taskRef:
name: integrationTestInDocker
Expand All @@ -68,7 +68,7 @@ spec:
key: workspace
resourceRef:
name: guestbook-resources-git
passedConstraint:
passedConstraints:
- deploy-bundle-test
params:
- name: dockerBuildFile
Expand All @@ -81,7 +81,7 @@ spec:
key: workspace
resourceRef:
name: guestbook-resources-git
passedConstraint:
passedConstraints:
- deploy-bundle-test
params:
- name: dockerBuildFile
Expand All @@ -94,12 +94,12 @@ spec:
key: workspace
resourceRef:
name: guestbook-resources-git
passedConstraint:
passedConstraints:
- int-test-osx
- int-test-linux
params:
- name: pathToFiles
value: guestbook/all-in-one/guestbook-all-in-one.yaml
clusterBindings:
- inputName: staging
key: staging
- inputName: targetCluster
key: stagingCluster
10 changes: 5 additions & 5 deletions examples/pipelines/kritis.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ spec:
key: workspace # bind to the name in the task
resourceRef:
name: kritis-resources-git
passedConstraint:
passedConstraints:
- unit-test-make
outputSourceBindings:
- name: kritisImage
Expand All @@ -42,13 +42,13 @@ spec:
key: workspace
resourceRef:
name: kritis-resources-image
passedConstraint: [build-push]
passedConstraints: [build-push]
params:
- name: pathToHelmCharts
value: kritis-charts
clusterBindings:
- inputName: test
key: test
- inputName: targetCluster
key: testCluster
- name: integration-test # 4. Run Integration Tests in test cluster
taskRef:
name: integration-test-in-docker
Expand All @@ -57,7 +57,7 @@ spec:
key: workspace
resourceRef:
name: kritis-resources-test-git
passedConstraint: [deploy-with-helm]
passedConstraints: [deploy-with-helm]
params:
- name: testArgs
value: "-e REMOTE_INTEGRATION=true"
35 changes: 35 additions & 0 deletions examples/smoke-test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
#!/bin/bash

# Copyright 2018 The Knative Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# smoke_test.sh will attempt to deploy all of the example CRDs to whatever
# is the current kubectl cluster context.
# It will not wait for any Runs to complete and instead will destroy the CRDs
# immediately after creation, so at the moment this pretty much just makes
# sure the structure is valid (and this might even have weird side effects..).

set -o xtrace
set -o errexit
set -o pipefail

kubectl apply -f examples/
kubectl apply -f examples/pipelines
kubectl apply -f examples/invocations

sleep 5

kubectl delete -f examples/invocations
kubectl delete -f examples/pipelines
kubectl delete -f examples/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one advantage of using IDE, it adds new line :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i wonder if i need to turn an extension on or something 🤔

19 changes: 13 additions & 6 deletions examples/test_tasks.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -48,14 +48,21 @@ spec:
- name: runTests
image: '${testImage}'
args: ['${testArgs}']
volumeMounts:
- name: gac
mountPath: gac.json
- name: cloudconfig
mountPath: /root/.config/gcloud
- name: dockerSocket
mountPath: /var/run/docker.sock
volumes:
- name: gac
source: ${workspace}/config/gac.json
destination: gac.json
hostPath:
path: ${workspace}/config/gac.json
- name: cloudconfig
source: ${workspace}/config/gcloud
destination: /root/.config/gcloud
hostPath:
path: ${workspace}/config/gcloud
- name: dockerSocket
source: /var/run/docker.sock
destination: /var/run/docker.sock
hostPath:
path: /var/run/docker.sock

3 changes: 3 additions & 0 deletions pkg/apis/pipeline/v1alpha1/pipeline_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,9 @@ type PipelineTask struct {
InputSourceBindings []SourceBinding `json:"inputSourceBindings,omitempty"`
// +optional
OutputSourceBindings []SourceBinding `json:"outputSourceBindings,omitempty"`
// TODO(#68) Cluster should become a type of Resource
// +optional
ClusterBindings []ClusterBinding `json:"clusterBindings,omitempty"`
// +optional
Params []Param `json:"params,omitempty"`
}
Expand Down
10 changes: 10 additions & 0 deletions pkg/apis/pipeline/v1alpha1/resource_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,16 @@ type PipelineResourceSpec struct {
Params []Param `json:"params"`
}

// TaskResource defines an input or output Resource declared as a requirement
// by a Task. The Name field will be used to refer to these Resources within
// the Task definition, and when provided as an Input, the Name will be the
// path to the volume mounted containing this Resource as an input (e.g.
// an input Resource named `workspace` will be mounted at `/workspace`).
type TaskResource struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

Name string `json:"name"`
Type PipelineResourceType `json:"type"`
}

// +genclient
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

Expand Down
19 changes: 3 additions & 16 deletions pkg/apis/pipeline/v1alpha1/task_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -69,27 +69,14 @@ type Task struct {
// Inputs are the requirements that a task needs to run a Build.
type Inputs struct {
// +optional
Sources []Source `json:"resources,omitempty"`
Resources []TaskResource `json:"resources,omitempty"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

// +optional
Params []Param `json:"params,omitempty"`
// TODO(#68) a cluster and/or deployment should be a type of Resource
// +optional
Clusters []Cluster `json:"clusters,omitempty"`
}

// Source is data which is required by a Build/Task for context
// (e.g. a repo from which to build an image). The name of the input will be
// used as the name of the volume containing this context which will be mounted
// into the container executed by the Build/Task, e.g. a Source with the
// name "workspace" would be mounted into "/workspace".
//
// TODO(#62): Something is wrong here, this should be a reference to a resource,
// could just be that the names and comments are out of date.
type Source struct {
// name of the source should match the name of the SourceBinding in the pipeline
Name string `json:"name"`
Type PipelineResourceType `json:"type"`
}

// Param defines arbitrary parameters needed by a task beyond typed inputs
// such as resources.
type Param struct {
Expand All @@ -103,7 +90,7 @@ type Outputs struct {
// +optional
Results []TestResult `json:"results,omitempty"`
// +optional
Sources []Source `json:"resources,omitempty"`
Resources []TaskResource `json:"resources,omitempty"`
}

// TestResult allows a task to specify the location where test logs
Expand Down
Loading