Skip to content

Commit

Permalink
feat(artifacts): retrieve subpath from unarchived ref artifact. Closes
Browse files Browse the repository at this point in the history
  • Loading branch information
AntoineDao authored Aug 31, 2020
1 parent 6a452cc commit fd3fca8
Show file tree
Hide file tree
Showing 20 changed files with 1,004 additions and 409 deletions.
4 changes: 4 additions & 0 deletions api/openapi-spec/swagger.json
Original file line number Diff line number Diff line change
Expand Up @@ -1934,6 +1934,10 @@
"s3": {
"description": "S3 contains S3 artifact location details",
"$ref": "#/definitions/io.argoproj.workflow.v1alpha1.S3Artifact"
},
"subPath": {
"description": "SubPath allows an artifact to be sourced from a subpath within the specified source",
"type": "string"
}
}
},
Expand Down
35 changes: 35 additions & 0 deletions docs/fields.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ Workflow is the definition of a workflow resource

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing.yaml)

- [`artifact-path-placeholders.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-path-placeholders.yaml)
Expand Down Expand Up @@ -335,6 +337,8 @@ WorkflowSpec is the specification of a Workflow.

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing.yaml)

- [`artifact-path-placeholders.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-path-placeholders.yaml)
Expand Down Expand Up @@ -671,6 +675,8 @@ CronWorkflowSpec is the specification of a CronWorkflow

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing.yaml)

- [`artifact-path-placeholders.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-path-placeholders.yaml)
Expand Down Expand Up @@ -970,6 +976,8 @@ WorkflowTemplateSpec is a spec of WorkflowTemplate.

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing.yaml)

- [`artifact-path-placeholders.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-path-placeholders.yaml)
Expand Down Expand Up @@ -1283,6 +1291,8 @@ Arguments to a template

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing.yaml)

- [`artifact-path-placeholders.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-path-placeholders.yaml)
Expand Down Expand Up @@ -1519,6 +1529,8 @@ Template is a reusable and composable unit of execution in a workflow

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing.yaml)

- [`artifact-path-placeholders.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-path-placeholders.yaml)
Expand Down Expand Up @@ -1906,6 +1918,8 @@ Outputs hold parameters, artifacts, and results from a step

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing.yaml)

- [`artifact-path-placeholders.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-path-placeholders.yaml)
Expand Down Expand Up @@ -1998,6 +2012,8 @@ Artifact indicates an artifact to place at a specified path

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing.yaml)

- [`artifact-path-placeholders.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-path-placeholders.yaml)
Expand Down Expand Up @@ -2060,6 +2076,7 @@ Artifact indicates an artifact to place at a specified path
|`path`|`string`|Path is the container path to the artifact|
|`raw`|[`RawArtifact`](#rawartifact)|Raw contains raw artifact location details|
|`s3`|[`S3Artifact`](#s3artifact)|S3 contains S3 artifact location details|
|`subPath`|`string`|SubPath allows an artifact to be sourced from a subpath within the specified source|

## Parameter

Expand Down Expand Up @@ -2401,6 +2418,8 @@ Inputs are the mechanism for passing parameters, artifacts, volumes from one tem

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing.yaml)

- [`artifact-path-placeholders.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-path-placeholders.yaml)
Expand Down Expand Up @@ -2559,6 +2578,8 @@ Pod metdata

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing.yaml)

- [`artifact-path-placeholders.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-path-placeholders.yaml)
Expand Down Expand Up @@ -2976,6 +2997,8 @@ WorkflowStep is a reference to a template to execute in a series of step

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing.yaml)

- [`artifactory-artifact.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifactory-artifact.yaml)
Expand Down Expand Up @@ -3239,6 +3262,8 @@ ArchiveStrategy describes how to archive files/directory when saving artifacts

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing-subpath.yaml)

- [`output-artifact-s3.yaml`](https://github.com/argoproj/argo/blob/master/examples/output-artifact-s3.yaml)
</details>

Expand Down Expand Up @@ -3766,6 +3791,8 @@ NoneStrategy indicates to skip tar process and upload the files or directory tre

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing-subpath.yaml)

- [`output-artifact-s3.yaml`](https://github.com/argoproj/argo/blob/master/examples/output-artifact-s3.yaml)
</details>

Expand Down Expand Up @@ -3826,6 +3853,8 @@ ObjectMeta is metadata that all persisted resources must have, which includes al

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing.yaml)

- [`artifact-path-placeholders.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-path-placeholders.yaml)
Expand Down Expand Up @@ -4348,6 +4377,8 @@ A single application container that you want to run within a pod.

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing.yaml)

- [`artifact-path-placeholders.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-path-placeholders.yaml)
Expand Down Expand Up @@ -4968,6 +4999,8 @@ PersistentVolumeClaimSpec describes the common attributes of storage devices and

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing.yaml)

- [`artifact-path-placeholders.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-path-placeholders.yaml)
Expand Down Expand Up @@ -5973,6 +6006,8 @@ ListMeta describes metadata that synthetic resources must have, including lists

- [`artifact-disable-archive.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-disable-archive.yaml)

- [`artifact-passing-subpath.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing-subpath.yaml)

- [`artifact-passing.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-passing.yaml)

- [`artifact-path-placeholders.yaml`](https://github.com/argoproj/argo/blob/master/examples/artifact-path-placeholders.yaml)
Expand Down
1 change: 1 addition & 0 deletions docs/swagger.md
Original file line number Diff line number Diff line change
Expand Up @@ -771,6 +771,7 @@ Artifact indicates an artifact to place at a specified path
| path | string | Path is the container path to the artifact | No |
| raw | [io.argoproj.workflow.v1alpha1.RawArtifact](#io.argoproj.workflow.v1alpha1.rawartifact) | Raw contains raw artifact location details | No |
| s3 | [io.argoproj.workflow.v1alpha1.S3Artifact](#io.argoproj.workflow.v1alpha1.s3artifact) | S3 contains S3 artifact location details | No |
| subPath | string | SubPath allows an artifact to be sourced from a subpath within the specified source | No |

#### io.argoproj.workflow.v1alpha1.ArtifactLocation

Expand Down
59 changes: 59 additions & 0 deletions examples/artifact-passing-subpath.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# This example demonstrates the ability to pass artifacts
# from one step to the next and filtering for a subdirectory
# key.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: artifact-passing-subpath-
spec:
entrypoint: artifact-example
templates:
- name: artifact-example
steps:
- - name: generate-artifact
template: whalesay
- - name: list-artifact
template: print-message-dir
arguments:
artifacts:
- name: message
from: "{{steps.generate-artifact.outputs.artifacts.hello-art}}"
- - name: consume-artifact
template: print-message
arguments:
artifacts:
- name: message
from: "{{steps.generate-artifact.outputs.artifacts.hello-art}}"
subPath: hello_world.txt

- name: whalesay
container:
image: docker/whalesay:latest
command: [sh, -c]
args: ["sleep 1; cowsay hello world | tee /tmp/hello_world.txt"]
outputs:
artifacts:
- name: hello-art
path: /tmp/
archive:
none: {}

- name: print-message-dir
inputs:
artifacts:
- name: message
path: /tmp/message
container:
image: alpine:latest
command: [sh, -c]
args: ["ls /tmp/message"]

- name: print-message
inputs:
artifacts:
- name: message
path: /tmp/message
container:
image: alpine:latest
command: [sh, -c]
args: ["cat /tmp/message"]
10 changes: 10 additions & 0 deletions manifests/base/crds/full/argoproj.io_clusterworkflowtemplates.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -573,6 +573,8 @@ spec:
- key
- secretKeySecret
type: object
subPath:
type: string
required:
- name
type: object
Expand Down Expand Up @@ -1688,6 +1690,8 @@ spec:
- key
- secretKeySecret
type: object
subPath:
type: string
required:
- name
type: object
Expand Down Expand Up @@ -2564,6 +2568,8 @@ spec:
- key
- secretKeySecret
type: object
subPath:
type: string
required:
- name
type: object
Expand Down Expand Up @@ -3513,6 +3519,8 @@ spec:
- key
- secretKeySecret
type: object
subPath:
type: string
required:
- name
type: object
Expand Down Expand Up @@ -3943,6 +3951,8 @@ spec:
- key
- secretKeySecret
type: object
subPath:
type: string
required:
- name
type: object
Expand Down
10 changes: 10 additions & 0 deletions manifests/base/crds/full/argoproj.io_cronworkflows.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -594,6 +594,8 @@ spec:
- key
- secretKeySecret
type: object
subPath:
type: string
required:
- name
type: object
Expand Down Expand Up @@ -1709,6 +1711,8 @@ spec:
- key
- secretKeySecret
type: object
subPath:
type: string
required:
- name
type: object
Expand Down Expand Up @@ -2585,6 +2589,8 @@ spec:
- key
- secretKeySecret
type: object
subPath:
type: string
required:
- name
type: object
Expand Down Expand Up @@ -3534,6 +3540,8 @@ spec:
- key
- secretKeySecret
type: object
subPath:
type: string
required:
- name
type: object
Expand Down Expand Up @@ -3964,6 +3972,8 @@ spec:
- key
- secretKeySecret
type: object
subPath:
type: string
required:
- name
type: object
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -316,6 +316,8 @@ spec:
- key
- secretKeySecret
type: object
subPath:
type: string
required:
- name
type: object
Expand Down
Loading

0 comments on commit fd3fca8

Please sign in to comment.