feat: allow exporting artifacts from task and test runs

Users can now specify artifacts to copy after task and test runs, for `exec`, `container` and `helm` modules. The artifacts are stored in `.garden/artifacts` and a link to each exported file (up to a limit of 5) is shown in the CLI after the test/task run.
garden-io · Nov 12, 2019 · a1e4c1c · a1e4c1c
1 parent 875dd8a
commit a1e4c1c
Show file tree

Hide file tree

Showing 87 changed files with 3,118 additions and 879 deletions.
diff --git a/.circleci/config.yml b/.circleci/config.yml
@@ -322,10 +322,15 @@ jobs:
   test-minikube:
     machine:
       image: 'ubuntu-1604:201903-01'
-      docker_layer_caching: true
+    parameters:
+      kubernetesVersion:
+        description: The Kubernetes version to run
+        type: string
     environment:
-      K8S_VERSION: v1.15.0
-      MINIKUBE_VERSION: v1.3.1
+      K8S_VERSION: <<parameters.kubernetesVersion>>
+      MINIKUBE_VERSION: v1.5.2
+      GARDEN_LOG_LEVEL: silly
+      GARDEN_LOGGER_TYPE: basic
     steps:
       - checkout
       - run:
@@ -354,10 +359,12 @@ jobs:
             sudo mv minikube /usr/local/bin
       - run:
           name: Start Minikube Cluster
-          command: sudo -E minikube start --vm-driver=none --cpus 2 --memory 4096 --kubernetes-version=$K8S_VERSION
+          command: |
+            sudo -E minikube start --vm-driver=none --cpus 2 --memory 4096 --kubernetes-version=$K8S_VERSION
+            sudo chown -R circleci:circleci /home/circleci/.kube /home/circleci/.minikube /etc/kubernetes
       - run:
           name: Integ tests
-          command: npm run integ
+          command: cd garden-service && npm run integ -- -b
       - run:
           name: Deploy demo-project
           command: sudo -E ./garden-service/bin/garden deploy --root examples/demo-project --logger-type basic
@@ -429,7 +436,10 @@ workflows:
       - test-dashboard:
           requires:
             - build
-      - test-minikube
+      - test-minikube:
+          kubernetesVersion: "1.16.2"
+      - test-minikube:
+          kubernetesVersion: "1.15.0"
       - test-windows:
           # Don't attempt to run dist tests for external PRs (they won't have access to the required keys)
           <<: *only-internal-prs
@@ -474,6 +484,10 @@ workflows:
             - test-dist
       - test-minikube:
           <<: *only-master
+          kubernetesVersion: "1.16.2"
+      - test-minikube:
+          <<: *only-master
+          kubernetesVersion: "1.15.0"
       - test-windows:
           <<: *only-master
           # Don't attempt to run dist tests for external PRs (they won't have access to the required keys)
@@ -517,6 +531,10 @@ workflows:
             - test-dist
       - test-minikube:
           <<: *only-tags
+          kubernetesVersion: "1.16.2"
+      - test-minikube:
+          <<: *only-tags
+          kubernetesVersion: "1.15.0"
       - test-windows:
           <<: *only-tags
           requires:

diff --git a/docs/reference/config.md b/docs/reference/config.md
@@ -652,7 +652,8 @@ POSIX-style path or filename of the directory or file(s) to copy to the target.
 
 [build](#build) > [dependencies](#builddependencies) > [copy](#builddependenciescopy) > target
 
-POSIX-style path or filename to copy the directory or file(s).
+POSIX-style path or filename to copy the directory or file(s), relative to the build directory.
+Defaults to to same as source path.
 
 | Type     | Required | Default                   |
 | -------- | -------- | ------------------------- |

diff --git a/docs/reference/module-types/container.md b/docs/reference/module-types/container.md
@@ -205,7 +205,8 @@ POSIX-style path or filename of the directory or file(s) to copy to the target.
 
 [build](#build) > [dependencies](#builddependencies) > [copy](#builddependenciescopy) > target
 
-POSIX-style path or filename to copy the directory or file(s).
+POSIX-style path or filename to copy the directory or file(s), relative to the build directory.
+Defaults to to same as source path.
 
 | Type     | Required | Default                   |
 | -------- | -------- | ------------------------- |
@@ -350,7 +351,7 @@ The list of services to deploy from this container module.
 
 [services](#services) > name
 
-Valid RFC1035/RFC1123 (DNS) label (may contain lowercase letters, numbers and dashes, must start with a letter and cannot end with a dash), cannot contain consecutive dashes or start with `garden` or be longer than 63 characters.
+Valid RFC1035/RFC1123 (DNS) label (may contain lowercase letters, numbers and dashes, must start with a letter, and cannot end with a dash), cannot contain consecutive dashes or start with `garden`, or be longer than 63 characters.
 
 | Type     | Required |
 | -------- | -------- |
@@ -728,7 +729,7 @@ Example:
 ```yaml
 services:
   - ports:
-      - containerPort: "8080"
+      - containerPort: 8080
 ```
 
 ### `services[].ports[].servicePort`
@@ -750,7 +751,7 @@ Example:
 ```yaml
 services:
   - ports:
-      - servicePort: "80"
+      - servicePort: 80
 ```
 
 ### `services[].ports[].hostPort`
@@ -873,11 +874,11 @@ Maximum duration (in seconds) of the test run.
 | -------- | -------- | ------- |
 | `number` | No       | `null`  |
 
-### `tests[].command[]`
+### `tests[].args[]`
 
-[tests](#tests) > command
+[tests](#tests) > args
 
-The command/entrypoint used to run the test inside the container.
+The arguments used to run the test inside the container.
 
 | Type            | Required |
 | --------------- | -------- |
@@ -887,16 +888,73 @@ Example:
 
 ```yaml
 tests:
-  - command:
-    - /bin/sh
-    - '-c'
+  - args:
+    - npm
+    - test
 ```
 
-### `tests[].args[]`
+### `tests[].artifacts[]`
 
-[tests](#tests) > args
+[tests](#tests) > artifacts
 
-The arguments used to run the test inside the container.
+Specify artifacts to copy out of the container after the run.
+Note: Depending on the provider, this may require the container image to include `sh` `tar`, in order to enable the file transfer.
+
+| Type            | Required |
+| --------------- | -------- |
+| `array[object]` | No       |
+
+Example:
+
+```yaml
+tests:
+  - artifacts:
+    - source: /report/**/*
+```
+
+### `tests[].artifacts[].source`
+
+[tests](#tests) > [artifacts](#testsartifacts) > source
+
+A POSIX-style path or glob to copy. Must be an absolute path. May contain wildcards.
+
+| Type     | Required |
+| -------- | -------- |
+| `string` | Yes      |
+
+Example:
+
+```yaml
+tests:
+  - artifacts:
+    - source: /report/**/*
+      - source: "/output/**/*"
+```
+
+### `tests[].artifacts[].target`
+
+[tests](#tests) > [artifacts](#testsartifacts) > target
+
+A POSIX-style path to copy the artifacts to, relative to the project artifacts directory.
+
+| Type     | Required | Default |
+| -------- | -------- | ------- |
+| `string` | No       | `"."`   |
+
+Example:
+
+```yaml
+tests:
+  - artifacts:
+    - source: /report/**/*
+      - target: "outputs/foo/"
+```
+
+### `tests[].command[]`
+
+[tests](#tests) > command
+
+The command/entrypoint used to run the test inside the container.
 
 | Type            | Required |
 | --------------- | -------- |
@@ -906,9 +964,9 @@ Example:
 
 ```yaml
 tests:
-  - args:
-    - npm
-    - test
+  - command:
+    - /bin/sh
+    - '-c'
 ```
 
 ### `tests[].env`
@@ -981,11 +1039,11 @@ Maximum duration (in seconds) of the task's execution.
 | -------- | -------- | ------- |
 | `number` | No       | `null`  |
 
-### `tasks[].command[]`
+### `tasks[].args[]`
 
-[tasks](#tasks) > command
+[tasks](#tasks) > args
 
-The command/entrypoint used to run the task inside the container.
+The arguments used to run the task inside the container.
 
 | Type            | Required |
 | --------------- | -------- |
@@ -995,16 +1053,73 @@ Example:
 
 ```yaml
 tasks:
-  - command:
-    - /bin/sh
-    - '-c'
+  - args:
+    - rake
+    - 'db:migrate'
 ```
 
-### `tasks[].args[]`
+### `tasks[].artifacts[]`
 
-[tasks](#tasks) > args
+[tasks](#tasks) > artifacts
 
-The arguments used to run the task inside the container.
+Specify artifacts to copy out of the container after the run.
+Note: Depending on the provider, this may require the container image to include `sh` `tar`, in order to enable the file transfer.
+
+| Type            | Required |
+| --------------- | -------- |
+| `array[object]` | No       |
+
+Example:
+
+```yaml
+tasks:
+  - artifacts:
+    - source: /report/**/*
+```
+
+### `tasks[].artifacts[].source`
+
+[tasks](#tasks) > [artifacts](#tasksartifacts) > source
+
+A POSIX-style path or glob to copy. Must be an absolute path. May contain wildcards.
+
+| Type     | Required |
+| -------- | -------- |
+| `string` | Yes      |
+
+Example:
+
+```yaml
+tasks:
+  - artifacts:
+    - source: /report/**/*
+      - source: "/output/**/*"
+```
+
+### `tasks[].artifacts[].target`
+
+[tasks](#tasks) > [artifacts](#tasksartifacts) > target
+
+A POSIX-style path to copy the artifacts to, relative to the project artifacts directory.
+
+| Type     | Required | Default |
+| -------- | -------- | ------- |
+| `string` | No       | `"."`   |
+
+Example:
+
+```yaml
+tasks:
+  - artifacts:
+    - source: /report/**/*
+      - target: "outputs/foo/"
+```
+
+### `tasks[].command[]`
+
+[tasks](#tasks) > command
+
+The command/entrypoint used to run the task inside the container.
 
 | Type            | Required |
 | --------------- | -------- |
@@ -1014,9 +1129,9 @@ Example:
 
 ```yaml
 tasks:
-  - args:
-    - rake
-    - 'db:migrate'
+  - command:
+    - /bin/sh
+    - '-c'
 ```
 
 ### `tasks[].env`
@@ -1112,16 +1227,22 @@ tests:
   - name:
     dependencies: []
     timeout: null
-    command:
     args:
+    artifacts:
+      - source:
+        target: .
+    command:
     env: {}
 tasks:
   - name:
     description:
     dependencies: []
     timeout: null
-    command:
     args:
+    artifacts:
+      - source:
+        target: .
+    command:
     env: {}
 ```