garden doesn't support directories as a test artifact

[lucas-sonnabend]

Bug

I wanted to use a folder as an artifact for tests

Current Behaviour

garden.yml extract:

    artifacts:
      - source: /tests/cypress/videos/*
        target: test-results/cypress/videos

error:

Command "/home/lucas/.garden/tools/kubectl/804c71b3e6ad5976/kubectl --context=garden-develop --namespace=garden-lucas-sonnabend exec -i 
test-cypress-e2e-6dd25b -c main -- sh -c cd / && touch .DS_Store && tar -c -f - $(ls tests/cypress/videos 2>/dev/null) .DS_Store" failed with
 code 2:

tar: extended: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
command terminated with exit code 2

Here's the full output:

tar: extended: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
.DS_Store0000644000000000000000000000000013642635722011237 0ustar  rootrootcommand terminated with exit code 2

Expected behavior

Either of two things: support a directory so that it treats all files/subdirecories within that as artifacts OR
fail with a much cleaner error message

Reproducible example

See description above

Workaround

using a wildcard works, like -source: /tests/cypress/videos/*

Your environment

$ garden version
0.11.11

kubectl version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.0", GitCommit:"70132b0f130acc0bed193d9ba59dd186f0e634cf", GitTreeState:"clean", BuildDate:"2019-12-07T21:20:10Z", GoVersion:"go1.13.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15+", GitVersion:"v1.15.10-eks-bac369", GitCommit:"bac3690554985327ae4d13e42169e8b1c2f37226", GitTreeState:"clean", BuildDate:"2020-02-26T01:12:54Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}

docker version
Client: Docker Engine - Community
 Version:           19.03.6
 API version:       1.40
 Go version:        go1.12.16
 Git commit:        369ce74a3c
 Built:             Thu Feb 13 01:27:52 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.6
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.16
  Git commit:       369ce74a3c
  Built:            Thu Feb 13 01:26:25 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.10
  GitCommit:        b34a5c8af56e510852c35414db4c1f4fa6172339
 runc:
  Version:          1.0.0-rc8+dev
  GitCommit:        3e425f80a8c931f88e6d94a8c831b9d5aa481657
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

[edvald]

I think this should work as you describe it, so we should fix that (as opposed to give a better error message). Will take a look.

[stale]

This issue has been automatically marked as stale because it hasn't had any activity in 60 days. It will be closed in 14 days if no further activity occurs (e.g. changing labels, comments, commits, etc.). Please feel free to tag a maintainer and ask them to remove the label if you think it doesn't apply. Thank you for submitting this issue and helping make Garden a better product!

[stale]

This issue has been automatically marked as stale because it hasn't had any activity in 60 days. It will be closed in 14 days if no further activity occurs (e.g. changing labels, comments, commits, etc.). Please feel free to tag a maintainer and ask them to remove the label if you think it doesn't apply. Thank you for submitting this issue and helping make Garden a better product!

[stale]

This issue has been automatically marked as stale because it hasn't had any activity in 60 days. It will be closed in 14 days if no further activity occurs (e.g. changing labels, comments, commits, etc.). Please feel free to tag a maintainer and ask them to remove the label if you think it doesn't apply. Thank you for submitting this issue and helping make Garden a better product!

[lucas-sonnabend]

Can you re-open this please?
We just ran into that again. There is a workaround, but that is rather cumbersome and easy to forget.

My workaround is to use a shell script that runs after the test/task and flattens all files into subfolder to be files:

# find all videos and copy them over, flattening the files (example test/bla/video.mp3 to test_bla_video.mp3)
find /tests/cypress/videos -type f  | sed 's/$/"/;s/^/"/;h;s/^..//;s/^/"/;y/\//_/;H;g;s/\n/ /g;s/^/cp /' | sh

[lucas-sonnabend]

A colleague just found a nicer workaround, if all the files you are copying have the same format/suffix:

artifacts:
      - source: /frontend/apps/cypress/cypress/snapshots/**/*.png
        target: test-results/frontend-cypress

I guess that means the glob for source is used to get all matching folders and files and pipes them to tar individually.
And that command then fails on the folders.
It should work if you filter out the folders from the glob result

[anderjan]

Any updates on this?
It would be great if we could have subdirectories in artifacts to keep the original structure of some outputs preserved in the artifact, basically like a recursive copy of the specified source directory of the artifact to the target directory. Would that be possible?

[edvald]

I'll take a look 👍 Need to write some more test cases and make sure they behave generally as we'd expect.

[edvald]

There we go: #2522