skaffold build fails if a Dockerfile includes a COPY with a wildcard

[jstrachan]

Expected behavior

Skaffold should build dockerfiles which use wildcards like this one (we kinda need wildcards for java)

FROM openjdk:8-jdk-slim
ENV PORT 8080
ENV CLASSPATH /opt/lib
EXPOSE 8080

# copy pom.xml and wildcards to avoid this command failing if there's no target/lib directory
COPY pom.xml target/lib* /opt/lib/

# NOTE we assume there's only 1 jar in the target dir
# but at least this means we don't have to guess the name
# we could do with a better way to know the name - or to always create an app.jar or something
COPY target/*.jar /opt/app.jar
WORKDIR /opt
CMD ["java", "-jar", "app.jar"]

Actual behavior

skaffold fails when it reaches the COPY with a wildcard:

bash-4.2# skaffold build -f skaffold.yaml
Starting build...
Sending build context to Docker daemon   5.12kB
Step 1/8 : FROM openjdk:8-jdk-slim
 ---> 954f983e50de
Step 2/8 : ENV PORT 8080
 ---> Using cache
 ---> 4276e5bee3e9
Step 3/8 : ENV CLASSPATH /opt/lib
 ---> Using cache
 ---> c2b3a6bdd877
Step 4/8 : EXPOSE 8080
 ---> Using cache
 ---> fa1f10e95851
Step 5/8 : COPY pom.xml target/lib* /opt/lib/
 ---> Using cache
 ---> df4796c8ab0b
Step 6/8 : COPY target/*.jar /opt/app.jar
FATA[0000] build step: running build for artifact: running build: No source files were specified

If I edit the Dockerfile to replace the *.jar with a concrete file name then it all works

Information

• Skaffold version: version...

master

• Operating system: ...

GKE pod running linux

• Contents of skaffold.yaml:

kind: Config
build:
  tagPolicy:
    envTemplate:
      template: "{{.DOCKER_REGISTRY}}/jstrachan/demo97:{{.VERSION}}"
  artifacts:
  - imageName: changeme
    workspace: .
    docker:
      dockerfilePath: Dockerfile
  local: {}
deploy:
  kubectl:
    manifests:
profiles:
- name: dev
  build:
    tagPolicy:
      envTemplate:
        template: "{{.DOCKER_REGISTRY}}/jstrachan/demo97:{{.DIGEST_HEX}}"
    artifacts:
    - docker: {}
    local: {}
  deploy:
    helm:
      releases:
      - name: demo97
        chartPath: charts/demo97
        setValueTemplates:
          image.repository: "{{.DOCKER_REGISTRY}}/jstrachan/demo97"
          image.tag: "{{.DIGEST_HEX}}"

Steps to reproduce the behavior

I'm using Jenkins X to test the above out but you should be able to reproduce on the CLI on a linux box.

Here's a sample project you can use to reproduce: https://github.com/jstrachan/demo97

then set the VERSION & DOCKER_REGISTRY env vars to something. e.g.

export VERSION="0.0.1"
export DOCKER_REGISTRY="gcr.io"

[jstrachan]

BTW this didn't use to be an issue for 0.4.x to 0.5.x of Skaffold so I guess something's changed in the handling of source

[dcherniv]

@jstrachan i cant reproduce on skaffold 0.7.0
skaffold version, on debian buster

dcherniv@debbie:~/Documents/$ skaffold-vanilla version
v0.7.0
dcherniv@debbie:~/Documents/$ 

skaffold-vanilla build

[...]
Step 13/24 : COPY gradle.properties /
 ---> f0aed681392c
Step 14/24 : COPY lib/*.jar /opt/app.jar
 ---> 068beefc1c80
[...]

Addendum. Can reproduce with your sample project.

[jstrachan]

@dcherniv I just reproduced on skaffold 0.7.0:

bash-4.2# ./skaffold-linux-amd64 version
v0.7.0
bash-4.2#  ./skaffold-linux-amd64 build -f sk.yml
Starting build...
Sending build context to Docker daemon   5.12kB
Step 1/8 : FROM openjdk:8-jdk-slim
 ---> 954f983e50de
Step 2/8 : ENV PORT 8080
 ---> Using cache
 ---> 4276e5bee3e9
Step 3/8 : ENV CLASSPATH /opt/lib
 ---> Using cache
 ---> c2b3a6bdd877
Step 4/8 : EXPOSE 8080
 ---> Using cache
 ---> fa1f10e95851
Step 5/8 : COPY pom.xml target/lib* /opt/lib/
 ---> Using cache
 ---> df4796c8ab0b
Step 6/8 : COPY target/*.jar /opt/app.jar
FATA[0000] build step: running build for artifact: running build: No source files were specified
bash-4.2# cat sk.yml
apiVersion: skaffold/v1alpha2
kind: Config
build:
  tagPolicy:
    envTemplate:
      template: "10.55.246.66:5000/jstrachan/demo97:1.2.3"
  artifacts:
  - imageName: changeme
    workspace: .
    docker:
      dockerfilePath: Dockerfile
  local: {}
deploy:
  kubectl:
    manifests:
profiles:
- name: dev
  build:
    tagPolicy:
      envTemplate:
        template: "10.55.246.66:5000/jstrachan/demo97:{{.DIGEST_HEX}}"
    artifacts:
    - docker: {}
    local: {}
bash-4.2# cat Dockerfile
FROM openjdk:8-jdk-slim
ENV PORT 8080
ENV CLASSPATH /opt/lib
EXPOSE 8080

# copy pom.xml and wildcards to avoid this command failing if there's no target/lib directory
COPY pom.xml target/lib* /opt/lib/

# NOTE we assume there's only 1 jar in the target dir
# but at least this means we don't have to guess the name
# we could do with a better way to know the name - or to always create an app.jar or something
#COPY target/demo97-0.0.3.jar /opt/app.jar
COPY target/*.jar  /opt/app.jar
WORKDIR /opt
CMD ["java", "-jar", "app.jar"]

I just checked its nothing to do with the envTemplate stuff or profiles to be sure - the same failure with this yaml on 0.7.0:

apiVersion: skaffold/v1alpha2
kind: Config
build:
  artifacts:
  - imageName: changeme:1.2.3
    docker:
      dockerfilePath: Dockerfile
  local: {}
deploy:
  kubectl:
    manifests:

[jstrachan]

could it be dependent on the docker daemon version I wonder? I'm running the above inside a pod in GKE using kubernetes v1.8.10-gke.0

[jguo13]

@jstrachan is your gke pod running linux running on windows? cause it might be this issue #693

[jstrachan]

its running linux - this is vanilla GKE stuff

[dgageot]

Should be fixed in 0.8.0 with #712