Skip to content

Commit

Permalink
Implement cache using docker's --cache-from flag
Browse files Browse the repository at this point in the history
With docker 1.13 a new build flag was added: --cache-from
This flag deprecated the buildcache tool that we used with docker 1.12

Details: moby/moby#26839
  • Loading branch information
databus23 committed Mar 13, 2017
1 parent 4766610 commit fe92ff5
Show file tree
Hide file tree
Showing 256 changed files with 6 additions and 26,891 deletions.
2 changes: 0 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@ FROM concourse/buildroot:iptables
ADD docker/ /usr/local/bin/
RUN /usr/local/bin/docker --version

ADD buildcache /usr/local/bin/buildcache

ADD assets/ /opt/resource/

ADD ecr-login /usr/local/bin/docker-credential-ecr-login
146 changes: 2 additions & 144 deletions Godeps/Godeps.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 0 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,12 +137,6 @@ version is the image's digest.
first pull `image:tag` from the Docker registry (so as to use cached
intermediate images when building). This will cause the resource to fail
if it is set to `true` and the image does not exist yet.
Note: Since docker 1.10 docker images [do not contain all necessary metadata to
restore the build cache](https://github.com/docker/docker/issues/20316).
Additional metadata needs to be saved and re-applied after a docker pull to have
subsequent builds skip identical intermediate layers. This additional
metadata is stored as a very small separate image (`image:${cache_tag}-buildcache`)
in the repository of this resource.

* `cache_tag`: *Optional.* Default `tag`. The specific tag to pull before
building when `cache` parameter is set. Instead of pulling the same tag
Expand Down
15 changes: 3 additions & 12 deletions assets/out
Original file line number Diff line number Diff line change
Expand Up @@ -115,13 +115,10 @@ elif [ -n "$build" ]; then
"$(cat "${load_base}/repository"):$(cat "${load_base}/tag")"
fi

cache_from=""
if [ "$cache" = "true" ]; then
docker_pull "${repository}:${cache_tag}"
docker create \
--cidfile=/tmp/container.cid \
"${repository}:${cache_tag}-buildcache" whatever && \
docker cp $(cat /tmp/container.cid):/buildcache.tgz /tmp/buildcache.tgz && \
docker load -i /tmp/buildcache.tgz
cache_from="--cache-from ${repository}:${cache_tag}"
fi

expanded_build_args=""
Expand All @@ -133,7 +130,7 @@ elif [ -n "$build" ]; then
expanded_build_args=${expanded_build_args}" "$(jq -r 'with_entries(.key |= "--build-arg " + . )|with_entries(.key = .key + "=" +.value)|keys|join(" ")' <$build_args_file)
fi

docker build -t "${repository}:${tag_name}" $expanded_build_args -f "$dockerfile" "$build"
docker build -t "${repository}:${tag_name}" $expanded_build_args -f "$dockerfile" $cache_from "$build"
elif [ -n "$load_file" ]; then
if [ -n "$load_repository" ]; then
docker load -i "$load_file"
Expand Down Expand Up @@ -181,12 +178,6 @@ if [ "$need_tag_as_latest" = "true" ] && [ "${tag_name}" != "latest" ] ; the
echo "${repository}:${tag_name} tagged as latest"
fi

if [ -n "$build" ] && [ "$cache" = "true" ]; then
buildcache save -o buildcache.tgz "${repository}:${tag_name}"
tar c buildcache.tgz | docker import - "${repository}:${cache_tag}-buildcache"
docker push "${repository}:${cache_tag}-buildcache"
fi

jq -n "{
version: {
digest: $(echo $digest | jq -R .)
Expand Down
2 changes: 0 additions & 2 deletions ci/scripts/build
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,6 @@ go build -o ./assets/print-metadata ./cmd/print-metadata/

curl -sSL -O "https://get.docker.com/builds/Linux/x86_64/docker-${DOCKER_VERSION}.tgz"
tar zxf "docker-${DOCKER_VERSION}.tgz"
go build -o ./buildcache github.com/concourse/docker-image-resource/vendor/github.com/tonistiigi/buildcache/cmd/buildcache


go build -o ./ecr-login github.com/concourse/docker-image-resource/vendor/github.com/awslabs/amazon-ecr-credential-helper/ecr-login/cmd

Expand Down
3 changes: 1 addition & 2 deletions scripts/deps
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,4 @@
godep save \
./... \
github.com/onsi/ginkgo/ginkgo \
github.com/awslabs/amazon-ecr-credential-helper/ecr-login/cmd \
github.com/tonistiigi/buildcache/cmd/buildcache
github.com/awslabs/amazon-ecr-credential-helper/ecr-login/cmd
1 change: 0 additions & 1 deletion vendor/github.com/Microsoft/go-winio/.gitignore

This file was deleted.

22 changes: 0 additions & 22 deletions vendor/github.com/Microsoft/go-winio/LICENSE

This file was deleted.

22 changes: 0 additions & 22 deletions vendor/github.com/Microsoft/go-winio/README.md

This file was deleted.

Loading

0 comments on commit fe92ff5

Please sign in to comment.