Caching does not work with ARG inside the Dockerfile #1910
Comments
|
Comparing the composite keys I see: vs I am pretty sure that TEST=123 is wrong, this should always be only set after the ARG command ran. |
…oogleContainerTools#1910 This seems to be required because otherwise build & optimize calculate different caches keys.
|
For my simple docker file this fix works: diff --git a/pkg/executor/build.go b/pkg/executor/build.go
index 995c6737..45b3cae7 100644
--- a/pkg/executor/build.go
+++ b/pkg/executor/build.go
@@ -242,6 +242,13 @@ func (s *stageBuilder) optimize(compositeKey CompositeCache, cfg v1.Config) erro
return errors.Wrap(err, "failed to get files used from context")
}
+ // Mutate the config for any commands that require it.
+ if command.MetadataOnly() {
+ if err := command.ExecuteCommand(&cfg, s.args); err != nil {
+ return err
+ }
+ }
+
compositeKey, err = s.populateCompositeKey(command, files, compositeKey, s.args, cfg.Env)
if err != nil {
return err
@@ -272,13 +279,6 @@ func (s *stageBuilder) optimize(compositeKey CompositeCache, cfg v1.Config) erro
s.cmds[i] = cacheCmd
}
}
-
- // Mutate the config for any commands that require it.
- if command.MetadataOnly() {
- if err := command.ExecuteCommand(&cfg, s.args); err != nil {
- return err
- }
- }
}
return nil
}Since |
|
Yeah my fix breaks down as soon as I have the following dockerfile: with the following difference in the composite key: vs |
|
I'm running into exactly the same issue. In my case, the FROM alpine:latest
ARG ARG1="ARG1_VALUE"
ENV ENV1="ENV1_VALUE"
RUN echo "hi"I call $ /kaniko/executor --destination=my.registry.com/vinman1:latest --context . --dockerfile ./Dockerfile --force --cache=true --cache-copy-layers=true --cache-ttl=672h --cache-repo=my.registry.com/vinman1/cache --compressed-caching=false --verbosity debughere's the debug log. Most notably, like mentioned above, the
I share @apollo13 s opinion that the A good example of how it should work is the
|
|
Great work @apollo13 👍 I've applied your code change to a local fork of the executor and can confirm that it fixes the issue I described above. However, while testing, I noticed that there seems to be another issue... Lets take this second example FROM alpine:latest
ARG ARG3="ARG3_VALUE"
ARG ARG2="ARG2_VALUE"
ARG ARG1="ARG1_VALUE"
RUN echo "hi"Notice the ordering of the ARG statements. I've noticed that ARGs are not always added to the environment within the layers of a composite key in the same order. Here's a second debug log. In particular, look at the
As you can see, there seems to be some re-ordering applied to the last 2 layers of the composite key, but not consistently between the |
|
@vinman1 There seems to be a bug in the |
|
Ok. @apollo13, I've re-synced my local fork with your last changes and can confirm that they fix both the original problem and the above ordering issue for all my test cases. Again, great work 👏 |
Note to maintainers: Attached a patch in #1915 -- not sure how to test that though, hints welcome.
Actual behavior
I am running kaniko with
--cache-copy-layers=trueand it fails to utilize the cache in certain situations.Expected behavior
I expect kaniko to reuse the cache, it seems as if the cache ids are generated wrongly
To Reproduce
Additional Information
Looking at the debug output the cache key changes:
Triage Notes for the Maintainers
--cacheflagThe text was updated successfully, but these errors were encountered: