invalid not-modified ETag with local network ADD in docker

[vytools]

I have a local network tarball that I'm adding in a dockerfile.

FROM alpine
ADD https://10.10..../...tar.gz package

when I build the dockerfile with buildkit I get an "invalid not-modified ETag" error anytime I rebuild even when I remove the old image before building. I'm suspicious this is not a buildkit issue but I thought I'd check just in case. Any help would be much appreciated.

Just FYI it rebuilds fine when I don't use buildkit.

[tonistiigi]

I think this should be misbehaving http server. Your server should only return 304 if it matches one of the previous Etag values. And from the spec it is supposed to send the matched Etag value back with the 304 response https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-None-Match but atm it doesn't seem to do that.

[tonistiigi]

If you have a way to trace the requests between your server and buildkit we could be sure. If you do that start the trace with empty buildkit state.

[vytools]

I'm sorry I'm not sure what you mean by "start the trace with an empty buildkit state" could you expound a bit?

[tonistiigi]

If you have a way to trace the traffic (for example in your server or with a mitm proxy) start dockerd with empty /var/lib/docker before doing that. Otherwise, the requests could be using cache from your previous runs and we don't get the full picture.

[ezpuzz]

+1 on this issue, it's blocking me from using buildkit. remote server in my case is api.github.com if that helps in debugging.

[hmage]

Also having problems. Using it to invalidate cache if there are new commits to the repo as per solution posted here.

FROM golang

## this line invalidates cache if there are new commits
ADD https://api.github.com/repos/hmage/norm/compare/master...HEAD /dev/null
RUN git clone https://github.com/hmage/norm /norm

[rwe]

I think this is a duplicate of #905 (ADD from url fails with 'invalid not-modified ETag'). Or rather that is a duplicate of this, though that issue's title is more general. I don't think this is "local network"-specific.

Note that I've included an analysis of the cause and fix in that other issue thread.

[rwe]

I believe this can be closed now that #1159 is merged in. Not sure why GitHub didn't close this one when that was merged.