Update docker/docker to master / v21.xx-dev (post libnetwork integration)

[thaJeztah]

follow-up to #2149

• relates to Post-libnetwork integration tasks moby#42471
• relates to Move libnetwork moby#42262

full diff: https://github.com/docker/docker/compare/v20.10.7..8dbd90ec00da

This applies the same / similar local changes to buildkit as were made in
the branch that moby is currently vendoring:
244e8cd...cpuguy83:update_libnetwork_import

Unfortunately, this (again) requires a "replace" rule (probably until we tagged
a release with this change). Go mod refuses to pick a version from master, and
(without the replace rule), does weird, reverting docker to a very old version,
predating the sirupsen rename:

> [vendored 4/4] RUN --mount=target=/src,rw   --mount=target=/go/pkg/mod,type=cache   go mod tidy && go mod vendor &&   mkdir /out && cp -r go.mod go.sum vendor /out:
#10 0.500 go: finding github.com/docker/docker 8dbd90ec00daa26dc45d7da2431c965dec99e8b4
#10 0.599 warning: ignoring symlink /src/examples/kube-consistent-hash
#10 1.027 go: github.com/moby/buildkit/util/testutil/integration imports
#10 1.027       github.com/docker/docker/testutil/daemon imports
#10 1.027   github.com/docker/docker/opts imports
#10 1.027   github.com/docker/libnetwork/ipamutils imports
#10 1.027   github.com/docker/libnetwork/osl imports
#10 1.027   github.com/Sirupsen/logrus: github.com/Sirupsen/logrus@v1.8.1: parsing go.mod:
#10 1.027   module declares its path as: github.com/sirupsen/logrus
#10 1.027           but was required as: github.com/Sirupsen/logrus
------
error: failed to solve: rpc error: code = Unknown desc = executor failed running [/bin/sh -c go mod tidy && go mod vendor &&   mkdir /out && cp -r go.mod go.sum vendor /out]: exit code: 1
make: *** [vendor] Error 1

[thaJeztah]

Rebased; this should also be ready for review now

@tonistiigi @cpuguy83 @AkihiroSuda

[cpuguy83]

LGTM

[thaJeztah]

w.r.t. "why do we need the replace rule" ?

I think the problem (or at least one of them) is that BuildKit depends on tonistiigi/fsutil:

• github.com/tonistiigi/fsutil@v0.0.0-20210525040343-5dfbf5db66b9
• which depends on github.com/docker/docker@v20.10.5 (see https://github.com/tonistiigi/fsutil/blob/5dfbf5db66b9/go.mod#L8)
  • This is "newer than" v20.10.3-0.20210608030740-0ad2293d0e5b+incompatible (AKA master)
  • So go.mod downgrades master to v20.10.5
• which depends on github.com/docker/libnetwork
• and because there's no go.mod in v20.10.5, it picks the latest "stable" release, which is libnetwork@v0.5.6 (there's some tags on that repo and this is the last tag that is not an -rc)
• which depends on github.com/Sirupsen/logrus (capital S) (in Godeps.json)
• and depends on github.com/docker/docker@v1.4.1 (in GoDeps.json)
  • So go.mod pick "latest `v1.x.x" version of docker/docker (v1.13.1)

[thaJeztah]

fsutil currently uses:

• github.com/docker/docker/pkg/fileutils
• github.com/docker/docker/pkg/archive (only in a test)
• github.com/docker/docker/pkg/reexec (only in a test)

The test could be moved to a submodule (have its own go.mod so that it doesn't affect project that use fsutil as a dependency)

For github.com/docker/docker/pkg/fileutils, it looks like fsutils (and "buildkit") is using the fileutils.NewPatternMatcher (used for .dockerignore, as well as COPY matching).

Docker uses fileutils for some other utilities in that package; fileutils.GetTotalUsedFds(), fileutils.ReadSymlinkedDirectory(), fileutils.CreateIfNotExists().

I think we could / should move the pattern matching to tonistiigi/fsutils, or BuildKit. Perhaps consider fsutils to be in the Moby org (?) WDYT?

[tonistiigi]

@thaJeztah So if fsutil dependency is changed to also point to master then this can be fixed? If you confirm this can be fixed as follow up I can merge current. If we would create a -dev tag in moby, would it help?

[thaJeztah]

I wanted to test that today, yes.

The swarmkit update was also revendored in moby, so perhaps that helps as well (will test when I get to my computer)

[thaJeztah]

@tonistiigi yes, that looks to be working; I opened tonistiigi/fsutil#104, and pushed a temporary commit to update fsutil from my branch).

(there's still a chance that once the replace for fsutil is removed, we'll end up with some transient dependencies on older versions, but probably not)

I think we should still look at #2150 (comment), but we can do so separately

[thaJeztah]

updated with latest fsutil (after tonistiigi/fsutil#104 was merged)

[tonistiigi]

I'll create a tracking issue for the replace rule.