add support for oci-layout build-context

[deitch]

Adds to buildx the oci-layout: build-context capabilities added to buildkit in buildkit 2827.

Recommended by @tonistiigi . Remains incomplete, some things I wasn't sure about. Will comment inline.

[deitch]

Pulling in the updated buildkit lib updated a whole bunch of other things. go mod worked its magic. It all worked and compiled, so I didn't complain.

[tonistiigi]

The value in here nees to include digest that you are expecting in https://github.com/moby/buildkit/blob/master/frontend/dockerfile/builder/build.go#L892

[deitch]

Ah, that is it. I went through our buildkit PR, and could not recall which part did this. That helps.

[deitch]

Take a look, I think I got it right.

[tonistiigi]

I think we should allow this input without the digest as well in here. If no digest is set it can look up index.json inside this dir and see if it has one descriptor and then set it as the expected digest. If there is ambiguity then it can produce an error.

[deitch]

I am hesitant about this, as it wades into a deep unresolved issue around what actually should be in index.json. I have seen those that only include tags, others that include entire image refs, making the layout directory essentially into a full registry. I have implemented. both ways.

containerd does a bit of a hybrid: directory is a full registry, but tag->manifest-hash is in the boltdb.

Let's get this working with the hash required, as is, and then let's deal with supporting hash-less - both in buildkit and in buildx - as a separate iteration.

[deitch]

Most things resolved, one open to be sure that this is right, another subject to discussion.

[deitch]

Fixed the listing issue, and rebased on master to resolve conflicts. That seems to have opened up go mod hell. Hopefully won't be too bad to fix.

[deitch]

Fixed the go mod issues. Phew.

I am seeing vet issues with:

# github.com/docker/buildx/bake
bake/bake.go:885:69: cannot use os.Stderr (variable of type *os.File) as type *configfile.ConfigFile in argument to authprovider.NewDockerAuthProvider
# github.com/docker/buildx/bake
vet: bake/bake.go:885:69: cannot use os.Stderr (variable of type *os.File) as *configfile.ConfigFile value in argument to authprovider.NewDockerAuthProvider

I didn't think I changed those, but perhaps downstream dependencies?

[deitch]

Yeah, it looks like the definition of github.com/moby/buildkit/session/auth/authprovider.NewDockerAuthProvider() changed in buildkit from:

func NewDockerAuthProvider(stderr io.Writer) session.Attachable {
	return &authProvider{
		config:      config.LoadDefaultConfigFile(stderr),
		seeds:       &tokenSeeds{dir: config.Dir()},
		loggerCache: map[string]struct{}{},
	}
}

to

func NewDockerAuthProvider(cfg *configfile.ConfigFile) session.Attachable {
	return &authProvider{
		authConfigCache: map[string]*types.AuthConfig{},
		config:          cfg,
		seeds:           &tokenSeeds{dir: config.Dir()},
		loggerCache:     map[string]struct{}{},
	}
}

I got it.

[deitch]

Figured out the failing CI test, updated, should be good now.

[deitch]

How do we stand here @tonistiigi ?

[deitch]

Been almost a month. I just rebased.

Is there anything else we need to get this merged in?

[tonistiigi]

Update line 109 with OCI layout directory and add an intented subtitle for OCI layout for the new content.

[deitch]

Done. Back to you @tonistiigi