Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix containerimage multi-platform unpack #3982

Merged
merged 2 commits into from
Jul 6, 2023
Merged

Fix containerimage multi-platform unpack #3982

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ff62ebf
tests: add unpack to scratch export test
jedevc Jun 30, 2023
535eac9
exporter: allow unpack on multi-platform images
jedevc Jun 30, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions client/client_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1077,7 +1077,7 @@
Exports: []ExportEntry{
{
Type: exp,
Attrs: map[string]string{},

Check failure on line 1080 in client/client_test.go

View workflow job for this annotation

GitHub Actions / test / run (./client, containerd, nydus, integration)

Failed: client/TestIntegration/TestSecurityModeSysfs/worker=containerd/secmode=insecure 

=== RUN   TestIntegration/TestSecurityModeSysfs/worker=containerd/secmode=insecure
=== PAUSE TestIntegration/TestSecurityModeSysfs/worker=containerd/secmode=insecure
=== CONT  TestIntegration/TestSecurityModeSysfs/worker=containerd/secmode=insecure
    client_test.go:1080: 
        	Error Trace:	/src/client/client_test.go:1080
        	            				/src/util/testutil/integration/run.go:90
        	            				/src/util/testutil/integration/run.go:204
        	Error:      	Received unexpected error:
        	            	process "mkdir /sys/fs/cgroup/securitytest" did not complete successfully: exit code: 1
        	            	github.com/moby/buildkit/util/stack.Enable
        	            		/src/util/stack/stack.go:77
        	            	github.com/moby/buildkit/util/grpcerrors.FromGRPC
        	            		/src/util/grpcerrors/grpcerrors.go:198
        	            	github.com/moby/buildkit/util/grpcerrors.UnaryClientInterceptor
        	            		/src/util/grpcerrors/intercept.go:41
        	            	google.golang.org/grpc.(*ClientConn).Invoke
        	            		/src/vendor/google.golang.org/grpc/call.go:35
        	            	github.com/moby/buildkit/api/services/control.(*controlClient).Solve
        	            		/src/api/services/control/control.pb.go:2208
        	            	github.com/moby/buildkit/client.(*Client).solve.func2
        	            		/src/client/solve.go:258
        	            	golang.org/x/sync/errgroup.(*Group).Go.func1
        	            		/src/vendor/golang.org/x/sync/errgroup/errgroup.go:75
        	            	runtime.goexit
        	            		/usr/local/go/src/runtime/asm_amd64.s:1598
        	            	failed to solve
        	            	github.com/moby/buildkit/client.(*Client).solve.func2
        	            		/src/client/solve.go:273
        	            	golang.org/x/sync/errgroup.(*Group).Go.func1
        	            		/src/vendor/golang.org/x/sync/errgroup/errgroup.go:75
        	            	runtime.goexit
        	            		/usr/local/go/src/runtime/asm_amd64.s:1598
        	Test:       	TestIntegration/TestSecurityModeSysfs/worker=containerd/secmode=insecure
    sandbox.go:285: stdout: /usr/bin/containerd --config /tmp/bktest_containerd1463936488/config.toml
    sandbox.go:285: stderr: /usr/bin/containerd --config /tmp/bktest_containerd1463936488/config.toml
    sandbox.go:288: > startCmd 2023-06-30 14:13:46.28148286 +0000 UTC m=+430.803366258 /usr/bin/containerd --config /tmp/bktest_containerd1463936488/config.toml
    sandbox.go:288: time="2023-06-30T14:13:46Z" level=warning msg="containerd config version `1` has been deprecated and will be removed in containerd v2.0, please switch to version `2`, see https://github.com/containerd/containerd/blob/main/docs/PLUGINS.md#version-header"
    sandbox.go:288: time="2023-06-30T14:13:46.308990502Z" level=info msg="starting containerd" revision=1677a17964311325ed1c31e2c0a3589ce6d5c30d version=v1.7.1
    sandbox.go:288: time="2023-06-30T14:13:46.331439718Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.aufs\"..." type=io.containerd.snapshotter.v1
    sandbox.go:288: time="2023-06-30T14:13:46.338190853Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.aufs\"..." error="aufs is not supported (modprobe aufs failed: exit status 1 \"modprobe: can't change directory to '/lib/modules': No such file or directory\\n\"): skip plugin" type=io.containerd.snapshotter.v1
    sandbox.go:288: time="2023-06-30T14:13:46.338219653Z" level=info msg="loading plugin \"io.containerd.content.v1.content\"..." type=io.containerd.content.v1
    sandbox.go:288: time="2023-06-30T14:13:46.338316154Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.native\"..." type=io.containerd.snapshotter.v1
    sandbox.go:288: time="2023-06-30T14:13:46.338375054Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.overlayfs\"..." type=io.containerd.snapshotter.v1
    sandbox.go:288: time="2023-06-30T14:13:46.338537455Z" level=info msg="loading plugin \"io.containerd.snapshot

Check failure on line 1080 in client/client_test.go

View workflow job for this annotation

GitHub Actions / test / run (./client, oci, nydus, integration)

Failed: client/TestIntegration/TestSecurityModeSysfs/worker=oci/secmode=insecure 

=== RUN   TestIntegration/TestSecurityModeSysfs/worker=oci/secmode=insecure
=== PAUSE TestIntegration/TestSecurityModeSysfs/worker=oci/secmode=insecure
=== CONT  TestIntegration/TestSecurityModeSysfs/worker=oci/secmode=insecure
    client_test.go:1080: 
        	Error Trace:	/src/client/client_test.go:1080
        	            				/src/util/testutil/integration/run.go:90
        	            				/src/util/testutil/integration/run.go:204
        	Error:      	Received unexpected error:
        	            	process "mkdir /sys/fs/cgroup/securitytest" did not complete successfully: exit code: 1
        	            	github.com/moby/buildkit/util/stack.Enable
        	            		/src/util/stack/stack.go:77
        	            	github.com/moby/buildkit/util/grpcerrors.FromGRPC
        	            		/src/util/grpcerrors/grpcerrors.go:198
        	            	github.com/moby/buildkit/util/grpcerrors.UnaryClientInterceptor
        	            		/src/util/grpcerrors/intercept.go:41
        	            	google.golang.org/grpc.(*ClientConn).Invoke
        	            		/src/vendor/google.golang.org/grpc/call.go:35
        	            	github.com/moby/buildkit/api/services/control.(*controlClient).Solve
        	            		/src/api/services/control/control.pb.go:2208
        	            	github.com/moby/buildkit/client.(*Client).solve.func2
        	            		/src/client/solve.go:258
        	            	golang.org/x/sync/errgroup.(*Group).Go.func1
        	            		/src/vendor/golang.org/x/sync/errgroup/errgroup.go:75
        	            	runtime.goexit
        	            		/usr/local/go/src/runtime/asm_amd64.s:1598
        	            	failed to solve
        	            	github.com/moby/buildkit/client.(*Client).solve.func2
        	            		/src/client/solve.go:273
        	            	golang.org/x/sync/errgroup.(*Group).Go.func1
        	            		/src/vendor/golang.org/x/sync/errgroup/errgroup.go:75
        	            	runtime.goexit
        	            		/usr/local/go/src/runtime/asm_amd64.s:1598
        	Test:       	TestIntegration/TestSecurityModeSysfs/worker=oci/secmode=insecure
    sandbox.go:285: stdout: /usr/bin/buildkitd --oci-worker=true --containerd-worker=false --oci-worker-gc=false --oci-worker-labels=org.mobyproject.buildkit.worker.sandbox=true --config=/tmp/bktest_config2082401416/buildkitd.toml --root /tmp/bktest_buildkitd2498608975 --addr unix:///tmp/bktest_buildkitd2498608975/buildkitd.sock --debug
    sandbox.go:285: stderr: /usr/bin/buildkitd --oci-worker=true --containerd-worker=false --oci-worker-gc=false --oci-worker-labels=org.mobyproject.buildkit.worker.sandbox=true --config=/tmp/bktest_config2082401416/buildkitd.toml --root /tmp/bktest_buildkitd2498608975 --addr unix:///tmp/bktest_buildkitd2498608975/buildkitd.sock --debug
    sandbox.go:288: > startCmd 2023-06-30 14:10:33.813836217 +0000 UTC m=+245.129392831 /usr/bin/buildkitd --oci-worker=true --containerd-worker=false --oci-worker-gc=false --oci-worker-labels=org.mobyproject.buildkit.worker.sandbox=true --config=/tmp/bktest_config2082401416/buildkitd.toml --root /tmp/bktest_buildkitd2498608975 --addr unix:///tmp/bktest_buildkitd2498608975/buildkitd.sock --debug
    sandbox.go:288: time="2023-06-30T14:10:33Z" level=info msg="auto snapshotter: using overlayfs"
    sandbox.go:288: time="2023-06-30T14:10:33Z" level=debug msg="could not read \"/tmp/bktest_buildkitd2498608975/net/cni\" for cleanup: open /tmp/bktest_buildkitd2498608975/net/cni: no such file or directory"
    sandbox.go:288: time="2023-06-30T14:10:33Z" level=debug msg="creating new network namespace qojoiq69j50jjun0dumihqadh"
    sandbox.go:288: time="2023-06-30T14:10:33Z" level=debug msg="finished creating network namespace qojoiq69j50jjun0dumihqadh"
    sandbox.go:288: time="2023-06-30T14:10:33Z" level=debug msg="finished setting up network namespace qojoiq69j50jjun0dumihqadh"
    sandbox.go:288: time="2023-06-30T14:10:33Z" level=info msg="found worker \"hdi27o75gqhbr88mwyp541lux\", labels=map[org.mobyproject.buildkit.worker.executor:oci org.mob
},
},
}
Expand Down Expand Up @@ -2108,6 +2108,7 @@
Attrs: map[string]string{
"name": target,
"push": "true",
"unpack": "true",
"compression": "uncompressed",
},
},
Expand Down
18 changes: 9 additions & 9 deletions exporter/containerimage/export.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -258,7 +258,7 @@ func (e *imageExporterInstance) Export(ctx context.Context, src *exporter.Source
}
tagDone(nil)

if src.Ref != nil && e.unpack {
if e.unpack {
if err := e.unpackImage(ctx, img, src, session.NewGroup(sessionID)); err != nil {
return nil, nil, err
}
Expand Down Expand Up @@ -366,16 +366,16 @@ func (e *imageExporterInstance) unpackImage(ctx context.Context, img images.Imag
return err
}

topLayerRef := src.Ref
if len(src.Refs) > 0 {
if r, ok := src.Refs[defaultPlatform()]; ok {
topLayerRef = r
} else {
return errors.Errorf("no reference for default platform %s", defaultPlatform())
}
ref, ok := src.FindRef(defaultPlatform())
if !ok {
return errors.Errorf("no reference for default platform %s", defaultPlatform())
}
if ref == nil {
// ref has no layers, so nothing to unpack
return nil
}

remotes, err := topLayerRef.GetRemotes(ctx, true, e.opts.RefCfg, false, s)
remotes, err := ref.GetRemotes(ctx, true, e.opts.RefCfg, false, s)
if err != nil {
return err
}
Expand Down
Loading