Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Crash during restart of a newly built service (concurrent map write) #12293

Closed
wuurrd opened this issue Nov 14, 2024 · 1 comment
Closed

Comments

@wuurrd
Copy link

wuurrd commented Nov 14, 2024

Description

docker-compose crash during shutdown:

` ⠹ Container platform-worker-1 Stopping

Steps To Reproduce

docker compose up --build with already running service.

It's racy, doesn't happen all the time.

Compose Version

❯ docker compose version   
Docker Compose version 2.29.5

Docker Environment

Client:
 Version:    27.2.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  0.17.1
    Path:     /usr/lib/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  2.29.5
    Path:     /usr/lib/docker/cli-plugins/docker-compose

Server:
 Containers: 5
  Running: 1
  Paused: 0
  Stopped: 4
 Images: 1088
 Server Version: 27.2.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: true
  Native Overlay Diff: false
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 nvidia runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c.m
 runc version: 
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.1.112-1-MANJARO
 Operating System: Manjaro Linux
 OSType: linux
 Architecture: x86_64
 CPUs: 24
 Total Memory: 62.71GiB
 Name: magic
 ID: defcd70a-8f8e-4cd0-98b9-78506b498ce9
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?


goroutine 1551 [running]:
github.com/docker/compose/v2/pkg/compose.(*convergence).stopDependentContainers(_, {_, _}, _, {{0xc0006eb440, 0x5}, {0x0, 0x0, 0x0}, 0x0, ...})
        github.com/docker/compose/v2/pkg/compose/convergence.go:235 +0x1b4
github.com/docker/compose/v2/pkg/compose.(*convergence).ensureService(_, {_, _}, _, {{0xc0006eb440, 0x5}, {0x0, 0x0, 0x0}, 0x0, ...}, ...)
        github.com/docker/compose/v2/pkg/compose/convergence.go:162 +0x694
github.com/docker/compose/v2/pkg/compose.(*composeService).create.(*convergence).apply.func1.1({0x56087affdf08, 0xc0007fc4e0})
        github.com/docker/compose/v2/pkg/compose/convergence.go:103 +0x10b
github.com/docker/compose/v2/internal/tracing.SpanWrapFunc.func1({0x56087affdf40, 0xc000361e50})
        github.com/docker/compose/v2/internal/tracing/wrap.go:43 +0x1af
github.com/docker/compose/v2/pkg/compose.(*composeService).create.(*convergence).apply.func1({0x56087affdf40, 0xc000361e50}, {0xc0006eb440, 0x5})
        github.com/docker/compose/v2/pkg/compose/convergence.go:104 +0x2e3
github.com/docker/compose/v2/pkg/compose.(*graphTraversal).run.func1()
        github.com/docker/compose/v2/pkg/compose/dependencies.go:184 +0x99
golang.org/x/sync/errgroup.(*Group).Go.func1()
        golang.org/x/sync@v0.8.0/errgroup/errgroup.go:78 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 58
        golang.org/x/sync@v0.8.0/errgroup/errgroup.go:75 +0x96

goroutine 1 [semacquire]:
sync.runtime_Semacquire(0xc0004766d8?)
        runtime/sema.go:71 +0x25
sync.(*WaitGroup).Wait(0x56087ae1b2e0?)
        sync/waitgroup.go:118 +0x48
golang.org/x/sync/errgroup.(*Group).Wait(0xc00084ea40)
        golang.org/x/sync@v0.8.0/errgroup/errgroup.go:56 +0x25
github.com/docker/compose/v2/pkg/progress.RunWithStatus({0x56087affdf08, 0xc000a07a70}, 0xc00069baf0, 0xc00040b680, {0x56087a6158d6, 0x7})
        github.com/docker/compose/v2/pkg/progress/writer.go:97 +0x1ef
github.com/docker/compose/v2/pkg/progress.Run({0x56087affdf08, 0xc000a07a70}, 0xc00084ea00, 0xc00040b680)
        github.com/docker/compose/v2/pkg/progress/writer.go:61 +0x85
github.com/docker/compose/v2/pkg/compose.(*composeService).Up(0xc000592180, {0x56087affdf08, 0xc000a07a70}, _, {{0xc000418900, {0xc0000d3120, 0x0, 0x2}, 0x0, 0x0, ...}, ...})
        github.com/docker/compose/v2/pkg/compose/up.go:40 +0x233
github.com/docker/compose/v2/cmd/compose.runUp({_, _}, {_, _}, {_, _}, {0x1, 0x0, {0x56087a613ca1, 0x6}, ...}, ...)
        github.com/docker/compose/v2/cmd/compose/up.go:295 +0xbf4
github.com/docker/compose/v2/cmd/compose.upCommand.func2({0x56087affdf08, 0xc000a07a70}, 0xc00072a7e0, {0xc0000d3120, 0x0, 0x2})
        github.com/docker/compose/v2/cmd/compose/up.go:136 +0x2d8
github.com/docker/compose/v2/cmd/compose.upCommand.(*ProjectOptions).WithServices.func4({0x56087affdf40, 0xc0005c6f00}, {0xc0000d3120, 0x0, 0x2})
        github.com/docker/compose/v2/cmd/compose/compose.go:170 +0x194
github.com/docker/compose/v2/cmd/compose.upCommand.(*ProjectOptions).WithServices.Adapt.func6({0x56087affdf40?, 0xc0005c6f00?}, 0x2?, {0xc0000d3120?, 0x56087afe3120?, 0x108d760?})
        github.com/docker/compose/v2/cmd/compose/compose.go:125 +0x30
github.com/docker/compose/v2/cmd/compose.upCommand.(*ProjectOptions).WithServices.Adapt.AdaptCmd.func7(0xc000430c08, {0xc0000d3120, 0x0, 0x2})
        github.com/docker/compose/v2/cmd/compose/compose.go:101 +0x143
github.com/docker/cli/cli-plugins/plugin.RunPlugin.func1.1.2(0xc000430c08, {0xc0000d3120, 0x0, 0x2})
        github.com/docker/cli@v27.3.0-rc.1+incompatible/cli-plugins/plugin/plugin.go:64 +0x6c
github.com/docker/compose/v2/cmd/cmdtrace.Setup.wrapRunE.func2(0xc000430c08?, {0xc0000d3120?, 0x0?, 0x2?})
        github.com/docker/compose/v2/cmd/cmdtrace/cmd_span.go:85 +0x63
github.com/spf13/cobra.(*Command).execute(0xc000430c08, {0xc000592760, 0x2, 0x2})
        github.com/spf13/cobra@v1.8.1/command.go:985 +0xaaa
github.com/spf13/cobra.(*Command).ExecuteC(0xc0004de308)
        github.com/spf13/cobra@v1.8.1/command.go:1117 +0x3ff
github.com/spf13/cobra.(*Command).Execute(...)
        github.com/spf13/cobra@v1.8.1/command.go:1041
github.com/docker/cli/cli-plugins/plugin.RunPlugin(0xc0002c4140, 0xc000430908, {{0x56087a61274d, 0x5}, {0x56087a61c6e7, 0xb}, {0x56087a985750, 0x6}, {0x0, 0x0}, ...})
        github.com/docker/cli@v27.3.0-rc.1+incompatible/cli-plugins/plugin/plugin.go:79 +0x145
github.com/docker/cli/cli-plugins/plugin.Run(0x56087afcd178, {{0x56087a61274d, 0x5}, {0x56087a61c6e7, 0xb}, {0x56087a985750, 0x6}, {0x0, 0x0}, {0x0, ...}})
        github.com/docker/cli@v27.3.0-rc.1+incompatible/cli-plugins/plugin/plugin.go:94 +0x165
main.pluginMain()
        github.com/docker/compose/v2/cmd/main.go:38 +0xa5
main.main()
        github.com/docker/compose/v2/cmd/main.go:98 +0x19c

goroutine 51 [IO wait]:
internal/poll.runtime_pollWait(0x7f349907d6e8, 0x72)
        runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc0001e4e00?, 0xc00011efbf?, 0x0)
        internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
        internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0001e4e00, {0xc00011efbf, 0x1, 0x1})
        internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc0001e4e00, {0xc00011efbf?, 0x0?, 0x0?})
        net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc00059c300, {0xc00011efbf?, 0x0?, 0x0?})
        net/net.go:189 +0x45
github.com/docker/cli/cli-plugins/socket.ConnectAndWait.func1()
        github.com/docker/cli@v27.3.0-rc.1+incompatible/cli-plugins/socket/socket.go:162 +0x45
created by github.com/docker/cli/cli-plugins/socket.ConnectAndWait in goroutine 1
        github.com/docker/cli@v27.3.0-rc.1+incompatible/cli-plugins/socket/socket.go:159 +0x118

goroutine 55 [chan receive]:
github.com/docker/compose/v2/cmd/compose.upCommand.AdaptCmd.func3.1()
        github.com/docker/compose/v2/cmd/compose/compose.go:95 +0x27
created by github.com/docker/compose/v2/cmd/compose.upCommand.AdaptCmd.func3 in goroutine 1
        github.com/docker/compose/v2/cmd/compose/compose.go:94 +0x10a

goroutine 53 [IO wait]:
internal/poll.runtime_pollWait(0x7f349907d5e0, 0x72)
        runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc0005c8480?, 0xc0004e3000?, 0x0)
        internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
        internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0005c8480, {0xc0004e3000, 0x1000, 0x1000})
        internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc0005c8480, {0xc0004e3000?, 0xc001088720?, 0x56087afd73d8?})
        net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc00059c8c0, {0xc0004e3000?, 0xc0005d3ce0?, 0x560878cca5e5?})
        net/net.go:189 +0x45
net/http.(*persistConn).Read(0xc0004aa900, {0xc0004e3000?, 0x560878fe25c5?, 0x56087ac18960?})
        net/http/transport.go:2052 +0x4a
bufio.(*Reader).fill(0xc000113440)
        bufio/bufio.go:110 +0x103
bufio.(*Reader).Peek(0xc000113440, 0x1)
        bufio/bufio.go:148 +0x53
net/http.(*persistConn).readLoop(0xc0004aa900)
        net/http/transport.go:2205 +0x185
created by net/http.(*Transport).dialConn in goroutine 52
        net/http/transport.go:1874 +0x154f

goroutine 54 [select]:
net/http.(*persistConn).writeLoop(0xc0004aa900)
        net/http/transport.go:2519 +0xe7
created by net/http.(*Transport).dialConn in goroutine 52
        net/http/transport.go:1875 +0x15a5

goroutine 39 [select]:
go.opentelemetry.io/otel/sdk/trace.(*batchSpanProcessor).processQueue(0xc000640280)
        go.opentelemetry.io/otel/sdk@v1.21.0/trace/batch_span_processor.go:312 +0x114
go.opentelemetry.io/otel/sdk/trace.NewBatchSpanProcessor.func1()
        go.opentelemetry.io/otel/sdk@v1.21.0/trace/batch_span_processor.go:128 +0x4e
created by go.opentelemetry.io/otel/sdk/trace.NewBatchSpanProcessor in goroutine 1
        go.opentelemetry.io/otel/sdk@v1.21.0/trace/batch_span_processor.go:126 +0x2e5

goroutine 66 [syscall]:
os/signal.signal_recv()
        runtime/sigqueue.go:152 +0x29
os/signal.loop()
        os/signal/signal_unix.go:23 +0x13
created by os/signal.Notify.func1.1 in goroutine 1
        os/signal/signal.go:151 +0x1f

goroutine 56 [chan receive]:
github.com/docker/compose/v2/cmd/compose.upCommand.(*ProjectOptions).WithServices.Adapt.AdaptCmd.func7.1()
        github.com/docker/compose/v2/cmd/compose/compose.go:95 +0x27
created by github.com/docker/compose/v2/cmd/compose.upCommand.(*ProjectOptions).WithServices.Adapt.AdaptCmd.func7 in goroutine 1
        github.com/docker/compose/v2/cmd/compose/compose.go:94 +0x10a

goroutine 57 [select]:
github.com/docker/compose/v2/pkg/progress.(*ttyWriter).Start(0xc000418c00, {0x56087affdc38, 0x56087c267b00})
        github.com/docker/compose/v2/pkg/progress/tty.go:54 +0xfa
github.com/docker/compose/v2/pkg/progress.RunWithStatus.func1()
        github.com/docker/compose/v2/pkg/progress/writer.go:83 +0x2a
golang.org/x/sync/errgroup.(*Group).Go.func1()
        golang.org/x/sync@v0.8.0/errgroup/errgroup.go:78 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1
        golang.org/x/sync@v0.8.0/errgroup/errgroup.go:75 +0x96

goroutine 58 [semacquire]:
sync.runtime_Semacquire(0x56087a5a44a8?)
        runtime/sema.go:71 +0x25
sync.(*WaitGroup).Wait(0x1?)
        sync/waitgroup.go:118 +0x48
golang.org/x/sync/errgroup.(*Group).Wait(0xc0007cd000)
        golang.org/x/sync@v0.8.0/errgroup/errgroup.go:56 +0x25
github.com/docker/compose/v2/pkg/compose.(*graphTraversal).visit(0xc000361e00, {0x56087affdf08, 0xc0004bdec0}, 0xc00036c8a0)
        github.com/docker/compose/v2/pkg/compose/dependencies.go:163 +0x21b
github.com/docker/compose/v2/pkg/compose.InDependencyOrder({0x56087affdf08, 0xc0004bdec0}, 0x56087a6d5db1?, 0xc0001e5400, {0x0, 0x0, 0xc000592180?})
        github.com/docker/compose/v2/pkg/compose/dependencies.go:88 +0xf1
github.com/docker/compose/v2/pkg/compose.(*convergence).apply(...)
        github.com/docker/compose/v2/pkg/compose/convergence.go:92
github.com/docker/compose/v2/pkg/compose.(*composeService).create(0xc000592180, {0x56087affdf08, 0xc0004bdec0}, 0xc00072a7e0, {0xc000418900, {0xc000418d00, 0x5, 0x8}, 0x0, 0x0, ...})
        github.com/docker/compose/v2/pkg/compose/create.go:118 +0x5c5
github.com/docker/compose/v2/pkg/compose.(*composeService).Up.func1({0x56087affdf08, 0xc0004bdec0})
        github.com/docker/compose/v2/pkg/compose/up.go:41 +0x85
github.com/docker/compose/v2/pkg/compose.(*composeService).Up.SpanWrapFunc.func5({0x56087affdf08, 0xc0004bde90})
        github.com/docker/compose/v2/internal/tracing/wrap.go:43 +0x13d
github.com/docker/compose/v2/pkg/progress.Run.func1({0x56087affdf08?, 0xc0004bde90?})
        github.com/docker/compose/v2/pkg/progress/writer.go:62 +0x22
github.com/docker/compose/v2/pkg/progress.RunWithStatus.func2()
        github.com/docker/compose/v2/pkg/progress/writer.go:90 +0x70
golang.org/x/sync/errgroup.(*Group).Go.func1()
        golang.org/x/sync@v0.8.0/errgroup/errgroup.go:78 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1
        golang.org/x/sync@v0.8.0/errgroup/errgroup.go:75 +0x96

goroutine 30 [IO wait]:
internal/poll.runtime_pollWait(0x7f349907d3d0, 0x72)
        runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc000630080?, 0xc00063c000?, 0x0)
        internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
        internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000630080, {0xc00063c000, 0x1000, 0x1000})
        internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc000630080, {0xc00063c000?, 0xc000a442a0?, 0x56087afd73d8?})
        net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc0000b4320, {0xc00063c000?, 0x0?, 0x0?})
        net/net.go:189 +0x45
net/http.(*persistConn).Read(0xc0005b2120, {0xc00063c000?, 0x560878fe25e5?, 0x56087ac18960?})
        net/http/transport.go:2052 +0x4a
bufio.(*Reader).fill(0xc000880120)
        bufio/bufio.go:110 +0x103
bufio.(*Reader).Peek(0xc000880120, 0x1)
        bufio/bufio.go:148 +0x53
net/http.(*persistConn).readLoop(0xc0005b2120)
        net/http/transport.go:2205 +0x185
created by net/http.(*Transport).dialConn in goroutine 29
        net/http/transport.go:1874 +0x154f

goroutine 31 [select]:
net/http.(*persistConn).writeLoop(0xc0005b2120)
        net/http/transport.go:2519 +0xe7
created by net/http.(*Transport).dialConn in goroutine 29
        net/http/transport.go:1875 +0x15a5

goroutine 60 [IO wait]:
internal/poll.runtime_pollWait(0x7f349907d2c8, 0x72)
        runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc0005c8c00?, 0xc0003e5000?, 0x0)
        internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
        internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0005c8c00, {0xc0003e5000, 0x1000, 0x1000})
        internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc0005c8c00, {0xc0003e5000?, 0xc0008803c0?, 0x56087afd73d8?})
        net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc0000b4758, {0xc0003e5000?, 0x0?, 0x0?})
        net/net.go:189 +0x45
net/http.(*persistConn).Read(0xc00076eea0, {0xc0003e5000?, 0x560878fe25c5?, 0x56087ac18960?})
        net/http/transport.go:2052 +0x4a
bufio.(*Reader).fill(0xc0007028a0)
        bufio/bufio.go:110 +0x103
bufio.(*Reader).Peek(0xc0007028a0, 0x1)
        bufio/bufio.go:148 +0x53
net/http.(*persistConn).readLoop(0xc00076eea0)
        net/http/transport.go:2205 +0x185
created by net/http.(*Transport).dialConn in goroutine 59
        net/http/transport.go:1874 +0x154f

goroutine 61 [select]:
net/http.(*persistConn).writeLoop(0xc00076eea0)
        net/http/transport.go:2519 +0xe7
created by net/http.(*Transport).dialConn in goroutine 59
        net/http/transport.go:1875 +0x15a5

goroutine 211 [select]:
google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc00046ebe0, 0x1)
        google.golang.org/grpc@v1.62.0/internal/transport/controlbuf.go:418 +0x113
google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0003b41c0)
        google.golang.org/grpc@v1.62.0/internal/transport/controlbuf.go:551 +0x7b
google.golang.org/grpc/internal/transport.newHTTP2Client.func6()
        google.golang.org/grpc@v1.62.0/internal/transport/http2_client.go:454 +0x85
created by google.golang.org/grpc/internal/transport.newHTTP2Client in goroutine 95
        google.golang.org/grpc@v1.62.0/internal/transport/http2_client.go:452 +0x22cb

goroutine 210 [IO wait]:
internal/poll.runtime_pollWait(0x7f349907cfb0, 0x72)
        runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc000b9c000?, 0xc0005d4000?, 0x0)
        internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
        internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000b9c000, {0xc0005d4000, 0x8000, 0x8000})
        internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc000b9c000, {0xc0005d4000?, 0x1060100000000?, 0x8?})
        net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc0001243c0, {0xc0005d4000?, 0x878010601?, 0xc000000000?})
        net/net.go:189 +0x45
bufio.(*Reader).Read(0xc000892360, {0xc000aa04a0, 0x9, 0xc000101508?})
        bufio/bufio.go:241 +0x197
io.ReadAtLeast({0x56087afd7760, 0xc000892360}, {0xc000aa04a0, 0x9, 0x9}, 0x9)
        io/io.go:335 +0x90
io.ReadFull(...)
        io/io.go:354
golang.org/x/net/http2.readFrameHeader({0xc000aa04a0, 0x9, 0xc00112cd38?}, {0x56087afd7760?, 0xc000892360?})
        golang.org/x/net@v0.28.0/http2/frame.go:237 +0x65
golang.org/x/net/http2.(*Framer).ReadFrame(0xc000aa0460)
        golang.org/x/net@v0.28.0/http2/frame.go:501 +0x85
google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc000a94908, 0xc0003b4150)
        google.golang.org/grpc@v1.62.0/internal/transport/http2_client.go:1593 +0x226
created by google.golang.org/grpc/internal/transport.newHTTP2Client in goroutine 95
        google.golang.org/grpc@v1.62.0/internal/transport/http2_client.go:400 +0x1d1e

goroutine 151 [select]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc000a02630, {0x56087affdf40, 0xc000a082d0})
        google.golang.org/grpc@v1.62.0/internal/grpcsync/callback_serializer.go:76 +0x10c
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 101
        google.golang.org/grpc@v1.62.0/internal/grpcsync/callback_serializer.go:52 +0x11a

goroutine 149 [IO wait]:
internal/poll.runtime_pollWait(0x7f349907d1c0, 0x72)
        runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc0001e4080?, 0xc00095e000?, 0x0)
        internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
        internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0001e4080, {0xc00095e000, 0x1000, 0x1000})
        internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc0001e4080, {0xc00095e000?, 0x0?, 0x56087afd73d8?})
        net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc0006ae0c0, {0xc00095e000?, 0x0?, 0x0?})
        net/net.go:189 +0x45
net/http.(*persistConn).Read(0xc00095c000, {0xc00095e000?, 0x560878fe25c5?, 0x56087ac18960?})
        net/http/transport.go:2052 +0x4a
bufio.(*Reader).fill(0xc00090e240)
        bufio/bufio.go:110 +0x103
bufio.(*Reader).Peek(0xc00090e240, 0x1)
        bufio/bufio.go:148 +0x53
net/http.(*persistConn).readLoop(0xc00095c000)
        net/http/transport.go:2205 +0x185
created by net/http.(*Transport).dialConn in goroutine 148
        net/http/transport.go:1874 +0x154f

goroutine 150 [select]:
net/http.(*persistConn).writeLoop(0xc00095c000)
        net/http/transport.go:2519 +0xe7
created by net/http.(*Transport).dialConn in goroutine 148
        net/http/transport.go:1875 +0x15a5

goroutine 103 [IO wait]:
internal/poll.runtime_pollWait(0x7f349907d0b8, 0x72)
        runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc0003d4680?, 0xc0004e2000?, 0x0)
        internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
        internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0003d4680, {0xc0004e2000, 0x1000, 0x1000})
        internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc0003d4680, {0xc0004e2000?, 0x0?, 0x56087afd73d8?})
        net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc0006ae018, {0xc0004e2000?, 0x0?, 0x0?})
        net/net.go:189 +0x45
net/http.(*persistConn).Read(0xc0004ba120, {0xc0004e2000?, 0x560878fe25c5?, 0x56087ac18960?})
        net/http/transport.go:2052 +0x4a
bufio.(*Reader).fill(0xc000112840)
        bufio/bufio.go:110 +0x103
bufio.(*Reader).Peek(0xc000112840, 0x1)
        bufio/bufio.go:148 +0x53
net/http.(*persistConn).readLoop(0xc0004ba120)
        net/http/transport.go:2205 +0x185
created by net/http.(*Transport).dialConn in goroutine 102
        net/http/transport.go:1874 +0x154f

goroutine 104 [select]:
net/http.(*persistConn).writeLoop(0xc0004ba120)
        net/http/transport.go:2519 +0xe7
created by net/http.(*Transport).dialConn in goroutine 102
        net/http/transport.go:1875 +0x15a5

goroutine 152 [select]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc000a02660, {0x56087affdf40, 0xc000a08320})
        google.golang.org/grpc@v1.62.0/internal/grpcsync/callback_serializer.go:76 +0x10c
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 101
        google.golang.org/grpc@v1.62.0/internal/grpcsync/callback_serializer.go:52 +0x11a

goroutine 153 [select]:
google.golang.org/grpc/internal/grpcsync.(*CallbackSerializer).run(0xc000a02690, {0x56087affdf40, 0xc000a08370})
        google.golang.org/grpc@v1.62.0/internal/grpcsync/callback_serializer.go:76 +0x10c
created by google.golang.org/grpc/internal/grpcsync.NewCallbackSerializer in goroutine 101
        google.golang.org/grpc@v1.62.0/internal/grpcsync/callback_serializer.go:52 +0x11a

goroutine 96 [select]:
github.com/moby/buildkit/util/tracing/otlptracegrpc.(*Connection).indefiniteBackgroundConnection(0xc00046e3c0)
        github.com/moby/buildkit@v0.16.0/util/tracing/otlptracegrpc/connection.go:134 +0x1cd
created by github.com/moby/buildkit/util/tracing/otlptracegrpc.(*Connection).StartConnection in goroutine 101
        github.com/moby/buildkit@v0.16.0/util/tracing/otlptracegrpc/connection.go:74 +0x125

goroutine 1553 [runnable]:
go.opentelemetry.io/otel/sdk/trace.(*tracer).newSpan(0xc00046fe50, {0x56087affdf08, 0xc0007fc750}, {0xc000e9e978, 0x12}, 0xc000778630)
        go.opentelemetry.io/otel/sdk@v1.21.0/trace/tracer.go:76 +0x98d
go.opentelemetry.io/otel/sdk/trace.(*tracer).Start(0xc00046fe50, {0x56087affdf08, 0xc0007fc750}, {0xc000e9e978, 0x12}, {0xc000727a50?, 0x0?, 0x0?})
        go.opentelemetry.io/otel/sdk@v1.21.0/trace/tracer.go:55 +0x1d3
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.(*Transport).RoundTrip(0xc000266e70, 0xc0002c4f00)
        go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp@v0.46.1/transport.go:106 +0x428
net/http.send(0xc0002c4f00, {0x56087afd7920, 0xc000266e70}, {0x7f349908a001?, 0xc0007fdf80?, 0x0?})
        net/http/client.go:259 +0x5e4
net/http.(*Client).send(0xc00061dce0, 0xc0002c4f00, {0xc00075abe0?, 0x560878c6cc2b?, 0x0?})
        net/http/client.go:180 +0x98
net/http.(*Client).do(0xc00061dce0, 0xc0002c4f00)
        net/http/client.go:725 +0x8bc
net/http.(*Client).Do(...)
        net/http/client.go:590
github.com/docker/docker/client.(*Client).doRequest(0xc0004fa3c0, 0xc0004fa3c0?)
        github.com/docker/docker@v27.3.0-rc.1+incompatible/client/request.go:141 +0x6d
github.com/docker/docker/client.(*Client).sendRequest(0xc0004fa3c0, {0x56087affdf08, 0xc0007fc750}, {0x56087a611451, 0x3}, {0x56087a617747?, 0x0?}, 0x0?, {0x0, 0x0}, ...)
        github.com/docker/docker@v27.3.0-rc.1+incompatible/client/request.go:125 +0xde
github.com/docker/docker/client.(*Client).get(...)
        github.com/docker/docker@v27.3.0-rc.1+incompatible/client/request.go:37
github.com/docker/docker/client.(*Client).ServerVersion(0x560878c6cc2b?, {0x56087affdf08?, 0xc0007fc750?})
        github.com/docker/docker@v27.3.0-rc.1+incompatible/client/version.go:12 +0x86
github.com/docker/compose/v2/pkg/compose.(*composeService).RuntimeVersion.func1()
        github.com/docker/compose/v2/pkg/compose/compose.go:314 +0x92
sync.(*Once).doSlow(0x1?, 0x0?)
        sync/once.go:76 +0xb4
sync.(*Once).Do(...)
        sync/once.go:67
github.com/docker/compose/v2/pkg/compose.(*composeService).RuntimeVersion(0x0?, {0x56087affdf08?, 0xc0007fc750?})
        github.com/docker/compose/v2/pkg/compose/compose.go:313 +0x4e
github.com/docker/compose/v2/pkg/compose.(*composeService).prepareContainerMACAddress(_, {_, _}, {{0xc0006eb398, 0x8}, {0x0, 0x0, 0x0}, 0x0, 0x0, ...}, ...)
        github.com/docker/compose/v2/pkg/compose/create.go:376 +0x4a
github.com/docker/compose/v2/pkg/compose.(*composeService).getCreateConfigs(_, {_, _}, _, {{0xc0006eb398, 0x8}, {0x0, 0x0, 0x0}, 0x0, ...}, ...)
        github.com/docker/compose/v2/pkg/compose/create.go:246 +0x42b
github.com/docker/compose/v2/pkg/compose.(*composeService).createMobyContainer(_, {_, _}, _, {{0xc0006eb398, 0x8}, {0x0, 0x0, 0x0}, 0x0, ...}, ...)
        github.com/docker/compose/v2/pkg/compose/convergence.go:582 +0x125
github.com/docker/compose/v2/pkg/compose.(*composeService).recreateContainer(_, {_, _}, _, {{0xc0006eb398, 0x8}, {0x0, 0x0, 0x0}, 0x0, ...}, ...)
        github.com/docker/compose/v2/pkg/compose/convergence.go:536 +0x516
github.com/docker/compose/v2/pkg/compose.(*convergence).ensureService.func3({0x56087affdf08?, 0xc0007fc750?})
        github.com/docker/compose/v2/pkg/compose/convergence.go:169 +0xee
github.com/docker/compose/v2/pkg/compose.(*convergence).ensureService.SpanWrapFuncForErrGroup.func7()
        github.com/docker/compose/v2/internal/tracing/wrap.go:66 +0x143
golang.org/x/sync/errgroup.(*Group).Go.func1()
        golang.org/x/sync@v0.8.0/errgroup/errgroup.go:78 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1550
        golang.org/x/sync@v0.8.0/errgroup/errgroup.go:75 +0x96

goroutine 376 [IO wait]:
internal/poll.runtime_pollWait(0x7f349907c980, 0x72)
        runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc000ab9300?, 0xc000b47000?, 0x0)
        internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
        internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000ab9300, {0xc000b47000, 0x1000, 0x1000})
        internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc000ab9300, {0xc000b47000?, 0x0?, 0x56087afd73d8?})
        net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc0006ae338, {0xc000b47000?, 0x0?, 0x0?})
        net/net.go:189 +0x45
net/http.(*persistConn).Read(0xc0005f6a20, {0xc000b47000?, 0x560878fe25c5?, 0x56087ac18960?})
        net/http/transport.go:2052 +0x4a
bufio.(*Reader).fill(0xc000a20b40)
        bufio/bufio.go:110 +0x103
bufio.(*Reader).Peek(0xc000a20b40, 0x1)
        bufio/bufio.go:148 +0x53
net/http.(*persistConn).readLoop(0xc0005f6a20)
        net/http/transport.go:2205 +0x185
created by net/http.(*Transport).dialConn in goroutine 375
        net/http/transport.go:1874 +0x154f

goroutine 266 [IO wait]:
internal/poll.runtime_pollWait(0x7f349907cb90, 0x72)
        runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc000ab9380?, 0xc00062c000?, 0x0)
        internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
        internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000ab9380, {0xc00062c000, 0x1000, 0x1000})
        internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc000ab9380, {0xc00062c000?, 0x0?, 0x56087afd73d8?})
        net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc0006ae0e0, {0xc00062c000?, 0x0?, 0x0?})
        net/net.go:189 +0x45
net/http.(*persistConn).Read(0xc00076e360, {0xc00062c000?, 0x560878fe25c5?, 0x56087ac18960?})
        net/http/transport.go:2052 +0x4a
bufio.(*Reader).fill(0xc00090e960)
        bufio/bufio.go:110 +0x103
bufio.(*Reader).Peek(0xc00090e960, 0x1)
        bufio/bufio.go:148 +0x53
net/http.(*persistConn).readLoop(0xc00076e360)
        net/http/transport.go:2205 +0x185
created by net/http.(*Transport).dialConn in goroutine 265
        net/http/transport.go:1874 +0x154f

goroutine 377 [select]:
net/http.(*persistConn).writeLoop(0xc0005f6a20)
        net/http/transport.go:2519 +0xe7
created by net/http.(*Transport).dialConn in goroutine 375
        net/http/transport.go:1875 +0x15a5

goroutine 267 [select]:
net/http.(*persistConn).writeLoop(0xc00076e360)
        net/http/transport.go:2519 +0xe7
created by net/http.(*Transport).dialConn in goroutine 265
        net/http/transport.go:1875 +0x15a5

goroutine 1549 [select]:
github.com/docker/compose/v2/pkg/compose.(*graphTraversal).visit.func1()
        github.com/docker/compose/v2/pkg/compose/dependencies.go:147 +0x10c
golang.org/x/sync/errgroup.(*Group).Go.func1()
        golang.org/x/sync@v0.8.0/errgroup/errgroup.go:78 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 58
        golang.org/x/sync@v0.8.0/errgroup/errgroup.go:75 +0x96

goroutine 414 [IO wait]:
internal/poll.runtime_pollWait(0x7f349907ca88, 0x72)
        runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0xc000631b00?, 0xc000569000?, 0x0)
        internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
        internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000631b00, {0xc000569000, 0x1000, 0x1000})
        internal/poll/fd_unix.go:165 +0x27a
net.(*netFD).Read(0xc000631b00, {0xc000569000?, 0x0?, 0x56087afd73d8?})
        net/fd_posix.go:55 +0x25
net.(*conn).Read(0xc00059c208, {0xc000569000?, 0x0?, 0x0?})
        net/net.go:189 +0x45
net/http.(*persistConn).Read(0xc0004aa120, {0xc000569000?, 0x560878fe25c5?, 0x56087ac18960?})
        net/http/transport.go:2052 +0x4a
bufio.(*Reader).fill(0xc000a211a0)
        bufio/bufio.go:110 +0x103
bufio.(*Reader).Peek(0xc000a211a0, 0x1)
        bufio/bufio.go:148 +0x53
net/http.(*persistConn).readLoop(0xc0004aa120)
        net/http/transport.go:2205 +0x185
created by net/http.(*Transport).dialConn in goroutine 413
        net/http/transport.go:1874 +0x154f

goroutine 415 [select]:
net/http.(*persistConn).writeLoop(0xc0004aa120)
        net/http/transport.go:2519 +0xe7
created by net/http.(*Transport).dialConn in goroutine 413
        net/http/transport.go:1875 +0x15a5

goroutine 1550 [semacquire]:
sync.runtime_Semacquire(0xc00128a498?)
        runtime/sema.go:71 +0x25
sync.(*WaitGroup).Wait(0x56087ae1a780?)
        sync/waitgroup.go:118 +0x48
golang.org/x/sync/errgroup.(*Group).Wait(0xc00109ef40)
        golang.org/x/sync@v0.8.0/errgroup/errgroup.go:56 +0x25
github.com/docker/compose/v2/pkg/compose.(*convergence).ensureService(_, {_, _}, _, {{0xc0006eb398, 0x8}, {0x0, 0x0, 0x0}, 0x0, ...}, ...)
        github.com/docker/compose/v2/pkg/compose/convergence.go:216 +0xd1f
github.com/docker/compose/v2/pkg/compose.(*composeService).create.(*convergence).apply.func1.1({0x56087affdf08, 0xc00108fa40})
        github.com/docker/compose/v2/pkg/compose/convergence.go:103 +0x10b
github.com/docker/compose/v2/internal/tracing.SpanWrapFunc.func1({0x56087affdf40, 0xc000361e50})
        github.com/docker/compose/v2/internal/tracing/wrap.go:43 +0x1af
github.com/docker/compose/v2/pkg/compose.(*composeService).create.(*convergence).apply.func1({0x56087affdf40, 0xc000361e50}, {0xc0006eb398, 0x8})
        github.com/docker/compose/v2/pkg/compose/convergence.go:104 +0x2e3
github.com/docker/compose/v2/pkg/compose.(*graphTraversal).run.func1()
        github.com/docker/compose/v2/pkg/compose/dependencies.go:184 +0x99
golang.org/x/sync/errgroup.(*Group).Go.func1()
        golang.org/x/sync@v0.8.0/errgroup/errgroup.go:78 +0x50
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 58
        golang.org/x/sync@v0.8.0/errgroup/errgroup.go:75 +0x96```
@wuurrd wuurrd changed the title [BUG] <title> [BUG] Crash during restart of a newly built service (concurrent map write) Nov 14, 2024
@ndeloof
Copy link
Contributor

ndeloof commented Nov 14, 2024

Already fixed by #12150
Please upgrade to latest release

@ndeloof ndeloof closed this as completed Nov 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants