Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SIGSEGV: segmentation violation #676

Closed
m-hm opened this issue Mar 19, 2024 · 25 comments
Closed

SIGSEGV: segmentation violation #676

m-hm opened this issue Mar 19, 2024 · 25 comments
Labels
bug Something isn't working question Further information is requested

Comments

@m-hm
Copy link

m-hm commented Mar 19, 2024 •
edited
Loading

What happened?

Run Laravel app

Build Type

Download from Github

Worker Mode

No

Operating System

GNU/Linux

CPU Architecture

x86_64

PHP configuration

frankenphp php-server -r public  -l :8032

Relevant log output

frankenphp php-server -r public  -l :8032                
2024/03/19 10:09:32.304 WARN    admin   admin endpoint disabled
2024/03/19 10:09:32.304 INFO    tls.cache.maintenance   started background certificate maintenance      {"cache": "0xc0003c9000"}
2024/03/19 10:09:32.304 INFO    FrankenPHP started 🐘   {"php_version": "8.3.4"}
2024/03/19 10:09:32.304 INFO    http.log        server running  {"name": "php", "protocols": ["h1", "h2", "h3"]}
2024/03/19 10:09:32.304 INFO    Caddy serving PHP app on :8032
2024/03/19 10:09:32.359 WARN    tls     storage cleaning happened too recently; skipping for now        {"storage": "FileStorage:/home/mohammad/.local/share/caddy", "instance": "4837012e-77e4-420f-a885-4849ae793e53", "try_again": "2024/03/20 10:09:32.359", "try_again_in": 86399.999999292}
2024/03/19 10:09:32.359 INFO    tls     finished cleaning storage units
SIGSEGV: segmentation violation
PC=0x7fa32c091b10 m=20 sigcode=1 addr=0x7fa330ec93f0
signal arrived during cgo execution

goroutine 89 gp=0xc000503c00 m=20 mp=0xc0000cd808 [syscall, locked to thread]:
runtime.cgocall(0x7fa32aead120, 0xc000556ce0)
        /usr/local/go/src/runtime/cgocall.go:157 +0x4b fp=0xc000556cb8 sp=0xc000556c80 pc=0x7fa329add1ab
github.com/dunglas/frankenphp._Cfunc_frankenphp_update_server_context(0x1, 0x6c, 0x0, 0x7fa3318ea090, 0x7fa3318ea0a0, 0x0, 0x0, 0x7fa3313721c0, 0x0, 0x0, ...)
        _cgo_gotypes.go:1060 +0x52 fp=0xc000556ce0 sp=0xc000556cb8 pc=0x7fa32ad248f2
github.com/dunglas/frankenphp.updateServerContext(0xc0007237a0, 0x1, 0x0)
        /go/src/app/frankenphp.go:419 +0x4e5 fp=0xc000556dd8 sp=0xc000556ce0 pc=0x7fa32ad26e45
github.com/dunglas/frankenphp.go_execute_script(0x7fa329b4d101?)
        /go/src/app/frankenphp.go:506 +0x10f fp=0xc000556e40 sp=0xc000556dd8 pc=0x7fa32ad2754f
_cgoexp_feab5cfeb5bc_go_execute_script(0x7fa329b1fd31?)
        _cgo_gotypes.go:1111 +0x16 fp=0xc000556e58 sp=0xc000556e40 pc=0x7fa32ad2cc56
runtime.cgocallbackg1(0x7fa32ad2cc40, 0x7fa33111f850, 0x0)
        /usr/local/go/src/runtime/cgocall.go:403 +0x2a5 fp=0xc000556f18 sp=0xc000556e58 pc=0x7fa329add8e5
runtime.cgocallbackg(0x7fa32ad2cc40, 0x7fa33111f850, 0x0)
        /usr/local/go/src/runtime/cgocall.go:322 +0x138 fp=0xc000556f90 sp=0xc000556f18 pc=0x7fa329add598
runtime.cgocallbackg(0x7fa32ad2cc40, 0x7fa33111f850, 0x0)
        <autogenerated>:1 +0x2b fp=0xc000556fb8 sp=0xc000556f90 pc=0x7fa329b5192b
runtime.cgocallback(0xc00025ba80, 0xc00025baa0, 0xc00025bac0)
        /usr/local/go/src/runtime/asm_amd64.s:1079 +0xcd fp=0xc000556fe0 sp=0xc000556fb8 pc=0x7fa329b4ef6d
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000556fe8 sp=0xc000556fe0 pc=0x7fa329b4f1c1

goroutine 1 gp=0xc0000061c0 m=nil [select (no cases), 12 minutes]:
runtime.gopark(0x7fa329ae44e5?, 0x10?, 0xf8?, 0x4e?, 0xc000764ee8?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000070f50 sp=0xc000070f30 pc=0x7fa329b16c8e
runtime.block()
        /usr/local/go/src/runtime/select.go:103 +0x26 fp=0xc000070f80 sp=0xc000070f50 pc=0x7fa329b28426
github.com/dunglas/frankenphp/caddy.cmdPHPServer({0x0?})
        /go/src/app/caddy/php-server.go:329 +0x27d0 fp=0xc000071b60 sp=0xc000070f80 pc=0x7fa32ae32db0
github.com/dunglas/frankenphp/caddy.init.2.func1.WrapCommandFuncForCobra.1(0xc00024cf08, {0x7fa32c0a048f?, 0x4?, 0x7fa32c0a0297?})
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/cmd/cobra.go:137 +0x2f fp=0xc000071ba0 sp=0xc000071b60 pc=0x7fa32ae331af
github.com/spf13/cobra.(*Command).execute(0xc00024cf08, {0xc000519e80, 0x4, 0x4})
        /go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:983 +0xaca fp=0xc000071d28 sp=0xc000071ba0 pc=0x7fa32a185f8a
github.com/spf13/cobra.(*Command).ExecuteC(0x7fa33059f040)
        /go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1115 +0x3ff fp=0xc000071e00 sp=0xc000071d28 pc=0x7fa32a18685f
github.com/spf13/cobra.(*Command).Execute(...)
        /go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1039
github.com/caddyserver/caddy/v2/cmd.Main()
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/cmd/main.go:66 +0x5b fp=0xc000071e60 sp=0xc000071e00 pc=0x7fa32a20975b
main.main()
        /go/src/app/caddy/frankenphp/main.go:24 +0x18d fp=0xc000071f50 sp=0xc000071e60 pc=0x7fa32aeac1cd
runtime.main()
        /usr/local/go/src/runtime/proc.go:271 +0x29d fp=0xc000071fe0 sp=0xc000071f50 pc=0x7fa329b1683d
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000071fe8 sp=0xc000071fe0 pc=0x7fa329b4f1c1

goroutine 17 gp=0xc000006700 m=nil [select, locked to thread]:
runtime.gopark(0xc0000ace10?, 0x2?, 0x8?, 0x0?, 0xc0000acdf4?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000acc90 sp=0xc0000acc70 pc=0x7fa329b16c8e
runtime.selectgo(0xc0000ace10, 0xc0000acdf0, 0x7fa3306fbd00?, 0x0, 0x0?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc0000acdb0 sp=0xc0000acc90 pc=0x7fa329b28b65
github.com/dunglas/frankenphp.go_fetch_request()
        /go/src/app/frankenphp.go:473 +0x75 fp=0xc0000ace48 sp=0xc0000acdb0 pc=0x7fa32ad27295
_cgoexp_feab5cfeb5bc_go_fetch_request(0x7fa331a55890)
        _cgo_gotypes.go:1103 +0x14 fp=0xc0000ace58 sp=0xc0000ace48 pc=0x7fa32ad2cc14
runtime.cgocallbackg1(0x7fa32ad2cc00, 0x7fa331a55890, 0x0)
        /usr/local/go/src/runtime/cgocall.go:403 +0x2a5 fp=0xc0000acf18 sp=0xc0000ace58 pc=0x7fa329add8e5
runtime.cgocallbackg(0x7fa32ad2cc00, 0x7fa331a55890, 0x0)
        /usr/local/go/src/runtime/cgocall.go:322 +0x138 fp=0xc0000acf90 sp=0xc0000acf18 pc=0x7fa329add598
runtime.cgocallbackg(0x7fa32ad2cc00, 0x7fa331a55890, 0x0)
        <autogenerated>:1 +0x2b fp=0xc0000acfb8 sp=0xc0000acf90 pc=0x7fa329b5192b
runtime.cgocallback(0x0, 0x0, 0x0)
        /usr/local/go/src/runtime/asm_amd64.s:1079 +0xcd fp=0xc0000acfe0 sp=0xc0000acfb8 pc=0x7fa329b4ef6d
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000acfe8 sp=0xc0000acfe0 pc=0x7fa329b4f1c1

goroutine 2 gp=0xc000006c40 m=nil [force gc (idle), 2 minutes]:
runtime.gopark(0x353a7da0ca4?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000098fa8 sp=0xc000098f88 pc=0x7fa329b16c8e
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.forcegchelper()
        /usr/local/go/src/runtime/proc.go:326 +0xb8 fp=0xc000098fe0 sp=0xc000098fa8 pc=0x7fa329b16af8
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000098fe8 sp=0xc000098fe0 pc=0x7fa329b4f1c1
created by runtime.init.6 in goroutine 1
        /usr/local/go/src/runtime/proc.go:314 +0x1a

goroutine 3 gp=0xc000007180 m=nil [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000099780 sp=0xc000099760 pc=0x7fa329b16c8e
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.bgsweep(0xc0000583f0)
        /usr/local/go/src/runtime/mgcsweep.go:318 +0xdf fp=0xc0000997c8 sp=0xc000099780 pc=0x7fa329aff85f
runtime.gcenable.gowrap1()
        /usr/local/go/src/runtime/mgc.go:203 +0x25 fp=0xc0000997e0 sp=0xc0000997c8 pc=0x7fa329af4125
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000997e8 sp=0xc0000997e0 pc=0x7fa329b4f1c1
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:203 +0x66

goroutine 4 gp=0xc000007340 m=nil [GC scavenge wait]:
runtime.gopark(0x3b9cb14a?, 0x3b9aca00?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000099f78 sp=0xc000099f58 pc=0x7fa329b16c8e
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.(*scavengerState).park(0x7fa33066a6e0)
        /usr/local/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000099fa8 sp=0xc000099f78 pc=0x7fa329afd1e9
runtime.bgscavenge(0xc0000583f0)
        /usr/local/go/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc000099fc8 sp=0xc000099fa8 pc=0x7fa329afd799
runtime.gcenable.gowrap2()
        /usr/local/go/src/runtime/mgc.go:204 +0x25 fp=0xc000099fe0 sp=0xc000099fc8 pc=0x7fa329af40c5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000099fe8 sp=0xc000099fe0 pc=0x7fa329b4f1c1
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:204 +0xa5

goroutine 5 gp=0xc000007c00 m=nil [finalizer wait, 12 minutes]:
runtime.gopark(0xc000098648?, 0x7fa329ae72a5?, 0xa8?, 0x1?, 0xc0000061c0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000098620 sp=0xc000098600 pc=0x7fa329b16c8e
runtime.runfinq()
        /usr/local/go/src/runtime/mfinal.go:194 +0x107 fp=0xc0000987e0 sp=0xc000098620 pc=0x7fa329af3167
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000987e8 sp=0xc0000987e0 pc=0x7fa329b4f1c1
created by runtime.createfing in goroutine 1
        /usr/local/go/src/runtime/mfinal.go:164 +0x3d

goroutine 6 gp=0xc00027e1c0 m=nil [GC worker (idle), 12 minutes]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00009a750 sp=0xc00009a730 pc=0x7fa329b16c8e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00009a7e0 sp=0xc00009a750 pc=0x7fa329af6225
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00009a7e8 sp=0xc00009a7e0 pc=0x7fa329b4f1c1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 7 gp=0xc00027e380 m=nil [GC worker (idle), 12 minutes]:
runtime.gopark(0x2a1e38d11cd?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00009af50 sp=0xc00009af30 pc=0x7fa329b16c8e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00009afe0 sp=0xc00009af50 pc=0x7fa329af6225
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00009afe8 sp=0xc00009afe0 pc=0x7fa329b4f1c1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 8 gp=0xc00027e540 m=nil [GC worker (idle), 12 minutes]:
runtime.gopark(0x2a1e38d1177?, 0x3?, 0x2e?, 0x4e?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00009b750 sp=0xc00009b730 pc=0x7fa329b16c8e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00009b7e0 sp=0xc00009b750 pc=0x7fa329af6225
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00009b7e8 sp=0xc00009b7e0 pc=0x7fa329b4f1c1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 9 gp=0xc00027e700 m=nil [GC worker (idle), 12 minutes]:
runtime.gopark(0x2a1e38d0c0d?, 0x1?, 0x9f?, 0xe0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00009bf50 sp=0xc00009bf30 pc=0x7fa329b16c8e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00009bfe0 sp=0xc00009bf50 pc=0x7fa329af6225
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00009bfe8 sp=0xc00009bfe0 pc=0x7fa329b4f1c1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 10 gp=0xc00027e8c0 m=nil [GC worker (idle), 12 minutes]:
runtime.gopark(0x2a1e38caf38?, 0x3?, 0x2b?, 0xc?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000094750 sp=0xc000094730 pc=0x7fa329b16c8e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000947e0 sp=0xc000094750 pc=0x7fa329af6225
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000947e8 sp=0xc0000947e0 pc=0x7fa329b4f1c1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 11 gp=0xc00027ea80 m=nil [GC worker (idle)]:
runtime.gopark(0x353a7eb0138?, 0x1?, 0xf9?, 0xbd?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000094f50 sp=0xc000094f30 pc=0x7fa329b16c8e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000094fe0 sp=0xc000094f50 pc=0x7fa329af6225
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000094fe8 sp=0xc000094fe0 pc=0x7fa329b4f1c1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 12 gp=0xc00027ec40 m=nil [GC worker (idle), 12 minutes]:
runtime.gopark(0x2a1e38d13ed?, 0x3?, 0xd6?, 0x4?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000095750 sp=0xc000095730 pc=0x7fa329b16c8e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000957e0 sp=0xc000095750 pc=0x7fa329af6225
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000957e8 sp=0xc0000957e0 pc=0x7fa329b4f1c1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 13 gp=0xc00027ee00 m=nil [GC worker (idle)]:
runtime.gopark(0x353a7f39486?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000095f50 sp=0xc000095f30 pc=0x7fa329b16c8e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000095fe0 sp=0xc000095f50 pc=0x7fa329af6225
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000095fe8 sp=0xc000095fe0 pc=0x7fa329b4f1c1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 14 gp=0xc000603880 m=nil [select, 2 minutes]:
runtime.gopark(0xc000372780?, 0x2?, 0x31?, 0x7a?, 0xc000372774?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000372618 sp=0xc0003725f8 pc=0x7fa329b16c8e
runtime.selectgo(0xc000372780, 0xc000372770, 0x2020202020200a3e?, 0x0, 0x20200a3e7370756f?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc000372738 sp=0xc000372618 pc=0x7fa329b28b65
github.com/golang/glog.(*fileSink).flushDaemon(0x7fa33066a138)
        /go/pkg/mod/github.com/golang/glog@v1.2.0/glog_file.go:351 +0xb9 fp=0xc0003727c8 sp=0xc000372738 pc=0x7fa32a4c0039
github.com/golang/glog.init.1.gowrap1()
        /go/pkg/mod/github.com/golang/glog@v1.2.0/glog_file.go:166 +0x25 fp=0xc0003727e0 sp=0xc0003727c8 pc=0x7fa32a4bf0c5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0003727e8 sp=0xc0003727e0 pc=0x7fa329b4f1c1
created by github.com/golang/glog.init.1 in goroutine 1
        /go/pkg/mod/github.com/golang/glog@v1.2.0/glog_file.go:166 +0x126

goroutine 16 gp=0xc0001028c0 m=nil [sync.Cond.Wait, 2 minutes]:
runtime.gopark(0x7fa32f844140?, 0xc0002fa780?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000abc70 sp=0xc0000abc50 pc=0x7fa329b16c8e
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
sync.runtime_notifyListWait(0xc0004be0d8, 0x1)
        /usr/local/go/src/runtime/sema.go:569 +0x15a fp=0xc0000abcc0 sp=0xc0000abc70 pc=0x7fa329b4b03a
sync.(*Cond).Wait(0x2a735c223d6e7265?)
        /usr/local/go/src/sync/cond.go:70 +0x85 fp=0xc0000abd00 sp=0xc0000abcc0 pc=0x7fa329b58ee5
github.com/maypok86/otter/internal/queue.(*Growable[...]).Pop(0x7fa32f83d320)
        /go/pkg/mod/github.com/maypok86/otter@v1.2.0/internal/queue/growable.go:71 +0x74 fp=0xc0000abd98 sp=0xc0000abd00 pc=0x7fa32ad36f14
github.com/maypok86/otter/internal/core.(*Cache[...]).process(0x7fa32f85a780)
        /go/pkg/mod/github.com/maypok86/otter@v1.2.0/internal/core/cache.go:386 +0xe8 fp=0xc0000abfc0 sp=0xc0000abd98 pc=0x7fa32ad34da8
github.com/maypok86/otter/internal/core.NewCache[...].gowrap2()
        /go/pkg/mod/github.com/maypok86/otter@v1.2.0/internal/core/cache.go:167 +0x25 fp=0xc0000abfe0 sp=0xc0000abfc0 pc=0x7fa32ad2bfc5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000abfe8 sp=0xc0000abfe0 pc=0x7fa329b4f1c1
created by github.com/maypok86/otter/internal/core.NewCache[...] in goroutine 1
        /go/pkg/mod/github.com/maypok86/otter@v1.2.0/internal/core/cache.go:167 +0x6b3

goroutine 52 gp=0xc000103dc0 m=nil [select, 12 minutes]:
runtime.gopark(0xc000371ee8?, 0x3?, 0x2f?, 0x72?, 0xc000371eda?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000371d80 sp=0xc000371d60 pc=0x7fa329b16c8e
runtime.selectgo(0xc000371ee8, 0xc000371ed4, 0xc000371ed0?, 0x2, 0xc00011c8f0?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc000371ea0 sp=0xc000371d80 pc=0x7fa329b28b65
github.com/caddyserver/certmagic.(*RingBufferRateLimiter).permit(0xc000626500)
        /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/ratelimiter.go:217 +0x86 fp=0xc000371f28 sp=0xc000371ea0 pc=0x7fa329f99286
github.com/caddyserver/certmagic.(*RingBufferRateLimiter).loop(0xc000626500)
        /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/ratelimiter.go:89 +0x8b fp=0xc000371fc8 sp=0xc000371f28 pc=0x7fa329f98c4b
github.com/caddyserver/certmagic.NewRateLimiter.gowrap1()
        /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/ratelimiter.go:45 +0x25 fp=0xc000371fe0 sp=0xc000371fc8 pc=0x7fa329f98b85
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000371fe8 sp=0xc000371fe0 pc=0x7fa329b4f1c1
created by github.com/caddyserver/certmagic.NewRateLimiter in goroutine 1
        /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/ratelimiter.go:45 +0x13c

goroutine 36 gp=0xc0005028c0 m=nil [chan receive, 12 minutes]:
runtime.gopark(0x7fa32a115c05?, 0x7fa32f5d8600?, 0x1?, 0x7e?, 0xc0003716d8?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000371648 sp=0xc000371628 pc=0x7fa329b16c8e
runtime.chanrecv(0xc00060eb40, 0x0, 0x1)
        /usr/local/go/src/runtime/chan.go:583 +0x3bf fp=0xc0003716c0 sp=0xc000371648 pc=0x7fa329adf83f
runtime.chanrecv1(0xc00060eb40?, 0xc0003717a8?)
        /usr/local/go/src/runtime/chan.go:442 +0x12 fp=0xc0003716e8 sp=0xc0003716c0 pc=0x7fa329adf452
github.com/caddyserver/caddy/v2.trapSignalsCrossPlatform.func1()
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/sigtrap.go:43 +0xe5 fp=0xc0003717e0 sp=0xc0003716e8 pc=0x7fa32a137125
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0003717e8 sp=0xc0003717e0 pc=0x7fa329b4f1c1
created by github.com/caddyserver/caddy/v2.trapSignalsCrossPlatform in goroutine 1
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/sigtrap.go:38 +0x1a

goroutine 37 gp=0xc000582380 m=nil [chan receive, 12 minutes]:
runtime.gopark(0x7fa32a115be5?, 0x7fa32f5d8600?, 0x1?, 0x0?, 0xc0000adda8?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000add18 sp=0xc0000adcf8 pc=0x7fa329b16c8e
runtime.chanrecv(0xc000052f60, 0xc0000adf70, 0x1)
        /usr/local/go/src/runtime/chan.go:583 +0x3bf fp=0xc0000add90 sp=0xc0000add18 pc=0x7fa329adf83f
runtime.chanrecv2(0xc000052f60?, 0xc000373780?)
        /usr/local/go/src/runtime/chan.go:447 +0x12 fp=0xc0000addb8 sp=0xc0000add90 pc=0x7fa329adf472
github.com/caddyserver/caddy/v2.trapSignalsPosix.func1()
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/sigtrap_posix.go:35 +0x105 fp=0xc0000adfe0 sp=0xc0000addb8 pc=0x7fa32a1369c5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000adfe8 sp=0xc0000adfe0 pc=0x7fa329b4f1c1
created by github.com/caddyserver/caddy/v2.trapSignalsPosix in goroutine 1
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/sigtrap_posix.go:31 +0x1a

goroutine 53 gp=0xc00027f340 m=nil [select, 12 minutes, locked to thread]:
runtime.gopark(0xc00036d7a8?, 0x2?, 0x60?, 0xd6?, 0xc00036d794?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00036d638 sp=0xc00036d618 pc=0x7fa329b16c8e
runtime.selectgo(0xc00036d7a8, 0xc00036d790, 0x0?, 0x0, 0x0?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc00036d758 sp=0xc00036d638 pc=0x7fa329b28b65
runtime.ensureSigM.func1()
        /usr/local/go/src/runtime/signal_unix.go:1034 +0x1a5 fp=0xc00036d7e0 sp=0xc00036d758 pc=0x7fa329b45b05
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00036d7e8 sp=0xc00036d7e0 pc=0x7fa329b4f1c1
created by runtime.ensureSigM in goroutine 36
        /usr/local/go/src/runtime/signal_unix.go:1017 +0xc8

goroutine 54 gp=0xc00027f500 m=8 mp=0xc0000cd008 [syscall, 12 minutes]:
runtime.notetsleepg(0x7fa3306fd4c0, 0xffffffffffffffff)
        /usr/local/go/src/runtime/lock_futex.go:246 +0x29 fp=0xc00036dfa0 sp=0xc00036df78 pc=0x7fa329ae58c9
os/signal.signal_recv()
        /usr/local/go/src/runtime/sigqueue.go:152 +0x29 fp=0xc00036dfc0 sp=0xc00036dfa0 pc=0x7fa329b4b429
os/signal.loop()
        /usr/local/go/src/os/signal/signal_unix.go:23 +0x13 fp=0xc00036dfe0 sp=0xc00036dfc0 pc=0x7fa32a116393
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00036dfe8 sp=0xc00036dfe0 pc=0x7fa329b4f1c1
created by os/signal.Notify.func1.1 in goroutine 36
        /usr/local/go/src/os/signal/signal.go:151 +0x1f

goroutine 38 gp=0xc0005828c0 m=nil [select, 4 minutes]:
runtime.gopark(0xc000431f40?, 0x3?, 0x60?, 0x97?, 0xc000431e3a?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00055ace0 sp=0xc00055acc0 pc=0x7fa329b16c8e
runtime.selectgo(0xc00055af40, 0xc000431e34, 0xc00011a3c0?, 0x0, 0x0?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc00055ae00 sp=0xc00055ace0 pc=0x7fa329b28b65
github.com/caddyserver/certmagic.(*Cache).maintainAssets(0xc0003c9000, 0x0)
        /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/maintain.go:69 +0x31f fp=0xc00055afc0 sp=0xc00055ae00 pc=0x7fa329f8e77f
github.com/caddyserver/certmagic.NewCache.gowrap1()
        /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/cache.go:127 +0x25 fp=0xc00055afe0 sp=0xc00055afc0 pc=0x7fa329f6c1e5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00055afe8 sp=0xc00055afe0 pc=0x7fa329b4f1c1
created by github.com/caddyserver/certmagic.NewCache in goroutine 1
        /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/cache.go:127 +0x1f6

goroutine 39 gp=0xc000583180 m=nil [select, 12 minutes]:
runtime.gopark(0xc000071fa8?, 0x2?, 0x0?, 0x0?, 0xc000071f9c?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000a8e40 sp=0xc0000a8e20 pc=0x7fa329b16c8e
runtime.selectgo(0xc0000a8fa8, 0xc000071f98, 0x0?, 0x0, 0x0?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc0000a8f60 sp=0xc0000a8e40 pc=0x7fa329b28b65
github.com/caddyserver/caddy/v2/modules/caddytls.(*TLS).keepStorageClean.func1()
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddytls/tls.go:540 +0x93 fp=0xc0000a8fe0 sp=0xc0000a8f60 pc=0x7fa32a76f0d3
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000a8fe8 sp=0xc0000a8fe0 pc=0x7fa329b4f1c1
created by github.com/caddyserver/caddy/v2/modules/caddytls.(*TLS).keepStorageClean in goroutine 1
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddytls/tls.go:532 +0xe5

goroutine 40 gp=0xc000583340 m=nil [IO wait]:
runtime.gopark(0xc000078008?, 0x0?, 0x40?, 0x33?, 0xc000557c58?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000557c20 sp=0xc000557c00 pc=0x7fa329b16c8e
runtime.netpollblock(0xc000557cb8?, 0x29adc906?, 0xa3?)
        /usr/local/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc000557c58 sp=0xc000557c20 pc=0x7fa329b0ed77
internal/poll.runtime_pollWait(0x7fa3321b16d0, 0x72)
        /usr/local/go/src/runtime/netpoll.go:345 +0x85 fp=0xc000557c78 sp=0xc000557c58 pc=0x7fa329b491e5
internal/poll.(*pollDesc).wait(0x3?, 0x1?, 0x0)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc000557ca0 sp=0xc000557c78 pc=0x7fa329c02927
internal/poll.(*pollDesc).waitRead(...)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc0003fa880)
        /usr/local/go/src/internal/poll/fd_unix.go:611 +0x2ac fp=0xc000557d48 sp=0xc000557ca0 pc=0x7fa329c07ccc
net.(*netFD).accept(0xc0003fa880)
        /usr/local/go/src/net/fd_unix.go:172 +0x29 fp=0xc000557e00 sp=0xc000557d48 pc=0x7fa329cb33c9
net.(*TCPListener).accept(0xc000623080)
        /usr/local/go/src/net/tcpsock_posix.go:159 +0x1e fp=0xc000557e28 sp=0xc000557e00 pc=0x7fa329ccd71e
net.(*TCPListener).Accept(0xc000623080)
        /usr/local/go/src/net/tcpsock.go:327 +0x30 fp=0xc000557e58 sp=0xc000557e28 pc=0x7fa329ccc910
github.com/caddyserver/caddy/v2.(*deleteListener).Accept(0x7fa329e11b74?)
        <autogenerated>:1 +0x24 fp=0xc000557e70 sp=0xc000557e58 pc=0x7fa32a13ba84
net/http.(*onceCloseListener).Accept(0xc00011e2d0?)
        <autogenerated>:1 +0x24 fp=0xc000557e88 sp=0xc000557e70 pc=0x7fa329e355e4
net/http.(*Server).Serve(0xc00052fc20, {0x7fa331a667a0, 0xc0006230a0})
        /usr/local/go/src/net/http/server.go:3255 +0x33e fp=0xc000557fb8 sp=0xc000557e88 pc=0x7fa329e119fe
github.com/caddyserver/caddy/v2/modules/caddyhttp.(*App).Start.gowrap1()
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/app.go:509 +0x28 fp=0xc000557fe0 sp=0xc000557fb8 pc=0x7fa32a932e88
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000557fe8 sp=0xc000557fe0 pc=0x7fa329b4f1c1
created by github.com/caddyserver/caddy/v2/modules/caddyhttp.(*App).Start in goroutine 1
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/app.go:509 +0x1b95

goroutine 66 gp=0xc00027fa40 m=13 mp=0xc000101808 [syscall, locked to thread]:
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0005fbfe8 sp=0xc0005fbfe0 pc=0x7fa329b4f1c1

goroutine 83 gp=0xc000502e00 m=14 mp=0xc000501008 [syscall, locked to thread]:
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000aefe8 sp=0xc0000aefe0 pc=0x7fa329b4f1c1

goroutine 84 gp=0xc00057ca80 m=15 mp=0xc000600808 [syscall, locked to thread]:
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000a9fe8 sp=0xc0000a9fe0 pc=0x7fa329b4f1c1

goroutine 85 gp=0xc000583500 m=16 mp=0xc000580808 [syscall, 2 minutes, locked to thread]:
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000555fe8 sp=0xc000555fe0 pc=0x7fa329b4f1c1

goroutine 86 gp=0xc00057d500 m=17 mp=0xc000601008 [syscall, locked to thread]:
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000aafe8 sp=0xc0000aafe0 pc=0x7fa329b4f1c1

goroutine 87 gp=0xc000503500 m=18 mp=0xc000501808 [syscall, locked to thread]:
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0005f9fe8 sp=0xc0005f9fe0 pc=0x7fa329b4f1c1

goroutine 88 gp=0xc000583c00 m=19 mp=0xc000581008 [syscall, 2 minutes, locked to thread]:
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000558fe8 sp=0xc000558fe0 pc=0x7fa329b4f1c1

goroutine 161 gp=0xc000416c40 m=nil [chan receive]:
runtime.gopark(0x2?, 0xc0005c8320?, 0x48?, 0x0?, 0xc000834370?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000834320 sp=0xc000834300 pc=0x7fa329b16c8e
runtime.chanrecv(0xc00080c240, 0x0, 0x1)
        /usr/local/go/src/runtime/chan.go:583 +0x3bf fp=0xc000834398 sp=0xc000834320 pc=0x7fa329adf83f
runtime.chanrecv1(0xc000834420?, 0xc0008343f8?)
        /usr/local/go/src/runtime/chan.go:442 +0x12 fp=0xc0008343c0 sp=0xc000834398 pc=0x7fa329adf452
github.com/dunglas/frankenphp.ServeHTTP({0x7fa32f822230, 0xc000474410}, 0xc0007237a0)
        /go/src/app/frankenphp.go:465 +0x20e fp=0xc000834480 sp=0xc0008343c0 pc=0x7fa32ad270ee
github.com/dunglas/frankenphp/caddy.FrankenPHPModule.ServeHTTP({{0xc0006145ca, 0x6}, {0xc000537f40, 0x1, 0x1}, 0xc0006145c8, 0x0, 0xc0003fa600}, {0x7fa32f822230, 0xc000474410}, ...)
        /go/src/app/caddy/caddy.go:275 +0x4ac fp=0xc000834820 sp=0xc000834480 pc=0x7fa32ae2d54c
github.com/dunglas/frankenphp/caddy.(*FrankenPHPModule).ServeHTTP(0xc000834900?, {0x7fa32f822230?, 0xc000474410?}, 0x7fa32f653c00?, {0x7fa32f814260?, 0xc0002cc4e0?})
        <autogenerated>:1 +0xe5 fp=0xc0008348d8 sp=0xc000834820 pc=0x7fa32ae336a5
github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapMiddleware.func1.1({0x7fa32f822230?, 0xc000474410?}, 0xc00030cbc0?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:331 +0x3a fp=0xc000834918 sp=0xc0008348d8 pc=0x7fa32a954cba
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x7fa32f814260?, {0x7fa32f822230?, 0xc000474410?}, 0xc000723440?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000834940 sp=0xc000834918 pc=0x7fa32a9398a9
github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.Compile.wrapRoute.func1.1({0x7fa32f822230, 0xc000474410}, 0xc000723440)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:300 +0x325 fp=0xc0008349b8 sp=0xc000834940 pc=0x7fa32a954a25
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x0?, {0x7fa32f822230?, 0xc000474410?}, 0xc00081c780?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc0008349e0 sp=0xc0008349b8 pc=0x7fa32a9398a9
github.com/caddyserver/caddy/v2/modules/caddyhttp/rewrite.Rewrite.ServeHTTP({{0x0, 0x0}, {0xc0004c6e00, 0x1d}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0, ...}, ...}, ...)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/rewrite/rewrite.go:137 +0x3f3 fp=0xc000834b68 sp=0xc0008349e0 pc=0x7fa32ab88093
github.com/caddyserver/caddy/v2/modules/caddyhttp/rewrite.(*Rewrite).ServeHTTP(0xc000834cb8?, {0x7fa32f822230?, 0xc000474410?}, 0x7fa32f653c00?, {0x7fa32f814260?, 0xc0002cc500?})
        <autogenerated>:1 +0xd6 fp=0xc000834c90 sp=0xc000834b68 pc=0x7fa32ab8a0b6
github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapMiddleware.func1.1({0x7fa32f822230?, 0xc000474410?}, 0xc0003fa380?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:331 +0x3a fp=0xc000834cd0 sp=0xc000834c90 pc=0x7fa32a954cba
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x7fa32f814260?, {0x7fa32f822230?, 0xc000474410?}, 0xc000723440?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000834cf8 sp=0xc000834cd0 pc=0x7fa32a9398a9
github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.Compile.wrapRoute.func1.1({0x7fa32f822230, 0xc000474410}, 0xc000723440)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:300 +0x325 fp=0xc000834d70 sp=0xc000834cf8 pc=0x7fa32a954a25
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x7fa32f826660?, {0x7fa32f822230?, 0xc000474410?}, 0xe?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000834d98 sp=0xc000834d70 pc=0x7fa32a9398a9
github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.Compile.wrapRoute.func1.1({0x7fa32f822230, 0xc000474410}, 0xc000723440)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:268 +0x244 fp=0xc000834e10 sp=0xc000834d98 pc=0x7fa32a954944
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0xc000476700?, {0x7fa32f822230?, 0xc000474410?}, 0x2?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000834e38 sp=0xc000834e10 pc=0x7fa32a9398a9
github.com/caddyserver/caddy/v2/modules/caddyhttp/encode.(*Encode).ServeHTTP(0xc00030ca80, {0x7fa32f821b40, 0xc0002c62a0}, 0xc000723440, {0x7fa32f814260, 0xc0002cc540})
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/encode/encode.go:138 +0x252 fp=0xc000834ea0 sp=0xc000834e38 pc=0x7fa32a99d812
github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapMiddleware.func1.1({0x7fa32f821b40?, 0xc0002c62a0?}, 0xc00030ca80?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:331 +0x3a fp=0xc000834ee0 sp=0xc000834ea0 pc=0x7fa32a954cba
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x7fa32f814260?, {0x7fa32f821b40?, 0xc0002c62a0?}, 0xc000723440?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000834f08 sp=0xc000834ee0 pc=0x7fa32a9398a9
github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.Compile.wrapRoute.func1.1({0x7fa32f821b40, 0xc0002c62a0}, 0xc000723440)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:300 +0x325 fp=0xc000834f80 sp=0xc000834f08 pc=0x7fa32a954a25
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0xc0001c0a08?, {0x7fa32f821b40?, 0xc0002c62a0?}, 0x7fa32f814260?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000834fa8 sp=0xc000834f80 pc=0x7fa32a9398a9
github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Subroute).ServeHTTP(0xc000622e00, {0x7fa32f821b40, 0xc0002c62a0}, 0xc000723440, {0x7fa32f814260, 0x7fa32f807dd8})
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/subroute.go:74 +0x67 fp=0xc000834fe8 sp=0xc000834fa8 pc=0x7fa32a95cb87
github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapMiddleware.func1.1({0x7fa32f821b40?, 0xc0002c62a0?}, 0xc000622e00?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:331 +0x3a fp=0xc000835028 sp=0xc000834fe8 pc=0x7fa32a954cba
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x7fa32f814260?, {0x7fa32f821b40?, 0xc0002c62a0?}, 0xc000723440?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000835050 sp=0xc000835028 pc=0x7fa32a9398a9
github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.Compile.wrapRoute.func1.1({0x7fa32f821b40, 0xc0002c62a0}, 0xc000723440)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:300 +0x325 fp=0xc0008350c8 sp=0xc000835050 pc=0x7fa32a954a25
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x7fa32f757000?, {0x7fa32f821b40?, 0xc0002c62a0?}, 0xc000476650?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc0008350f0 sp=0xc0008350c8 pc=0x7fa32a9398a9
github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Server).enforcementHandler(0x10?, {0x7fa32f821b40?, 0xc0002c62a0?}, 0x0?, {0x7fa32f814260?, 0xc000622fe0?})
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/server.go:429 +0x24b fp=0xc0008351c8 sp=0xc0008350f0 pc=0x7fa32a9571ab
github.com/caddyserver/caddy/v2/modules/caddyhttp.(*App).Provision.(*Server).wrapPrimaryRoute.func1({0x7fa32f821b40?, 0xc0002c62a0?}, 0x7fa329b81c0f?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/server.go:405 +0x35 fp=0xc000835208 sp=0xc0008351c8 pc=0x7fa32a930555
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0xc000200e00?, {0x7fa32f821b40?, 0xc0002c62a0?}, 0xc000723440?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000835230 sp=0xc000835208 pc=0x7fa32a9398a9
github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Server).ServeHTTP(0xc00050eb08, {0x7fa32f821b40, 0xc0002c62a0}, 0xc0007230e0)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/server.go:341 +0xc47 fp=0xc000835b60 sp=0xc000835230 pc=0x7fa32a955fa7
net/http.serverHandler.ServeHTTP({0xc000472ed0?}, {0x7fa32f821b40?, 0xc0002c62a0?}, 0x6?)
        /usr/local/go/src/net/http/server.go:3137 +0x8e fp=0xc000835b90 sp=0xc000835b60 pc=0x7fa329e115ce
net/http.(*conn).serve(0xc00011e2d0, {0x7fa32f826660, 0xc00024e120})
        /usr/local/go/src/net/http/server.go:2039 +0x5e8 fp=0xc000835fb8 sp=0xc000835b90 pc=0x7fa329e0c8a8
net/http.(*Server).Serve.gowrap3()
        /usr/local/go/src/net/http/server.go:3285 +0x28 fp=0xc000835fe0 sp=0xc000835fb8 pc=0x7fa329e11de8
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000835fe8 sp=0xc000835fe0 pc=0x7fa329b4f1c1
created by net/http.(*Server).Serve in goroutine 40
        /usr/local/go/src/net/http/server.go:3285 +0x4b4

goroutine 178 gp=0xc00080e540 m=nil [IO wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0xb?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0006b8da8 sp=0xc0006b8d88 pc=0x7fa329b16c8e
runtime.netpollblock(0x7fa329b737b8?, 0x29adc906?, 0xa3?)
        /usr/local/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc0006b8de0 sp=0xc0006b8da8 pc=0x7fa329b0ed77
internal/poll.runtime_pollWait(0x7fa3321b15d8, 0x72)
        /usr/local/go/src/runtime/netpoll.go:345 +0x85 fp=0xc0006b8e00 sp=0xc0006b8de0 pc=0x7fa329b491e5
internal/poll.(*pollDesc).wait(0xc000680180?, 0xc000472ee1?, 0x0)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0006b8e28 sp=0xc0006b8e00 pc=0x7fa329c02927
internal/poll.(*pollDesc).waitRead(...)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000680180, {0xc000472ee1, 0x1, 0x1})
        /usr/local/go/src/internal/poll/fd_unix.go:164 +0x27a fp=0xc0006b8ec0 sp=0xc0006b8e28 pc=0x7fa329c03c1a
net.(*netFD).Read(0xc000680180, {0xc000472ee1?, 0x0?, 0x0?})
        /usr/local/go/src/net/fd_posix.go:55 +0x25 fp=0xc0006b8f08 sp=0xc0006b8ec0 pc=0x7fa329cb13e5
net.(*conn).Read(0xc000532018, {0xc000472ee1?, 0x0?, 0x0?})
        /usr/local/go/src/net/net.go:179 +0x45 fp=0xc0006b8f50 sp=0xc0006b8f08 pc=0x7fa329cc48a5
net.(*TCPConn).Read(0x0?, {0xc000472ee1?, 0x0?, 0x0?})
        <autogenerated>:1 +0x25 fp=0xc0006b8f80 sp=0xc0006b8f50 pc=0x7fa329cd5c25
net/http.(*connReader).backgroundRead(0xc000472ed0)
        /usr/local/go/src/net/http/server.go:681 +0x37 fp=0xc0006b8fc8 sp=0xc0006b8f80 pc=0x7fa329e06817
net/http.(*connReader).startBackgroundRead.gowrap2()
        /usr/local/go/src/net/http/server.go:677 +0x25 fp=0xc0006b8fe0 sp=0xc0006b8fc8 pc=0x7fa329e06745
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0006b8fe8 sp=0xc0006b8fe0 pc=0x7fa329b4f1c1
created by net/http.(*connReader).startBackgroundRead in goroutine 161
        /usr/local/go/src/net/http/server.go:677 +0xba

rax    0x55
rbx    0x7fa331540b28
rcx    0x0
rdx    0xd
rdi    0x7fa32c5e92ff
rsi    0x7fa330ec93f0
rbp    0x7fa32c5e92ff
rsp    0x7fa33111f668
r8     0xfc
r9     0x8080808080808001
r10    0x7fa32c5e92ff
r11    0x1
r12    0xe
r13    0x4
r14    0x20
r15    0x7fa3315b9480
rip    0x7fa32c091b10
rflags 0x10246
cs     0x33
fs     0x0
gs     0x0
@m-hm m-hm added the bug Something isn't working label Mar 19, 2024
@dunglas
Copy link
Owner

dunglas commented Mar 19, 2024

Can you show your Dockerfile, please?

@m-hm
Copy link
Author

m-hm commented Mar 19, 2024

No Dockerfile, Download from https://github.com/dunglas/frankenphp/releases/download/v1.1.1/frankenphp-linux-x86_64

@dunglas
Copy link
Owner

dunglas commented Mar 19, 2024

I tried v1.1.1 on an Ubuntu (x86_64) machine and a Mac and could not reproduce the crash.
Could you provide a minimal reproducer, please? Also, could you try downgrading to v1.1.0 to test if it fixes the issue?

Thanks.

@dunglas dunglas added the question Further information is requested label Mar 19, 2024
@emaia
Copy link

emaia commented Mar 19, 2024 •
edited
Loading

Hi! I'm also getting this same error sporadically but with some frequency.
I'm running on WSL2 with local binary, downloaded from https://github.com/dunglas/frankenphp/releases/tag/v1.1.1 Same error with v1.1.0

Caddyfile:

{
    frankenphp
    order php_server before file_server
}

example.localhost {
    root * public/
    encode zstd br gzip
    php_server
}

I'm running a Laravel app with bun run dev (to compile Tailwind with Vite). This error happens when I change some file (JS, CSS or Blade) and hot reload occurs.

@dunglas
Copy link
Owner

dunglas commented Mar 19, 2024

I suspect that this is this bug: php/php-src#13648

Could you try to gather a stack trace using a debug build to confirm that this is it? In this case, a workaround is to use the Docker image or to compile FrankenPHP with a dynamic link to libphp locally.

@emaia
Copy link

emaia commented Mar 22, 2024

@dunglas It works without errors using Docker version.

My output with debug binary:

SIGSEGV: segmentation violation
PC=0x7f4d9ac01a1b m=24 sigcode=1 addr=0x7f4d4462be70
signal arrived during cgo execution

goroutine 182 gp=0xc000bb21c0 m=24 mp=0xc000601008 [syscall, locked to thread]:
runtime.cgocall(0x7f4d99240ce0, 0xc000bbdce0)
        /usr/local/go/src/runtime/cgocall.go:157 +0x4b fp=0xc000bbdcb8 sp=0xc000bbdc80 pc=0x7f4d97e70d6b
github.com/dunglas/frankenphp._Cfunc_frankenphp_update_server_context(0x1, 0xa, 0x0, 0x7f4d43bc9120, 0x7f4d43bc9140, 0x0, 0x0, 0x7f4d43bc9150, 0x0, 0x0, ...)
        _cgo_gotypes.go:1060 +0x52 fp=0xc000bbdce0 sp=0xc000bbdcb8 pc=0x7f4d990b84b2
github.com/dunglas/frankenphp.updateServerContext(0xc000d06b40, 0x1, 0x0)
        /go/src/app/frankenphp.go:419 +0x4e5 fp=0xc000bbddd8 sp=0xc000bbdce0 pc=0x7f4d990baa05
github.com/dunglas/frankenphp.go_execute_script(0x7f4d97ee0d01?)
        /go/src/app/frankenphp.go:506 +0x10f fp=0xc000bbde40 sp=0xc000bbddd8 pc=0x7f4d990bb10f
_cgoexp_feab5cfeb5bc_go_execute_script(0x7f4d9e3f16b0?)
        _cgo_gotypes.go:1111 +0x16 fp=0xc000bbde58 sp=0xc000bbde40 pc=0x7f4d990c0816
runtime.cgocallbackg1(0x7f4d990c0800, 0x7f4d455ca850, 0x0)
        /usr/local/go/src/runtime/cgocall.go:403 +0x2a5 fp=0xc000bbdf18 sp=0xc000bbde58 pc=0x7f4d97e714a5
runtime.cgocallbackg(0x7f4d990c0800, 0x7f4d455ca850, 0x0)
        /usr/local/go/src/runtime/cgocall.go:322 +0x138 fp=0xc000bbdf90 sp=0xc000bbdf18 pc=0x7f4d97e71158
runtime.cgocallbackg(0x7f4d990c0800, 0x7f4d455ca850, 0x0)
        <autogenerated>:1 +0x2b fp=0xc000bbdfb8 sp=0xc000bbdf90 pc=0x7f4d97ee54eb
runtime.cgocallback(0x0, 0x0, 0x0)
        /usr/local/go/src/runtime/asm_amd64.s:1079 +0xcd fp=0xc000bbdfe0 sp=0xc000bbdfb8 pc=0x7f4d97ee2b2d
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000bbdfe8 sp=0xc000bbdfe0 pc=0x7f4d97ee2d81

goroutine 1 gp=0xc0000061c0 m=nil [select (no cases)]:
runtime.gopark(0xc00091f840?, 0x7f4d97fa1b4c?, 0x77?, 0x2?, 0x4?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000af9820 sp=0xc000af9800 pc=0x7f4d97eaa84e
runtime.block()
        /usr/local/go/src/runtime/select.go:103 +0x26 fp=0xc000af9850 sp=0xc000af9820 pc=0x7f4d97ebbfe6
github.com/caddyserver/caddy/v2/cmd.cmdRun({0x0?})
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/cmd/commandfuncs.go:283 +0xbfc fp=0xc000af9b60 sp=0xc000af9850 pc=0x7f4d98598a9c
github.com/caddyserver/caddy/v2/cmd.init.1.func2.WrapCommandFuncForCobra.1(0xc000156c08, {0x7f4d9ac1048f?, 0x4?, 0x7f4d9ac10297?})
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/cmd/cobra.go:137 +0x2f fp=0xc000af9ba0 sp=0xc000af9b60 pc=0x7f4d985a52af
github.com/spf13/cobra.(*Command).execute(0xc000156c08, {0x7f4d9f2e8000, 0x0, 0x0})
        /go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:983 +0xaca fp=0xc000af9d28 sp=0xc000af9ba0 pc=0x7f4d98519b4a
github.com/spf13/cobra.(*Command).ExecuteC(0x7f4d9f189040)
        /go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1115 +0x3ff fp=0xc000af9e00 sp=0xc000af9d28 pc=0x7f4d9851a41f
github.com/spf13/cobra.(*Command).Execute(...)
        /go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1039
github.com/caddyserver/caddy/v2/cmd.Main()
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/cmd/main.go:66 +0x5b fp=0xc000af9e60 sp=0xc000af9e00 pc=0x7f4d9859d31b
main.main()
        /go/src/app/caddy/frankenphp/main.go:24 +0x18d fp=0xc000af9f50 sp=0xc000af9e60 pc=0x7f4d9923fd8d
runtime.main()
        /usr/local/go/src/runtime/proc.go:271 +0x29d fp=0xc000af9fe0 sp=0xc000af9f50 pc=0x7f4d97eaa3fd
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000af9fe8 sp=0xc000af9fe0 pc=0x7f4d97ee2d81

goroutine 17 gp=0xc000006700 m=nil [select, locked to thread]:
runtime.gopark(0xc0000d6e10?, 0x2?, 0x8?, 0x0?, 0xc0000d6df4?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000d6c90 sp=0xc0000d6c70 pc=0x7f4d97eaa84e
runtime.selectgo(0xc0000d6e10, 0xc0000d6df0, 0x7f4d9f2e8000?, 0x0, 0x0?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc0000d6db0 sp=0xc0000d6c90 pc=0x7f4d97ebc725
github.com/dunglas/frankenphp.go_fetch_request()
        /go/src/app/frankenphp.go:473 +0x75 fp=0xc0000d6e48 sp=0xc0000d6db0 pc=0x7f4d990bae55
_cgoexp_feab5cfeb5bc_go_fetch_request(0x7f4d5001a890)
        _cgo_gotypes.go:1103 +0x14 fp=0xc0000d6e58 sp=0xc0000d6e48 pc=0x7f4d990c07d4
runtime.cgocallbackg1(0x7f4d990c07c0, 0x7f4d5001a890, 0x0)
        /usr/local/go/src/runtime/cgocall.go:403 +0x2a5 fp=0xc0000d6f18 sp=0xc0000d6e58 pc=0x7f4d97e714a5
runtime.cgocallbackg(0x7f4d990c07c0, 0x7f4d5001a890, 0x0)
        /usr/local/go/src/runtime/cgocall.go:322 +0x138 fp=0xc0000d6f90 sp=0xc0000d6f18 pc=0x7f4d97e71158
runtime.cgocallbackg(0x7f4d990c07c0, 0x7f4d5001a890, 0x0)
        <autogenerated>:1 +0x2b fp=0xc0000d6fb8 sp=0xc0000d6f90 pc=0x7f4d97ee54eb
runtime.cgocallback(0x0, 0x0, 0x0)
        /usr/local/go/src/runtime/asm_amd64.s:1079 +0xcd fp=0xc0000d6fe0 sp=0xc0000d6fb8 pc=0x7f4d97ee2b2d
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000d6fe8 sp=0xc0000d6fe0 pc=0x7f4d97ee2d81

goroutine 2 gp=0xc000006c40 m=nil [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c2fa8 sp=0xc0000c2f88 pc=0x7f4d97eaa84e
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.forcegchelper()
        /usr/local/go/src/runtime/proc.go:326 +0xb8 fp=0xc0000c2fe0 sp=0xc0000c2fa8 pc=0x7f4d97eaa6b8
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c2fe8 sp=0xc0000c2fe0 pc=0x7f4d97ee2d81
created by runtime.init.6 in goroutine 1
        /usr/local/go/src/runtime/proc.go:314 +0x1a

goroutine 3 gp=0xc000007180 m=nil [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c3780 sp=0xc0000c3760 pc=0x7f4d97eaa84e
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.bgsweep(0xc0000ea000)
        /usr/local/go/src/runtime/mgcsweep.go:318 +0xdf fp=0xc0000c37c8 sp=0xc0000c3780 pc=0x7f4d97e9341f
runtime.gcenable.gowrap1()
        /usr/local/go/src/runtime/mgc.go:203 +0x25 fp=0xc0000c37e0 sp=0xc0000c37c8 pc=0x7f4d97e87ce5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c37e8 sp=0xc0000c37e0 pc=0x7f4d97ee2d81
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:203 +0x66

goroutine 4 gp=0xc000007340 m=nil [GC scavenge wait]:
runtime.gopark(0x7f4d9f2569e0?, 0x7f4d9b122860?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c3f78 sp=0xc0000c3f58 pc=0x7f4d97eaa84e
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
runtime.(*scavengerState).park(0x7f4d9f2569e0)
        /usr/local/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc0000c3fa8 sp=0xc0000c3f78 pc=0x7f4d97e90da9
runtime.bgscavenge(0xc0000ea000)
        /usr/local/go/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc0000c3fc8 sp=0xc0000c3fa8 pc=0x7f4d97e91359
runtime.gcenable.gowrap2()
        /usr/local/go/src/runtime/mgc.go:204 +0x25 fp=0xc0000c3fe0 sp=0xc0000c3fc8 pc=0x7f4d97e87c85
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c3fe8 sp=0xc0000c3fe0 pc=0x7f4d97ee2d81
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:204 +0xa5

goroutine 5 gp=0xc000007c00 m=nil [finalizer wait]:
runtime.gopark(0xc0000c2648?, 0x7f4d97e7ae65?, 0xa8?, 0x1?, 0xc0000061c0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c2620 sp=0xc0000c2600 pc=0x7f4d97eaa84e
runtime.runfinq()
        /usr/local/go/src/runtime/mfinal.go:194 +0x107 fp=0xc0000c27e0 sp=0xc0000c2620 pc=0x7f4d97e86d27
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c27e8 sp=0xc0000c27e0 pc=0x7f4d97ee2d81
created by runtime.createfing in goroutine 1
        /usr/local/go/src/runtime/mfinal.go:164 +0x3d

goroutine 6 gp=0xc0002b21c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c4750 sp=0xc0000c4730 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000c47e0 sp=0xc0000c4750 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c47e8 sp=0xc0000c47e0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 18 gp=0xc000102700 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000be750 sp=0xc0000be730 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000be7e0 sp=0xc0000be750 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000be7e8 sp=0xc0000be7e0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 34 gp=0xc000502000 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000508750 sp=0xc000508730 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0005087e0 sp=0xc000508750 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0005087e8 sp=0xc0005087e0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 7 gp=0xc0002b2380 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c4f50 sp=0xc0000c4f30 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000c4fe0 sp=0xc0000c4f50 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c4fe8 sp=0xc0000c4fe0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 19 gp=0xc0001028c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000bef50 sp=0xc0000bef30 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000befe0 sp=0xc0000bef50 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000befe8 sp=0xc0000befe0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 35 gp=0xc0005021c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000508f50 sp=0xc000508f30 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000508fe0 sp=0xc000508f50 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000508fe8 sp=0xc000508fe0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 8 gp=0xc0002b2540 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c5750 sp=0xc0000c5730 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000c57e0 sp=0xc0000c5750 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c57e8 sp=0xc0000c57e0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 20 gp=0xc000102a80 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000bf750 sp=0xc0000bf730 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000bf7e0 sp=0xc0000bf750 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000bf7e8 sp=0xc0000bf7e0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 36 gp=0xc000502380 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000509750 sp=0xc000509730 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0005097e0 sp=0xc000509750 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0005097e8 sp=0xc0005097e0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 9 gp=0xc0002b2700 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c5f50 sp=0xc0000c5f30 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000c5fe0 sp=0xc0000c5f50 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c5fe8 sp=0xc0000c5fe0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 21 gp=0xc000102c40 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000bff50 sp=0xc0000bff30 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000bffe0 sp=0xc0000bff50 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000bffe8 sp=0xc0000bffe0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 37 gp=0xc000502540 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000509f50 sp=0xc000509f30 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000509fe0 sp=0xc000509f50 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000509fe8 sp=0xc000509fe0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 10 gp=0xc0002b28c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000504750 sp=0xc000504730 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0005047e0 sp=0xc000504750 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0005047e8 sp=0xc0005047e0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 22 gp=0xc000102e00 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c0750 sp=0xc0000c0730 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000c07e0 sp=0xc0000c0750 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c07e8 sp=0xc0000c07e0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 38 gp=0xc000502700 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00050a750 sp=0xc00050a730 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00050a7e0 sp=0xc00050a750 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00050a7e8 sp=0xc00050a7e0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 11 gp=0xc0002b2a80 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000504f50 sp=0xc000504f30 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000504fe0 sp=0xc000504f50 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000504fe8 sp=0xc000504fe0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 23 gp=0xc000102fc0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c0f50 sp=0xc0000c0f30 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000c0fe0 sp=0xc0000c0f50 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c0fe8 sp=0xc0000c0fe0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 39 gp=0xc0005028c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00050af50 sp=0xc00050af30 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00050afe0 sp=0xc00050af50 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00050afe8 sp=0xc00050afe0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 24 gp=0xc000103180 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c1750 sp=0xc0000c1730 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000c17e0 sp=0xc0000c1750 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c17e8 sp=0xc0000c17e0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 12 gp=0xc0002b2c40 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000505750 sp=0xc000505730 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0005057e0 sp=0xc000505750 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0005057e8 sp=0xc0005057e0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 40 gp=0xc000502a80 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00050b750 sp=0xc00050b730 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00050b7e0 sp=0xc00050b750 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00050b7e8 sp=0xc00050b7e0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 13 gp=0xc0002b2e00 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000505f50 sp=0xc000505f30 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000505fe0 sp=0xc000505f50 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000505fe8 sp=0xc000505fe0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 25 gp=0xc000103340 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000c1f50 sp=0xc0000c1f30 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0000c1fe0 sp=0xc0000c1f50 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000c1fe8 sp=0xc0000c1fe0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 41 gp=0xc000502c40 m=nil [GC worker (idle)]:
runtime.gopark(0x7f4d9f2ea7e0?, 0x1?, 0x94?, 0x57?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00050bf50 sp=0xc00050bf30 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00050bfe0 sp=0xc00050bf50 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00050bfe8 sp=0xc00050bfe0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 14 gp=0xc0002b2fc0 m=nil [GC worker (idle)]:
runtime.gopark(0x7f4d9f2ea7e0?, 0x1?, 0xab?, 0x71?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000506750 sp=0xc000506730 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0005067e0 sp=0xc000506750 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0005067e8 sp=0xc0005067e0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 26 gp=0xc000103500 m=nil [GC worker (idle)]:
runtime.gopark(0x75b188bf8f6b?, 0x1?, 0xbc?, 0x5d?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00011e750 sp=0xc00011e730 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00011e7e0 sp=0xc00011e750 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00011e7e8 sp=0xc00011e7e0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 42 gp=0xc000502e00 m=nil [GC worker (idle)]:
runtime.gopark(0x75af833a6444?, 0x1?, 0xbe?, 0x9b?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00011a750 sp=0xc00011a730 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00011a7e0 sp=0xc00011a750 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00011a7e8 sp=0xc00011a7e0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 15 gp=0xc0002b3180 m=nil [GC worker (idle)]:
runtime.gopark(0x75b188bb7a40?, 0x1?, 0x64?, 0x4b?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000506f50 sp=0xc000506f30 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc000506fe0 sp=0xc000506f50 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000506fe8 sp=0xc000506fe0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 27 gp=0xc0001036c0 m=nil [GC worker (idle)]:
runtime.gopark(0x75b188b472a3?, 0x1?, 0xff?, 0x4c?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00011ef50 sp=0xc00011ef30 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00011efe0 sp=0xc00011ef50 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00011efe8 sp=0xc00011efe0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 43 gp=0xc000502fc0 m=nil [GC worker (idle)]:
runtime.gopark(0x7f4d9f2ea7e0?, 0x1?, 0xa0?, 0xd6?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00011af50 sp=0xc00011af30 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00011afe0 sp=0xc00011af50 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00011afe8 sp=0xc00011afe0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 16 gp=0xc0002b3340 m=nil [GC worker (idle)]:
runtime.gopark(0x7f4d9f2ea7e0?, 0x1?, 0x63?, 0x3e?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000507750 sp=0xc000507730 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc0005077e0 sp=0xc000507750 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0005077e8 sp=0xc0005077e0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 28 gp=0xc000103880 m=nil [GC worker (idle)]:
runtime.gopark(0x7f4d9f2ea7e0?, 0x1?, 0xea?, 0x69?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00011f750 sp=0xc00011f730 pc=0x7f4d97eaa84e
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1310 +0xe5 fp=0xc00011f7e0 sp=0xc00011f750 pc=0x7f4d97e89de5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00011f7e8 sp=0xc00011f7e0 pc=0x7f4d97ee2d81
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1234 +0x1c

goroutine 50 gp=0xc0005836c0 m=nil [select]:
runtime.gopark(0xc000121f80?, 0x2?, 0xf1?, 0xb5?, 0xc000121f74?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000121e18 sp=0xc000121df8 pc=0x7f4d97eaa84e
runtime.selectgo(0xc000121f80, 0xc000121f70, 0x0?, 0x0, 0x0?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc000121f38 sp=0xc000121e18 pc=0x7f4d97ebc725
github.com/golang/glog.(*fileSink).flushDaemon(0x7f4d9f256438)
        /go/pkg/mod/github.com/golang/glog@v1.2.0/glog_file.go:351 +0xb9 fp=0xc000121fc8 sp=0xc000121f38 pc=0x7f4d98853bf9
github.com/golang/glog.init.1.gowrap1()
        /go/pkg/mod/github.com/golang/glog@v1.2.0/glog_file.go:166 +0x25 fp=0xc000121fe0 sp=0xc000121fc8 pc=0x7f4d98852c85
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000121fe8 sp=0xc000121fe0 pc=0x7f4d97ee2d81
created by github.com/golang/glog.init.1 in goroutine 1
        /go/pkg/mod/github.com/golang/glog@v1.2.0/glog_file.go:166 +0x126

goroutine 30 gp=0xc000583c00 m=nil [sync.Cond.Wait]:
runtime.gopark(0x7f4d9e42e220?, 0xc000314080?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000d5c70 sp=0xc0000d5c50 pc=0x7f4d97eaa84e
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:408
sync.runtime_notifyListWait(0xc0005100d8, 0x3)
        /usr/local/go/src/runtime/sema.go:569 +0x15a fp=0xc0000d5cc0 sp=0xc0000d5c70 pc=0x7f4d97edebfa
sync.(*Cond).Wait(0xc00011b538?)
        /usr/local/go/src/sync/cond.go:70 +0x85 fp=0xc0000d5d00 sp=0xc0000d5cc0 pc=0x7f4d97eecaa5
github.com/maypok86/otter/internal/queue.(*Growable[...]).Pop(0x7f4d9e427400)
        /go/pkg/mod/github.com/maypok86/otter@v1.2.0/internal/queue/growable.go:71 +0x74 fp=0xc0000d5d98 sp=0xc0000d5d00 pc=0x7f4d990caad4
github.com/maypok86/otter/internal/core.(*Cache[...]).process(0x7f4d9e444860)
        /go/pkg/mod/github.com/maypok86/otter@v1.2.0/internal/core/cache.go:386 +0xe8 fp=0xc0000d5fc0 sp=0xc0000d5d98 pc=0x7f4d990c8968
github.com/maypok86/otter/internal/core.NewCache[...].gowrap2()
        /go/pkg/mod/github.com/maypok86/otter@v1.2.0/internal/core/cache.go:167 +0x25 fp=0xc0000d5fe0 sp=0xc0000d5fc0 pc=0x7f4d990bfb85
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000d5fe8 sp=0xc0000d5fe0 pc=0x7f4d97ee2d81
created by github.com/maypok86/otter/internal/core.NewCache[...] in goroutine 1
        /go/pkg/mod/github.com/maypok86/otter@v1.2.0/internal/core/cache.go:167 +0x6b3

goroutine 66 gp=0xc000583500 m=nil [select]:
runtime.gopark(0xc0001216e8?, 0x3?, 0x0?, 0x0?, 0xc0001216da?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000121580 sp=0xc000121560 pc=0x7f4d97eaa84e
runtime.selectgo(0xc0001216e8, 0xc0001216d4, 0xc0001216d0?, 0x2, 0xc000428140?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc0001216a0 sp=0xc000121580 pc=0x7f4d97ebc725
github.com/caddyserver/certmagic.(*RingBufferRateLimiter).permit(0xc0000f8eb0)
        /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/ratelimiter.go:217 +0x86 fp=0xc000121728 sp=0xc0001216a0 pc=0x7f4d9832ce46
github.com/caddyserver/certmagic.(*RingBufferRateLimiter).loop(0xc0000f8eb0)
        /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/ratelimiter.go:89 +0x8b fp=0xc0001217c8 sp=0xc000121728 pc=0x7f4d9832c80b
github.com/caddyserver/certmagic.NewRateLimiter.gowrap1()
        /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/ratelimiter.go:45 +0x25 fp=0xc0001217e0 sp=0xc0001217c8 pc=0x7f4d9832c745
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0001217e8 sp=0xc0001217e0 pc=0x7f4d97ee2d81
created by github.com/caddyserver/certmagic.NewRateLimiter in goroutine 1
        /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/ratelimiter.go:45 +0x13c

goroutine 46 gp=0xc0008228c0 m=nil [chan receive]:
runtime.gopark(0x7f4d984a97c5?, 0x7f4d9e1c26e0?, 0x1?, 0x9e?, 0xc000695ed8?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000695e48 sp=0xc000695e28 pc=0x7f4d97eaa84e
runtime.chanrecv(0xc0000eec00, 0x0, 0x1)
        /usr/local/go/src/runtime/chan.go:583 +0x3bf fp=0xc000695ec0 sp=0xc000695e48 pc=0x7f4d97e733ff
runtime.chanrecv1(0xc0000eec00?, 0xc000695fa8?)
        /usr/local/go/src/runtime/chan.go:442 +0x12 fp=0xc000695ee8 sp=0xc000695ec0 pc=0x7f4d97e73012
github.com/caddyserver/caddy/v2.trapSignalsCrossPlatform.func1()
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/sigtrap.go:43 +0xe5 fp=0xc000695fe0 sp=0xc000695ee8 pc=0x7f4d984cace5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000695fe8 sp=0xc000695fe0 pc=0x7f4d97ee2d81
created by github.com/caddyserver/caddy/v2.trapSignalsCrossPlatform in goroutine 1
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/sigtrap.go:38 +0x1a

goroutine 47 gp=0xc000822a80 m=nil [chan receive]:
runtime.gopark(0x7f4d984a97a5?, 0x7f4d9e1c26e0?, 0x1?, 0x0?, 0xc0000d4da8?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000d4d18 sp=0xc0000d4cf8 pc=0x7f4d97eaa84e
runtime.chanrecv(0xc00051f2c0, 0xc0000d4f70, 0x1)
        /usr/local/go/src/runtime/chan.go:583 +0x3bf fp=0xc0000d4d90 sp=0xc0000d4d18 pc=0x7f4d97e733ff
runtime.chanrecv2(0xc00051f2c0?, 0xc000696780?)
        /usr/local/go/src/runtime/chan.go:447 +0x12 fp=0xc0000d4db8 sp=0xc0000d4d90 pc=0x7f4d97e73032
github.com/caddyserver/caddy/v2.trapSignalsPosix.func1()
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/sigtrap_posix.go:35 +0x105 fp=0xc0000d4fe0 sp=0xc0000d4db8 pc=0x7f4d984ca585
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000d4fe8 sp=0xc0000d4fe0 pc=0x7f4d97ee2d81
created by github.com/caddyserver/caddy/v2.trapSignalsPosix in goroutine 1
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/sigtrap_posix.go:31 +0x1a

goroutine 68 gp=0xc000682700 m=nil [select, locked to thread]:
runtime.gopark(0xc000691fa8?, 0x2?, 0x60?, 0x1e?, 0xc000691f94?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000691e38 sp=0xc000691e18 pc=0x7f4d97eaa84e
runtime.selectgo(0xc000691fa8, 0xc000691f90, 0x0?, 0x0, 0x0?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc000691f58 sp=0xc000691e38 pc=0x7f4d97ebc725
runtime.ensureSigM.func1()
        /usr/local/go/src/runtime/signal_unix.go:1034 +0x1a5 fp=0xc000691fe0 sp=0xc000691f58 pc=0x7f4d97ed96c5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000691fe8 sp=0xc000691fe0 pc=0x7f4d97ee2d81
created by runtime.ensureSigM in goroutine 46
        /usr/local/go/src/runtime/signal_unix.go:1017 +0xc8

goroutine 69 gp=0xc0006828c0 m=12 mp=0xc000580808 [syscall]:
runtime.notetsleepg(0x7f4d9f2e97c0, 0xffffffffffffffff)
        /usr/local/go/src/runtime/lock_futex.go:246 +0x29 fp=0xc0006927a0 sp=0xc000692778 pc=0x7f4d97e79489
os/signal.signal_recv()
        /usr/local/go/src/runtime/sigqueue.go:152 +0x29 fp=0xc0006927c0 sp=0xc0006927a0 pc=0x7f4d97edefe9
os/signal.loop()
        /usr/local/go/src/os/signal/signal_unix.go:23 +0x13 fp=0xc0006927e0 sp=0xc0006927c0 pc=0x7f4d984a9f53
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0006927e8 sp=0xc0006927e0 pc=0x7f4d97ee2d81
created by os/signal.Notify.func1.1 in goroutine 46
        /usr/local/go/src/os/signal/signal.go:151 +0x1f

goroutine 84 gp=0xc000822fc0 m=nil [IO wait]:
runtime.gopark(0x7f4d97e88096?, 0x7f4d9f2ea7e0?, 0xe8?, 0x7b?, 0x7f4d97e7ac25?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000d7b50 sp=0xc0000d7b30 pc=0x7f4d97eaa84e
runtime.netpollblock(0x7f4d97e7aabb?, 0x97e704c6?, 0x4d?)
        /usr/local/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc0000d7b88 sp=0xc0000d7b50 pc=0x7f4d97ea2937
internal/poll.runtime_pollWait(0x7f4d508e26d0, 0x72)
        /usr/local/go/src/runtime/netpoll.go:345 +0x85 fp=0xc0000d7ba8 sp=0xc0000d7b88 pc=0x7f4d97edcda5
internal/poll.(*pollDesc).wait(0x3?, 0xc000429f80?, 0x0)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0000d7bd0 sp=0xc0000d7ba8 pc=0x7f4d97f964e7
internal/poll.(*pollDesc).waitRead(...)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc000455e80)
        /usr/local/go/src/internal/poll/fd_unix.go:611 +0x2ac fp=0xc0000d7c78 sp=0xc0000d7bd0 pc=0x7f4d97f9b88c
net.(*netFD).accept(0xc000455e80)
        /usr/local/go/src/net/fd_unix.go:172 +0x29 fp=0xc0000d7d30 sp=0xc0000d7c78 pc=0x7f4d98046f89
net.(*TCPListener).accept(0xc000494da0)
        /usr/local/go/src/net/tcpsock_posix.go:159 +0x1e fp=0xc0000d7d58 sp=0xc0000d7d30 pc=0x7f4d980612de
net.(*TCPListener).Accept(0xc000494da0)
        /usr/local/go/src/net/tcpsock.go:327 +0x30 fp=0xc0000d7d88 sp=0xc0000d7d58 pc=0x7f4d980604d0
github.com/caddyserver/caddy/v2.(*deleteListener).Accept(0x7f4d9f1510e0?)
        <autogenerated>:1 +0x24 fp=0xc0000d7da0 sp=0xc0000d7d88 pc=0x7f4d984cf644
net/http.(*onceCloseListener).Accept(0x7f4d9e410628?)
        <autogenerated>:1 +0x24 fp=0xc0000d7db8 sp=0xc0000d7da0 pc=0x7f4d981c91a4
net/http.(*Server).Serve(0xc000409680, {0x7f4d501b5b98, 0xc000494dc0})
        /usr/local/go/src/net/http/server.go:3255 +0x33e fp=0xc0000d7ee8 sp=0xc0000d7db8 pc=0x7f4d981a55be
github.com/caddyserver/caddy/v2.replaceLocalAdminServer.func2()
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/admin.go:449 +0x133 fp=0xc0000d7fe0 sp=0xc0000d7ee8 pc=0x7f4d984ac8d3
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000d7fe8 sp=0xc0000d7fe0 pc=0x7f4d97ee2d81
created by github.com/caddyserver/caddy/v2.replaceLocalAdminServer in goroutine 1
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/admin.go:445 +0x4b2

goroutine 85 gp=0xc000823180 m=nil [select]:
runtime.gopark(0xc0006c6f40?, 0x3?, 0x0?, 0x0?, 0xc0006c6e3a?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0006c6ce0 sp=0xc0006c6cc0 pc=0x7f4d97eaa84e
runtime.selectgo(0xc0006c6f40, 0xc0006c6e34, 0x1?, 0x0, 0x0?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc0006c6e00 sp=0xc0006c6ce0 pc=0x7f4d97ebc725
github.com/caddyserver/certmagic.(*Cache).maintainAssets(0xc000224280, 0x0)
        /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/maintain.go:69 +0x31f fp=0xc0006c6fc0 sp=0xc0006c6e00 pc=0x7f4d9832233f
github.com/caddyserver/certmagic.NewCache.gowrap1()
        /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/cache.go:127 +0x25 fp=0xc0006c6fe0 sp=0xc0006c6fc0 pc=0x7f4d982ffda5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0006c6fe8 sp=0xc0006c6fe0 pc=0x7f4d97ee2d81
created by github.com/caddyserver/certmagic.NewCache in goroutine 1
        /go/pkg/mod/github.com/caddyserver/certmagic@v0.20.0/cache.go:127 +0x1f6

goroutine 87 gp=0xc000823a40 m=nil [select]:
runtime.gopark(0xc0006c7fa0?, 0x2?, 0x8?, 0x10?, 0xc0006c7f8c?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0006c7e30 sp=0xc0006c7e10 pc=0x7f4d97eaa84e
runtime.selectgo(0xc0006c7fa0, 0xc0006c7f88, 0x7f4d97e722f7?, 0x0, 0x0?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc0006c7f50 sp=0xc0006c7e30 pc=0x7f4d97ebc725
github.com/caddyserver/caddy/v2/modules/caddyhttp/reverseproxy.(*metricsUpstreamsHealthyUpdater).Init.func1()
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/reverseproxy/metrics.go:61 +0xca fp=0xc0006c7fe0 sp=0xc0006c7f50 pc=0x7f4d98f5972a
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0006c7fe8 sp=0xc0006c7fe0 pc=0x7f4d97ee2d81
created by github.com/caddyserver/caddy/v2/modules/caddyhttp/reverseproxy.(*metricsUpstreamsHealthyUpdater).Init in goroutine 1
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/reverseproxy/metrics.go:48 +0x4f

goroutine 88 gp=0xc000823c00 m=nil [select]:
runtime.gopark(0xc0006ddfa8?, 0x2?, 0x0?, 0x0?, 0xc0006ddf9c?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000a3ee40 sp=0xc000a3ee20 pc=0x7f4d97eaa84e
runtime.selectgo(0xc000a3efa8, 0xc0006ddf98, 0x0?, 0x0, 0x0?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc000a3ef60 sp=0xc000a3ee40 pc=0x7f4d97ebc725
github.com/caddyserver/caddy/v2/modules/caddytls.(*TLS).keepStorageClean.func1()
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddytls/tls.go:540 +0x93 fp=0xc000a3efe0 sp=0xc000a3ef60 pc=0x7f4d98b02c93
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000a3efe8 sp=0xc000a3efe0 pc=0x7f4d97ee2d81
created by github.com/caddyserver/caddy/v2/modules/caddytls.(*TLS).keepStorageClean in goroutine 1
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddytls/tls.go:532 +0xe5

goroutine 32 gp=0xc000a00700 m=nil [select]:
runtime.gopark(0xc00011ff70?, 0x2?, 0xa0?, 0xfe?, 0xc00011ff64?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00011fe08 sp=0xc00011fde8 pc=0x7f4d97eaa84e
runtime.selectgo(0xc00011ff70, 0xc00011ff60, 0x0?, 0x0, 0xc00011ff10?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc00011ff28 sp=0xc00011fe08 pc=0x7f4d97ebc725
github.com/caddyserver/caddy/v2/modules/caddypki.(*PKI).maintenance(0xc00068f740)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddypki/maintain.go:38 +0xef fp=0xc00011ffc8 sp=0xc00011ff28 pc=0x7f4d98aee08f
github.com/caddyserver/caddy/v2/modules/caddypki.(*PKI).Start.gowrap1()
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddypki/pki.go:113 +0x25 fp=0xc00011ffe0 sp=0xc00011ffc8 pc=0x7f4d98aef745
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00011ffe8 sp=0xc00011ffe0 pc=0x7f4d97ee2d81
created by github.com/caddyserver/caddy/v2/modules/caddypki.(*PKI).Start in goroutine 1
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddypki/pki.go:113 +0x47f

goroutine 33 gp=0xc000a00a80 m=nil [IO wait]:
runtime.gopark(0xc00050e200?, 0xc0000d29c8?, 0xd8?, 0x3d?, 0x7f4d98293805?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000d2978 sp=0xc0000d2958 pc=0x7f4d97eaa84e
runtime.netpollblock(0xc0009800c4?, 0x9e3fad28?, 0x4d?)
        /usr/local/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc0000d29b0 sp=0xc0000d2978 pc=0x7f4d97ea2937
internal/poll.runtime_pollWait(0x7f4d508e24e0, 0x72)
        /usr/local/go/src/runtime/netpoll.go:345 +0x85 fp=0xc0000d29d0 sp=0xc0000d29b0 pc=0x7f4d97edcda5
internal/poll.(*pollDesc).wait(0xc0003a8680?, 0x0?, 0x0)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0000d29f8 sp=0xc0000d29d0 pc=0x7f4d97f964e7
internal/poll.(*pollDesc).waitRead(...)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).RawRead(0xc0003a8680, 0xc0008fe000)
        /usr/local/go/src/internal/poll/fd_unix.go:708 +0x12a fp=0xc0000d2a58 sp=0xc0000d29f8 pc=0x7f4d97f9c50a
net.(*rawConn).Read(0xc00040eec8, 0xc00050e200?)
        /usr/local/go/src/net/rawconn.go:44 +0x36 fp=0xc0000d2a90 sp=0xc0000d2a58 pc=0x7f4d9805c356
golang.org/x/net/internal/socket.(*syscaller).recvmmsg(0xc0004420d8, {0x7f4d9e40b500?, 0xc00040eec8?}, {0xc00050e200?, 0x7f4d9e3f39f8?, 0x0?}, 0x7f4d97e88096?)
        /go/pkg/mod/golang.org/x/net@v0.22.0/internal/socket/mmsghdr_unix.go:120 +0x70 fp=0xc0000d2ac0 sp=0xc0000d2a90 pc=0x7f4d98293c30
golang.org/x/net/internal/socket.(*Conn).recvMsgs(0xc00052aae0, {0xc0003c5808, 0x8, 0x8}, 0x0)
        /go/pkg/mod/golang.org/x/net@v0.22.0/internal/socket/rawconn_mmsg.go:24 +0x176 fp=0xc0000d2b80 sp=0xc0000d2ac0 pc=0x7f4d98294e16
golang.org/x/net/internal/socket.(*Conn).RecvMsgs(...)
        /go/pkg/mod/golang.org/x/net@v0.22.0/internal/socket/socket.go:267
golang.org/x/net/ipv4.(*payloadHandler).ReadBatch(0xc0009744c0, {0xc0003c5808?, 0x0?, 0x2?}, 0x0?)
        /go/pkg/mod/golang.org/x/net@v0.22.0/ipv4/batch.go:80 +0x57 fp=0xc0000d2bf0 sp=0xc0000d2b80 pc=0x7f4d9829c557
golang.org/x/net/ipv4.(*PacketConn).ReadBatch(0x7f4d9f255040?, {0xc0003c5808?, 0x0?, 0x0?}, 0x0?)
        <autogenerated>:1 +0x27 fp=0xc0000d2c28 sp=0xc0000d2bf0 pc=0x7f4d982a2087
github.com/quic-go/quic-go.(*oobConn).ReadPacket(0xc000826120)
        /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/sys_conn_oob.go:174 +0x6e2 fp=0xc0000d2dd0 sp=0xc0000d2c28 pc=0x7f4d98471742
github.com/quic-go/quic-go.(*Transport).listen(0xc0004093b0, {0x7f4d9e416d90, 0xc000826120})
        /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/transport.go:337 +0x149 fp=0xc0000d2fb8 sp=0xc0000d2dd0 pc=0x7f4d98473449
github.com/quic-go/quic-go.(*Transport).init.func1.gowrap1()
        /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/transport.go:241 +0x28 fp=0xc0000d2fe0 sp=0xc0000d2fb8 pc=0x7f4d98472cc8
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000d2fe8 sp=0xc0000d2fe0 pc=0x7f4d97ee2d81
created by github.com/quic-go/quic-go.(*Transport).init.func1 in goroutine 1
        /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/transport.go:241 +0x43b

goroutine 98 gp=0xc000a00c40 m=nil [select]:
runtime.gopark(0xc00011c6d0?, 0x3?, 0x0?, 0x0?, 0xc00011c672?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc00011c4d0 sp=0xc00011c4b0 pc=0x7f4d97eaa84e
runtime.selectgo(0xc00011c6d0, 0xc00011c66c, 0x0?, 0x0, 0x0?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc00011c5f0 sp=0xc00011c4d0 pc=0x7f4d97ebc725
github.com/quic-go/quic-go.(*Transport).runSendQueue(0xc0004093b0)
        /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/transport.go:266 +0x107 fp=0xc00011c7c8 sp=0xc00011c5f0 pc=0x7f4d98472e07
github.com/quic-go/quic-go.(*Transport).init.func1.gowrap2()
        /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/transport.go:242 +0x25 fp=0xc00011c7e0 sp=0xc00011c7c8 pc=0x7f4d98472c65
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00011c7e8 sp=0xc00011c7e0 pc=0x7f4d97ee2d81
created by github.com/quic-go/quic-go.(*Transport).init.func1 in goroutine 1
        /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/transport.go:242 +0x478

goroutine 99 gp=0xc000a00fc0 m=nil [select]:
runtime.gopark(0xc00011cf90?, 0x2?, 0x0?, 0x0?, 0xc00011ce9c?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000d3cf8 sp=0xc0000d3cd8 pc=0x7f4d97eaa84e
runtime.selectgo(0xc0000d3f90, 0xc00011ce98, 0x0?, 0x0, 0x0?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc0000d3e18 sp=0xc0000d3cf8 pc=0x7f4d97ebc725
github.com/quic-go/quic-go.(*baseServer).run(0xc0001c3e60)
        /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/server.go:286 +0x129 fp=0xc0000d3fc8 sp=0xc0000d3e18 pc=0x7f4d98466249
github.com/quic-go/quic-go.newServer.gowrap1()
        /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/server.go:272 +0x25 fp=0xc0000d3fe0 sp=0xc0000d3fc8 pc=0x7f4d984660e5
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000d3fe8 sp=0xc0000d3fe0 pc=0x7f4d97ee2d81
created by github.com/quic-go/quic-go.newServer in goroutine 1
        /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/server.go:272 +0x496

goroutine 100 gp=0xc000a01180 m=nil [select]:
runtime.gopark(0xc0006cae78?, 0x5?, 0x0?, 0x0?, 0xc0006cad86?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0006cabd8 sp=0xc0006cabb8 pc=0x7f4d97eaa84e
runtime.selectgo(0xc0006cae78, 0xc0006cad7c, 0x0?, 0x0, 0x0?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc0006cacf8 sp=0xc0006cabd8 pc=0x7f4d97ebc725
github.com/quic-go/quic-go.(*baseServer).runSendQueue(0xc0001c3e60)
        /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/server.go:299 +0x1bf fp=0xc0006cafc8 sp=0xc0006cacf8 pc=0x7f4d9846653f
github.com/quic-go/quic-go.newServer.gowrap2()
        /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/server.go:273 +0x25 fp=0xc0006cafe0 sp=0xc0006cafc8 pc=0x7f4d98466085
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0006cafe8 sp=0xc0006cafe0 pc=0x7f4d97ee2d81
created by github.com/quic-go/quic-go.newServer in goroutine 1
        /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/server.go:273 +0x4d6

goroutine 101 gp=0xc000a01340 m=nil [select]:
runtime.gopark(0xc0000d8ea8?, 0x3?, 0x60?, 0x0?, 0xc0000d8e92?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0000d8d38 sp=0xc0000d8d18 pc=0x7f4d97eaa84e
runtime.selectgo(0xc0000d8ea8, 0xc0000d8e8c, 0xc0000d8e80?, 0x0, 0xc00083a1c8?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc0000d8e58 sp=0xc0000d8d38 pc=0x7f4d97ebc725
github.com/quic-go/quic-go.(*baseServer).accept(0xc0001c3e60, {0x7f4d9e410778, 0xc000974460})
        /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/server.go:321 +0x9d fp=0xc0000d8ee8 sp=0xc0000d8e58 pc=0x7f4d984667fd
github.com/quic-go/quic-go.(*EarlyListener).Accept(...)
        /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/server.go:160
github.com/caddyserver/caddy/v2.(*fakeCloseQuicListener).Accept(0xc000976360, {0xc0008fe020?, 0x0?})
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/listeners.go:677 +0x31 fp=0xc0000d8f30 sp=0xc0000d8ee8 pc=0x7f4d984c29b1
github.com/quic-go/quic-go/http3.(*Server).ServeListener(0xc00083a160, {0x7f4d9e40b350, 0xc000976360})
        /go/pkg/mod/github.com/quic-go/quic-go@v0.41.0/http3/server.go:283 +0x11b fp=0xc0000d8fb8 sp=0xc0000d8f30 pc=0x7f4d9848d0db
github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Server).serveHTTP3.gowrap1()
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/server.go:576 +0x28 fp=0xc0000d8fe0 sp=0xc0000d8fb8 pc=0x7f4d98ceb668
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000d8fe8 sp=0xc0000d8fe0 pc=0x7f4d97ee2d81
created by github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Server).serveHTTP3 in goroutine 1
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/server.go:576 +0x339

goroutine 102 gp=0xc000a01500 m=nil [IO wait]:
runtime.gopark(0xc00006ef01?, 0xc000a42b28?, 0x55?, 0x12?, 0xc000a42a88?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000a42a50 sp=0xc000a42a30 pc=0x7f4d97eaa84e
runtime.netpollblock(0x0?, 0x97e704c6?, 0x4d?)
        /usr/local/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc000a42a88 sp=0xc000a42a50 pc=0x7f4d97ea2937
internal/poll.runtime_pollWait(0x7f4d508e25d8, 0x72)
        /usr/local/go/src/runtime/netpoll.go:345 +0x85 fp=0xc000a42aa8 sp=0xc000a42a88 pc=0x7f4d97edcda5
internal/poll.(*pollDesc).wait(0x7?, 0x0?, 0x0)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc000a42ad0 sp=0xc000a42aa8 pc=0x7f4d97f964e7
internal/poll.(*pollDesc).waitRead(...)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc0003a8600)
        /usr/local/go/src/internal/poll/fd_unix.go:611 +0x2ac fp=0xc000a42b78 sp=0xc000a42ad0 pc=0x7f4d97f9b88c
net.(*netFD).accept(0xc0003a8600)
        /usr/local/go/src/net/fd_unix.go:172 +0x29 fp=0xc000a42c30 sp=0xc000a42b78 pc=0x7f4d98046f89
net.(*TCPListener).accept(0xc00052aa40)
        /usr/local/go/src/net/tcpsock_posix.go:159 +0x1e fp=0xc000a42c58 sp=0xc000a42c30 pc=0x7f4d980612de
net.(*TCPListener).Accept(0xc00052aa40)
        /usr/local/go/src/net/tcpsock.go:327 +0x30 fp=0xc000a42c88 sp=0xc000a42c58 pc=0x7f4d980604d0
github.com/caddyserver/caddy/v2.(*deleteListener).Accept(0xc0003624dd?)
        <autogenerated>:1 +0x24 fp=0xc000a42ca0 sp=0xc000a42c88 pc=0x7f4d984cf644
crypto/tls.(*listener).Accept(0xc0008d2180)
        /usr/local/go/src/crypto/tls/tls.go:66 +0x27 fp=0xc000a42ce0 sp=0xc000a42ca0 pc=0x7f4d980f7327
github.com/caddyserver/caddy/v2/modules/caddyhttp.(*http2Listener).Accept(0xc000976390)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/http2listener.go:35 +0x5e fp=0xc000a42e70 sp=0xc000a42ce0 pc=0x7f4d98cd433e
net/http.(*onceCloseListener).Accept(0xc000442900?)
        <autogenerated>:1 +0x24 fp=0xc000a42e88 sp=0xc000a42e70 pc=0x7f4d981c91a4
net/http.(*Server).Serve(0xc0004091d0, {0x7f4d9e40c3a0, 0xc000976390})
        /usr/local/go/src/net/http/server.go:3255 +0x33e fp=0xc000a42fb8 sp=0xc000a42e88 pc=0x7f4d981a55be
github.com/caddyserver/caddy/v2/modules/caddyhttp.(*App).Start.gowrap1()
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/app.go:509 +0x28 fp=0xc000a42fe0 sp=0xc000a42fb8 pc=0x7f4d98cc6a48
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000a42fe8 sp=0xc000a42fe0 pc=0x7f4d97ee2d81
created by github.com/caddyserver/caddy/v2/modules/caddyhttp.(*App).Start in goroutine 1
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/app.go:509 +0x1b95

goroutine 103 gp=0xc000a016c0 m=nil [IO wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000a43c20 sp=0xc000a43c00 pc=0x7f4d97eaa84e
runtime.netpollblock(0x0?, 0x97e704c6?, 0x4d?)
        /usr/local/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc000a43c58 sp=0xc000a43c20 pc=0x7f4d97ea2937
internal/poll.runtime_pollWait(0x7f4d508e23e8, 0x72)
        /usr/local/go/src/runtime/netpoll.go:345 +0x85 fp=0xc000a43c78 sp=0xc000a43c58 pc=0x7f4d97edcda5
internal/poll.(*pollDesc).wait(0x9?, 0xc000a39f80?, 0x0)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc000a43ca0 sp=0xc000a43c78 pc=0x7f4d97f964e7
internal/poll.(*pollDesc).waitRead(...)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc0003a8780)
        /usr/local/go/src/internal/poll/fd_unix.go:611 +0x2ac fp=0xc000a43d48 sp=0xc000a43ca0 pc=0x7f4d97f9b88c
net.(*netFD).accept(0xc0003a8780)
        /usr/local/go/src/net/fd_unix.go:172 +0x29 fp=0xc000a43e00 sp=0xc000a43d48 pc=0x7f4d98046f89
net.(*TCPListener).accept(0xc00052abe0)
        /usr/local/go/src/net/tcpsock_posix.go:159 +0x1e fp=0xc000a43e28 sp=0xc000a43e00 pc=0x7f4d980612de
net.(*TCPListener).Accept(0xc00052abe0)
        /usr/local/go/src/net/tcpsock.go:327 +0x30 fp=0xc000a43e58 sp=0xc000a43e28 pc=0x7f4d980604d0
github.com/caddyserver/caddy/v2.(*deleteListener).Accept(0x7f4d9f1510e0?)
        <autogenerated>:1 +0x24 fp=0xc000a43e70 sp=0xc000a43e58 pc=0x7f4d984cf644
net/http.(*onceCloseListener).Accept(0x7f4d9e410628?)
        <autogenerated>:1 +0x24 fp=0xc000a43e88 sp=0xc000a43e70 pc=0x7f4d981c91a4
net/http.(*Server).Serve(0xc0004094a0, {0x7f4d501b5b98, 0xc00052ac20})
        /usr/local/go/src/net/http/server.go:3255 +0x33e fp=0xc000a43fb8 sp=0xc000a43e88 pc=0x7f4d981a55be
github.com/caddyserver/caddy/v2/modules/caddyhttp.(*App).Start.gowrap1()
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/app.go:509 +0x28 fp=0xc000a43fe0 sp=0xc000a43fb8 pc=0x7f4d98cc6a48
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000a43fe8 sp=0xc000a43fe0 pc=0x7f4d97ee2d81
created by github.com/caddyserver/caddy/v2/modules/caddyhttp.(*App).Start in goroutine 1
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/app.go:509 +0x1b95

goroutine 54 gp=0xc000a01a40 m=nil [IO wait]:
runtime.gopark(0xcdb4da524ae8b237?, 0x50263a591be2d575?, 0x37?, 0xb2?, 0xb?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0006c4818 sp=0xc0006c47f8 pc=0x7f4d97eaa84e
runtime.netpollblock(0x7f4d97f07378?, 0x97e704c6?, 0x4d?)
        /usr/local/go/src/runtime/netpoll.go:573 +0xf7 fp=0xc0006c4850 sp=0xc0006c4818 pc=0x7f4d97ea2937
internal/poll.runtime_pollWait(0x7f4d508e22f0, 0x72)
        /usr/local/go/src/runtime/netpoll.go:345 +0x85 fp=0xc0006c4870 sp=0xc0006c4850 pc=0x7f4d97edcda5
internal/poll.(*pollDesc).wait(0xc00036a400?, 0xc0001ce900?, 0x0)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0006c4898 sp=0xc0006c4870 pc=0x7f4d97f964e7
internal/poll.(*pollDesc).waitRead(...)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc00036a400, {0xc0001ce900, 0x900, 0x900})
        /usr/local/go/src/internal/poll/fd_unix.go:164 +0x27a fp=0xc0006c4930 sp=0xc0006c4898 pc=0x7f4d97f977da
net.(*netFD).Read(0xc00036a400, {0xc0001ce900?, 0x7f4d508f2958?, 0xc000153938?})
        /usr/local/go/src/net/fd_posix.go:55 +0x25 fp=0xc0006c4978 sp=0xc0006c4930 pc=0x7f4d98044fa5
net.(*conn).Read(0xc0000c6960, {0xc0001ce900?, 0xc0006c4a58?, 0x7f4d97e7aabb?})
        /usr/local/go/src/net/net.go:179 +0x45 fp=0xc0006c49c0 sp=0xc0006c4978 pc=0x7f4d98058465
net.(*TCPConn).Read(0xc000362540?, {0xc0001ce900?, 0x7f4d97e7c7db?, 0x6c4b30?})
        <autogenerated>:1 +0x25 fp=0xc0006c49f0 sp=0xc0006c49c0 pc=0x7f4d980697e5
crypto/tls.(*atLeastReader).Read(0xc000153938, {0xc0001ce900?, 0x0?, 0xc000153938?})
        /usr/local/go/src/crypto/tls/conn.go:806 +0x3b fp=0xc0006c4a38 sp=0xc0006c49f0 pc=0x7f4d980bbefb
bytes.(*Buffer).ReadFrom(0xc000362630, {0x7f4d9e3fbca0, 0xc000153938})
        /usr/local/go/src/bytes/buffer.go:211 +0x98 fp=0xc0006c4a90 sp=0xc0006c4a38 pc=0x7f4d97ef94f8
crypto/tls.(*Conn).readFromUntil(0xc000362388, {0x7f4d9e3fc0e0, 0xc0000c6960}, 0xc0006c4aa0?)
        /usr/local/go/src/crypto/tls/conn.go:828 +0xde fp=0xc0006c4ac8 sp=0xc0006c4a90 pc=0x7f4d980bc0de
crypto/tls.(*Conn).readRecordOrCCS(0xc000362388, 0x0)
        /usr/local/go/src/crypto/tls/conn.go:626 +0x3cf fp=0xc0006c4d48 sp=0xc0006c4ac8 pc=0x7f4d980b91ef
crypto/tls.(*Conn).readRecord(...)
        /usr/local/go/src/crypto/tls/conn.go:588
crypto/tls.(*Conn).Read(0xc000362388, {0xc0005fa9e0, 0x9, 0x7f4d97ebcc26?})
        /usr/local/go/src/crypto/tls/conn.go:1370 +0x156 fp=0xc0006c4db8 sp=0xc0006c4d48 pc=0x7f4d980bfa96
io.ReadAtLeast({0x7f4d504aed00, 0xc000362388}, {0xc0005fa9e0, 0x9, 0x9}, 0x9)
        /usr/local/go/src/io/io.go:335 +0x90 fp=0xc0006c4e00 sp=0xc0006c4db8 pc=0x7f4d97ef0930
io.ReadFull(...)
        /usr/local/go/src/io/io.go:354
golang.org/x/net/http2.readFrameHeader({0xc0005fa9e0, 0x9, 0x0?}, {0x7f4d504aed00?, 0xc000362388?})
        /go/pkg/mod/golang.org/x/net@v0.22.0/http2/frame.go:237 +0x65 fp=0xc0006c4e50 sp=0xc0006c4e00 pc=0x7f4d985b06c5
golang.org/x/net/http2.(*Framer).ReadFrame(0xc0005fa9a0)
        /go/pkg/mod/golang.org/x/net@v0.22.0/http2/frame.go:498 +0x85 fp=0xc0006c4ef8 sp=0xc0006c4e50 pc=0x7f4d985b0e05
golang.org/x/net/http2.(*serverConn).readFrames(0xc00019fba0)
        /go/pkg/mod/golang.org/x/net@v0.22.0/http2/server.go:820 +0x87 fp=0xc0006c4fc8 sp=0xc0006c4ef8 pc=0x7f4d985bbac7
golang.org/x/net/http2.(*serverConn).serve.gowrap7()
        /go/pkg/mod/golang.org/x/net@v0.22.0/http2/server.go:932 +0x25 fp=0xc0006c4fe0 sp=0xc0006c4fc8 pc=0x7f4d985bd125
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0006c4fe8 sp=0xc0006c4fe0 pc=0x7f4d97ee2d81
created by golang.org/x/net/http2.(*serverConn).serve in goroutine 80
        /go/pkg/mod/golang.org/x/net@v0.22.0/http2/server.go:932 +0x56a

goroutine 80 gp=0xc000822540 m=nil [select]:
runtime.gopark(0xc0006257c8?, 0x5?, 0xf5?, 0x29?, 0xc00062543e?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc0006252d0 sp=0xc0006252b0 pc=0x7f4d97eaa84e
runtime.selectgo(0xc0006257c8, 0xc000625434, 0xc000d009c0?, 0x0, 0x0?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc0006253f0 sp=0xc0006252d0 pc=0x7f4d97ebc725
golang.org/x/net/http2.(*serverConn).serve(0xc00019fba0)
        /go/pkg/mod/golang.org/x/net@v0.22.0/http2/server.go:940 +0x897 fp=0xc000625868 sp=0xc0006253f0 pc=0x7f4d985bc937
golang.org/x/net/http2.(*Server).ServeConn(0xc000974370, {0x7f4d9e41aea0, 0xc000362388}, 0xc000625b30)
        /go/pkg/mod/golang.org/x/net@v0.22.0/http2/server.go:531 +0xbad fp=0xc000625b08 sp=0xc000625868 pc=0x7f4d985bab8d
golang.org/x/net/http2.ConfigureServer.func1(0xc0004091d0, 0xc000362388, {0x7f4d9e3fc740, 0xc00052a840})
        /go/pkg/mod/golang.org/x/net@v0.22.0/http2/server.go:321 +0x125 fp=0xc000625b90 sp=0xc000625b08 pc=0x7f4d985b9f25
net/http.(*conn).serve(0xc000442900, {0x7f4d9e410740, 0xc000b85aa0})
        /usr/local/go/src/net/http/server.go:1947 +0x12f3 fp=0xc000625fb8 sp=0xc000625b90 pc=0x7f4d981a1173
net/http.(*Server).Serve.gowrap3()
        /usr/local/go/src/net/http/server.go:3285 +0x28 fp=0xc000625fe0 sp=0xc000625fb8 pc=0x7f4d981a59a8
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000625fe8 sp=0xc000625fe0 pc=0x7f4d97ee2d81
created by net/http.(*Server).Serve in goroutine 102
        /usr/local/go/src/net/http/server.go:3285 +0x4b4

goroutine 114 gp=0xc0009896c0 m=18 mp=0xc00052d808 [syscall, locked to thread]:
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0006c9fe8 sp=0xc0006c9fe0 pc=0x7f4d97ee2d81

goroutine 131 gp=0xc000989c00 m=19 mp=0xc000600808 [syscall, locked to thread]:
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0006c5fe8 sp=0xc0006c5fe0 pc=0x7f4d97ee2d81

goroutine 132 gp=0xc000bb28c0 m=20 mp=0xc000680808 [syscall, locked to thread]:
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000ab3fe8 sp=0xc000ab3fe0 pc=0x7f4d97ee2d81

goroutine 181 gp=0xc000683340 m=23 mp=0xc0009f0008 [syscall, locked to thread]:
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000bb9fe8 sp=0xc000bb9fe0 pc=0x7f4d97ee2d81

goroutine 384 gp=0xc000a92540 m=nil [chan receive]:
runtime.gopark(0x2?, 0xc0008344e0?, 0x51?, 0x0?, 0xc0006202e0?)
        /usr/local/go/src/runtime/proc.go:402 +0xce fp=0xc000620290 sp=0xc000620270 pc=0x7f4d97eaa84e
runtime.chanrecv(0xc000834480, 0x0, 0x1)
        /usr/local/go/src/runtime/chan.go:583 +0x3bf fp=0xc000620308 sp=0xc000620290 pc=0x7f4d97e733ff
runtime.chanrecv1(0xc000620390?, 0xc000620368?)
        /usr/local/go/src/runtime/chan.go:442 +0x12 fp=0xc000620330 sp=0xc000620308 pc=0x7f4d97e73012
github.com/dunglas/frankenphp.ServeHTTP({0x7f4d9e40c310, 0xc0004f8870}, 0xc000d06b40)
        /go/src/app/frankenphp.go:465 +0x20e fp=0xc0006203f0 sp=0xc000620330 pc=0x7f4d990bacae
github.com/dunglas/frankenphp/caddy.FrankenPHPModule.ServeHTTP({{0x7f4d9ac2dfcd, 0x10}, {0xc0003733f0, 0x1, 0x1}, 0xc000125534, 0x0, 0xc00050d780}, {0x7f4d9e40c310, 0xc0004f8870}, ...)
        /go/src/app/caddy/caddy.go:275 +0x4ac fp=0xc000620790 sp=0xc0006203f0 pc=0x7f4d991c110c
github.com/dunglas/frankenphp/caddy.(*FrankenPHPModule).ServeHTTP(0xc000620870?, {0x7f4d9e40c310?, 0xc0004f8870?}, 0x7f4d9e23dce0?, {0x7f4d9e3fe340?, 0xc000053060?})
        <autogenerated>:1 +0xe5 fp=0xc000620848 sp=0xc000620790 pc=0x7f4d991c7265
github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapMiddleware.func1.1({0x7f4d9e40c310?, 0xc0004f8870?}, 0xc000314c40?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:331 +0x3a fp=0xc000620888 sp=0xc000620848 pc=0x7f4d98ce887a
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x7f4d9e3fe340?, {0x7f4d9e40c310?, 0xc0004f8870?}, 0xc000d067e0?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc0006208b0 sp=0xc000620888 pc=0x7f4d98ccd469
github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.Compile.wrapRoute.func1.1({0x7f4d9e40c310, 0xc0004f8870}, 0xc000d067e0)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:300 +0x325 fp=0xc000620928 sp=0xc0006208b0 pc=0x7f4d98ce85e5
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x0?, {0x7f4d9e40c310?, 0xc0004f8870?}, 0xc000768e80?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000620950 sp=0xc000620928 pc=0x7f4d98ccd469
github.com/caddyserver/caddy/v2/modules/caddyhttp/rewrite.Rewrite.ServeHTTP({{0x0, 0x0}, {0xc0001176e0, 0x1d}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0, ...}, ...}, ...)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/rewrite/rewrite.go:137 +0x3f3 fp=0xc000620ad8 sp=0xc000620950 pc=0x7f4d98f1bc53
github.com/caddyserver/caddy/v2/modules/caddyhttp/rewrite.(*Rewrite).ServeHTTP(0xc000620c28?, {0x7f4d9e40c310?, 0xc0004f8870?}, 0x7f4d9e23dce0?, {0x7f4d9e3fe340?, 0xc000053080?})
        <autogenerated>:1 +0xd6 fp=0xc000620c00 sp=0xc000620ad8 pc=0x7f4d98f1dc76
github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapMiddleware.func1.1({0x7f4d9e40c310?, 0xc0004f8870?}, 0xc00050d000?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:331 +0x3a fp=0xc000620c40 sp=0xc000620c00 pc=0x7f4d98ce887a
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x7f4d9e3fe340?, {0x7f4d9e40c310?, 0xc0004f8870?}, 0xc000d067e0?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000620c68 sp=0xc000620c40 pc=0x7f4d98ccd469
github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.Compile.wrapRoute.func1.1({0x7f4d9e40c310, 0xc0004f8870}, 0xc000d067e0)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:300 +0x325 fp=0xc000620ce0 sp=0xc000620c68 pc=0x7f4d98ce85e5
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x7f4d9e410740?, {0x7f4d9e40c310?, 0xc0004f8870?}, 0xe?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000620d08 sp=0xc000620ce0 pc=0x7f4d98ccd469
github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.Compile.wrapRoute.func1.1({0x7f4d9e40c310, 0xc0004f8870}, 0xc000d067e0)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:268 +0x244 fp=0xc000620d80 sp=0xc000620d08 pc=0x7f4d98ce8504
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0xc0009b9750?, {0x7f4d9e40c310?, 0xc0004f8870?}, 0x2?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000620da8 sp=0xc000620d80 pc=0x7f4d98ccd469
github.com/caddyserver/caddy/v2/modules/caddyhttp/encode.(*Encode).ServeHTTP(0xc000314b00, {0x7f4d9e40ce80, 0xc0007b2078}, 0xc000d067e0, {0x7f4d9e3fe340, 0xc0000530c0})
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/encode/encode.go:138 +0x252 fp=0xc000620e10 sp=0xc000620da8 pc=0x7f4d98d313d2
github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapMiddleware.func1.1({0x7f4d9e40ce80?, 0xc0007b2078?}, 0xc00078c120?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:331 +0x3a fp=0xc000620e50 sp=0xc000620e10 pc=0x7f4d98ce887a
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0xc0007c8ee0?, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0x4?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000620e78 sp=0xc000620e50 pc=0x7f4d98ccd469
github.com/caddyserver/caddy/v2/modules/caddyhttp.VarsMiddleware.ServeHTTP(0xc00015eea0, {0x7f4d9e40ce80, 0xc0007b2078}, 0xc000d067e0, {0x7f4d9e3fe340, 0xc000d00d20})
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/vars.go:61 +0x273 fp=0xc000620f50 sp=0xc000620e78 pc=0x7f4d98cf0b13
github.com/caddyserver/caddy/v2/modules/caddyhttp.(*VarsMiddleware).ServeHTTP(0xc0007c8fb8?, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0x7f4d9e23dce0?, {0x7f4d9e3fe340?, 0xc000d00d20?})
        <autogenerated>:1 +0x38 fp=0xc000620f90 sp=0xc000620f50 pc=0x7f4d98cfa238
github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapMiddleware.func1.1({0x7f4d9e40ce80?, 0xc0007b2078?}, 0xc0003a4dd0?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:331 +0x3a fp=0xc000620fd0 sp=0xc000620f90 pc=0x7f4d98ce887a
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x7f4d9e3fe340?, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0xc000d067e0?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000620ff8 sp=0xc000620fd0 pc=0x7f4d98ccd469
github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.Compile.wrapRoute.func1.1({0x7f4d9e40ce80, 0xc0007b2078}, 0xc000d067e0)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:300 +0x325 fp=0xc000621070 sp=0xc000620ff8 pc=0x7f4d98ce85e5
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0xc0001f6a08?, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0x7f4d9e3fe340?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000621098 sp=0xc000621070 pc=0x7f4d98ccd469
github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Subroute).ServeHTTP(0xc000495720, {0x7f4d9e40ce80, 0xc0007b2078}, 0xc000d067e0, {0x7f4d9e3fe340, 0x7f4d9e3f1eb8})
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/subroute.go:74 +0x67 fp=0xc0006210d8 sp=0xc000621098 pc=0x7f4d98cf0747
github.com/caddyserver/caddy/v2/modules/caddyhttp.wrapMiddleware.func1.1({0x7f4d9e40ce80?, 0xc0007b2078?}, 0xc000495720?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:331 +0x3a fp=0xc000621118 sp=0xc0006210d8 pc=0x7f4d98ce887a
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x7f4d9e3fe340?, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0xc000d067e0?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000621140 sp=0xc000621118 pc=0x7f4d98ccd469
github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.Compile.wrapRoute.func1.1({0x7f4d9e40ce80, 0xc0007b2078}, 0xc000d067e0)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:300 +0x325 fp=0xc0006211b8 sp=0xc000621140 pc=0x7f4d98ce85e5
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x7f4d9e410740?, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0xe?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc0006211e0 sp=0xc0006211b8 pc=0x7f4d98ccd469
github.com/caddyserver/caddy/v2/modules/caddyhttp.RouteList.Compile.wrapRoute.func1.1({0x7f4d9e40ce80, 0xc0007b2078}, 0xc000d067e0)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/routes.go:268 +0x244 fp=0xc000621258 sp=0xc0006211e0 pc=0x7f4d98ce8504
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0x7f4d9e3410e0?, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0xc0009b9690?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc000621280 sp=0xc000621258 pc=0x7f4d98ccd469
github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Server).enforcementHandler(0x10?, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0x0?, {0x7f4d9e3fe340?, 0xc000495980?})
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/server.go:429 +0x24b fp=0xc000621358 sp=0xc000621280 pc=0x7f4d98cead6b
github.com/caddyserver/caddy/v2/modules/caddyhttp.(*App).Provision.(*Server).wrapPrimaryRoute.func1({0x7f4d9e40ce80?, 0xc0007b2078?}, 0x7f4d97f157cf?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/server.go:405 +0x35 fp=0xc000621398 sp=0xc000621358 pc=0x7f4d98cc4115
github.com/caddyserver/caddy/v2/modules/caddyhttp.HandlerFunc.ServeHTTP(0xc00022fb20?, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0xc000d067e0?)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/caddyhttp.go:58 +0x29 fp=0xc0006213c0 sp=0xc000621398 pc=0x7f4d98ccd469
github.com/caddyserver/caddy/v2/modules/caddyhttp.(*Server).ServeHTTP(0xc0002662c8, {0x7f4d9e40ce80, 0xc0007b2078}, 0xc000d06480)
        /go/pkg/mod/github.com/caddyserver/caddy/v2@v2.7.6/modules/caddyhttp/server.go:341 +0xc47 fp=0xc000621cf0 sp=0xc0006213c0 pc=0x7f4d98ce9b67
net/http.serverHandler.ServeHTTP({0x80?}, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0xc000152a68?)
        /usr/local/go/src/net/http/server.go:3137 +0x8e fp=0xc000621d20 sp=0xc000621cf0 pc=0x7f4d981a518e
net/http.initALPNRequest.ServeHTTP({{0x7f4d9e410740?, 0xc000b85ad0?}, 0xc000362388?, {0xc0004091d0?}}, {0x7f4d9e40ce80, 0xc0007b2078}, 0xc000d06480)
        /usr/local/go/src/net/http/server.go:3745 +0x231 fp=0xc000621eb0 sp=0xc000621d20 pc=0x7f4d981a8351
net/http.(*initALPNRequest).ServeHTTP(0xc0007db5c0?, {0x7f4d9e40ce80?, 0xc0007b2078?}, 0x0?)
        <autogenerated>:1 +0x4f fp=0xc000621ef8 sp=0xc000621eb0 pc=0x7f4d981c942f
net/http.Handler.ServeHTTP-fm({0x7f4d9e40ce80?, 0xc0007b2078?}, 0xc00019fba0?)
        <autogenerated>:1 +0x3b fp=0xc000621f28 sp=0xc000621ef8 pc=0x7f4d981ca55b
golang.org/x/net/http2.(*serverConn).runHandler(0x7f4d97eb29dd?, 0x0?, 0x0?, 0x0?)
        /go/pkg/mod/golang.org/x/net@v0.22.0/http2/server.go:2368 +0xbb fp=0xc000621fb0 sp=0xc000621f28 pc=0x7f4d985c42bb
golang.org/x/net/http2.(*serverConn).scheduleHandler.gowrap1()
        /go/pkg/mod/golang.org/x/net@v0.22.0/http2/server.go:2303 +0x2c fp=0xc000621fe0 sp=0xc000621fb0 pc=0x7f4d985c3eac
runtime.goexit({})
        /usr/local/go/src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000621fe8 sp=0xc000621fe0 pc=0x7f4d97ee2d81
created by golang.org/x/net/http2.(*serverConn).scheduleHandler in goroutine 80
        /go/pkg/mod/golang.org/x/net@v0.22.0/http2/server.go:2303 +0x21d

rax    0x55
rbx    0x7f4d43bcc710
rcx    0x0
rdx    0xd
rdi    0x7f4d9b247ffc
rsi    0x7f4d4462be70
rbp    0x7f4d9b247ffc
rsp    0x7f4d455ca638
r8     0x51
r9     0x8080808080808001
r10    0x7f4d9b247ffc
r11    0x246
r12    0xe
r13    0x7f4d43bc9120
r14    0xa
r15    0x0
rip    0x7f4d9ac01a1b
rflags 0x10246
cs     0x33
fs     0x0
gs     0x0

@emaia
Copy link

emaia commented Mar 25, 2024

Another update: I'm using Laravel Debugbar in my dev env and if I disable it, this crash doesn't happen.

@jeremyj11
Copy link
Contributor

jeremyj11 commented May 29, 2024 •
edited
Loading

I was gettting the same crash, just updated to v1.1.5 and now just get "Segmentation fault" after a page reload and caddy exits.

Ubuntu 20.04
Not using docker, downloaded frankenphp-linux-x86_64 from github releases
Freshly created Laravel 11 app, migrating code over from Laravel 8 app
Works fine with php-fpm 8.3

2024/05/29 13:41:51.204 DEBUG http.handlers.rewrite rewrote request {"request": {"remote_ip": "127.0.0.1", "remote_port": "41848", "client_ip": "127.0.0.1", "proto": "HTTP/1.1", "method": "GET", "host": "heycast.test", "uri": "/api/schedule/Agent", "headers": {"Accept": ["application/json, text/plain, */*"], "User-Agent": ["Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"], "Sec-Gpc": ["1"], "Accept-Language": ["en-AU,en;q=0.9"], "Origin": ["http://localhost:4200"], "Referer": ["http://localhost:4200/"], "Accept-Encoding": ["gzip, deflate"], "Connection": ["keep-alive"], "Account": ["Agent"], "Authorization": []}}, "method": "GET", "uri": "/index.php"} Segmentation fault

My caddyfile:

{
  debug
  frankenphp
  order php_server before file_server
}

heycast.test:80 {
  root * /home/me/htdocs/mydomain/public
  encode zstd br gzip
  php_server
}

any advice?

@dunglas
Copy link
Owner

dunglas commented May 29, 2024

@jeremyj11 would you be able to provide us a minimal reproducer triggering the crash, or to gather a stack trace by following these steps? https://frankenphp.dev/docs/contributing/#debugging-segmentation-faults-with-static-builds

Thanks!

@theaungmyatmoe
Copy link

I am facing the same issue in my latest docker build.

docker run -it --rm breeze-settings sh
/var/www/html/apps/settings $ php artisan octane:start --server=frankenphp --host=0.0.0.0 --port=80 --watch

   InvalidArgumentException

  Unable to determine admin port. Please specify the [--admin-port] option.

  at vendor/laravel/octane/src/Commands/StartFrankenPhpCommand.php:234
    230▕         $defaultPort = 2019;
    231▕
    232▕         return tap($defaultPort + ($this->getPort() - 8000), function ($adminPort) {
    233▕             if ($adminPort < 0) {
  ➜ 234▕                 throw new InvalidArgumentException(
    235▕                     'Unable to determine admin port. Please specify the [--admin-port] option.',
    236▕                 );
    237▕             }
    238▕         });

      +29 vendor frames

  30  artisan:13
      Illuminate\Foundation\Application::handleCommand(Object(Symfony\Component\Console\Input\ArgvInput))

/var/www/html/apps/settings $ php artisan octane:start --server=frankenphp --admin-port=2019 --host=0.0.0.0 --port=80 --watch

   INFO  Server running…

  Local: http://0.0.0.0:80

  Press Ctrl+C to stop the server


   WARN  server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server

   INFO  SIGSEGV: segmentation violation
/var/www/html/apps/settings $

@theaungmyatmoe
Copy link

@dunglas You can reproduce with this:

ARG PHP_VERSION=8.3
ARG FRANKENPHP_VERSION=latest

FROM dunglas/frankenphp:${FRANKENPHP_VERSION}-php${PHP_VERSION}-alpine

Here is supervisor config.

[program:octane]
process_name=%(program_name)s_%(process_num)02d
command=php %(ENV_APP_PATH)s/artisan octane:start --server=frankenphp --host=0.0.0.0 --port=80 --watch
#command=php %(ENV_APP_PATH)s/artisan octane:start --server=frankenphp --host=localhost --port=443 --admin-port=2019 --https --http-redirect
user=%(ENV_USER)s
autostart=true
autorestart=true
environment=LARAVEL_OCTANE="1"
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

@theaungmyatmoe
Copy link

Here is trace log that might help you to debug.

{"level":"info","ts":1717008503.903582,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"warn","ts":1717008503.904085,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":16}
{"level":"info","ts":1717008503.9044886,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1717008503.9045434,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1717008503.9045532,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"info","ts":1717008503.9046576,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x40001d0c00"}
{"level":"info","ts":1717008503.9051423,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0x40001d0c00"}
Error: loading initial config: loading new config: loading http app module: provision http: loading pki app module: provision pki: provisioning CA 'local': generating root: saving root certificate: mkdir /data/caddy/pki: permission denied
{"level":"info","ts":1717008504.383381,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"warn","ts":1717008504.3852797,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":16}
{"level":"info","ts":1717008504.3882213,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//127.0.0.1:2019","//localhost:2019","//[::1]:2019"]}
{"level":"info","ts":1717008504.3883011,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1717008504.388307,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"info","ts":1717008504.3886786,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x40003f4500"}
{"level":"info","ts":1717008504.3888597,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0x40003f4500"}
Error: loading initial config: loading new config: loading http app module: provision http: loading pki app module: provision pki: provisioning CA 'local': generating root: saving root certificate: mkdir /data/caddy/pki: permission denied
{"level":"info","ts":1717008505.101319,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"warn","ts":1717008505.1025002,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":16}
{"level":"info","ts":1717008505.104571,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1717008505.104849,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1717008505.104858,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"info","ts":1717008505.1066546,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x4000598480"}
{"level":"info","ts":1717008505.1066604,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0x4000598480"}
Error: loading initial config: loading new config: loading http app module: provision http: loading pki app module: provision pki: provisioning CA 'local': generating root: saving root certificate: mkdir /data/caddy/pki: permission denied
{"level":"info","ts":1717008505.8572512,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"warn","ts":1717008505.8585649,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":16}
{"level":"info","ts":1717008505.8616953,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1717008505.8617735,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1717008505.8617826,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"info","ts":1717008505.8622727,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x40005dec00"}
{"level":"info","ts":1717008505.8622785,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0x40005dec00"}
Error: loading initial config: loading new config: loading http app module: provision http: loading pki app module: provision pki: provisioning CA 'local': generating root: saving root certificate: mkdir /data/caddy/pki: permission denied
{"level":"info","ts":1717008506.997365,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"warn","ts":1717008506.9984906,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":16}
{"level":"info","ts":1717008507.0004244,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1717008507.000491,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1717008507.0005004,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"info","ts":1717008507.001825,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x40005f5e80"}
{"level":"info","ts":1717008507.0021057,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0x40005f5e80"}
Error: loading initial config: loading new config: loading http app module: provision http: loading pki app module: provision pki: provisioning CA 'local': generating root: saving root certificate: mkdir /data/caddy/pki: permission denied
{"level":"info","ts":1717008508.938355,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"warn","ts":1717008508.9387708,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":16}
{"level":"info","ts":1717008508.9415684,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"info","ts":1717008508.94166,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1717008508.9416745,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"info","ts":1717008508.9421663,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x40003a8500"}
{"level":"info","ts":1717008508.9421775,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0x40003a8500"}
Error: loading initial config: loading new config: loading http app module: provision http: loading pki app module: provision pki: provisioning CA 'local': generating root: saving root certificate: mkdir /data/caddy/pki: permission denied

@emaia
Copy link

emaia commented May 29, 2024

Another update: I'm using Laravel Debugbar in my dev env and if I disable it, this crash doesn't happen.

This reduced occurrences of this bug but it continues to happen. Changing Laravel APP_DEBUG to false also helps to minimize occurrences.

@dunglas
Copy link
Owner

dunglas commented May 29, 2024

@aungmyatmoethegreat there is a known segfault with Alpine/musl and Octane. Could you try to switch to the Debian variant to see if this fixes the issue please?

@jeremyj11
Copy link
Contributor

@jeremyj11 would you be able to provide us a minimal reproducer triggering the crash, or to gather a stack trace by following these steps? https://frankenphp.dev/docs/contributing/#debugging-segmentation-faults-with-static-builds

Thanks!

Here you go:


#0  get_nominal_size (end=0x7f936a6e020c "\223\177", p=0x7f936a6df6d0 "\300\241y\\\223\177") at src/malloc/mallocng/meta.h:163
#1  __libc_free (p=p@entry=0x7f936a6df6d0) at src/malloc/mallocng/free.c:110
#2  0x00007f93b5ea4654 in __libc_realloc (p=0x7f936a6df6d0, n=256) at src/malloc/mallocng/realloc.c:49
#3  0x00007f93b4cb5b5f in OPENSSL_LH_insert ()
#4  0x00007f93b4cb7607 in namemap_add_name ()
#5  0x00007f93b4cb8498 in get_legacy_cipher_names ()
#6  0x00007f93b4cb5f02 in OPENSSL_LH_doall_arg_thunk ()
#7  0x00007f93b4cd887e in OBJ_NAME_do_all ()
#8  0x00007f93b4cb7daf in ossl_namemap_stored ()
#9  0x00007f93b4c9a65e in evp_generic_do_all ()
#10 0x00007f93b4ca0dca in EVP_KEYMGMT_do_all_provided ()
#11 0x00007f93b4e47cd5 in OSSL_DECODER_CTX_new_for_pkey ()
#12 0x00007f93b4ce00f0 in pem_read_bio_key ()
#13 0x00007f93b4ce0554 in PEM_read_bio_PrivateKey ()
#14 0x00007f93b41ca9fc in php_openssl_pkey_from_zval (val=0x7f935ca180d0, public_key=0, passphrase=0x7f935fc00268 "", passphrase_len=0, arg_num=1)
    at /go/src/app/dist/static-php-cli/source/php-src/ext/openssl/openssl.c:3659
#15 0x00007f93b41ce38a in zif_openssl_pkey_get_private (execute_data=0x7f935ca18080, return_value=0x7f935ca18060) at /go/src/app/dist/static-php-cli/source/php-src/ext/openssl/openssl.c:4889
#16 0x00007f93b3ff5468 in ZEND_DO_ICALL_SPEC_RETVAL_USED_HANDLER () at /go/src/app/dist/static-php-cli/source/php-src/Zend/zend_vm_execute.h:1337
#17 execute_ex (ex=0x7f935ca16020) at /go/src/app/dist/static-php-cli/source/php-src/Zend/zend_vm_execute.h:57216
#18 0x00007f93b40127f0 in zend_execute (op_array=0x7f935ca65000, return_value=0x0) at /go/src/app/dist/static-php-cli/source/php-src/Zend/zend_vm_execute.h:61604
#19 0x00007f93b3e7e83f in zend_execute_scripts (type=8, retval=0x0, file_count=3) at /go/src/app/dist/static-php-cli/source/php-src/Zend/zend.c:1891
#20 0x00007f93b3dbec37 in php_execute_script (primary_file=0x7f935f8763e0) at /go/src/app/dist/static-php-cli/source/php-src/main/main.c:2531
#21 0x00007f93b3d2c0c1 in frankenphp_execute_script (file_name=0x0) at frankenphp.c:851
#22 0x00007f93b3d29717 in _cgo_b7d6fd74a0c8_Cfunc_frankenphp_execute_script (v=0xc0005d2dd8) at /tmp/go-build/cgo-gcc-prolog:55
#23 0x00007f93b29c7d61 in runtime.asmcgocall () at /usr/local/go/src/runtime/asm_amd64.s:918
#24 0x000000c00028f180 in ?? ()
#25 0x00007f935f876550 in ?? ()
#26 0x00007f935f8765b4 in ?? ()
#27 0x0000000000000000 in ?? ()

@theaungmyatmoe
Copy link

theaungmyatmoe commented May 30, 2024 •
edited
Loading

Hi @dunglas , I can now reproduce the issue with alpine.
After adding $PHPIZE_DEPS and PHP grpc, protobuf extensions, the server cannot started and throw the previous error in both images.

Here is step to reproduce.

ARG PHP_VERSION=8.3
ARG FRANKENPHP_VERSION=latest

FROM dunglas/frankenphp:${FRANKENPHP_VERSION}-php${PHP_VERSION}-alpine


RUN apk add --no-cache \
    $PHPIZE_DEPS \ # <- this
    supervisor \
    nodejs \
    npm  \
    libzip-dev \
    libxml2-dev \
    librdkafka-dev \
    g++ \
    make \
    autoconf \
    && apk del autoconf g++ make  \
    && rm -rf /tmp/*  \
    && rm -rf /var/cache/apk/*


RUN install-php-extensions \
    redis \
    rdkafka \
    exif \
    pdo_mysql \
    zip \
    sockets \
    pcntl \
    opcache \
    mongodb \
    gd \
    opcache \
    intl \
    bcmath\ 
    grpc \ # <- this
    protobuf <- this

Supervisor config

command=php %(ENV_APP_PATH)s/artisan octane:start --server=frankenphp --host=0.0.0.0 --port=80 --admin-port=2019 --watch

@theaungmyatmoe
Copy link

Franken is fine without grpc and prtobuf extensions and started to work. Then I cannot use grpc. 🥲
It might be an issue with $PHP_DEPS? You can reference below. Thanks.

docker-library/php/issues/412

@dunglas
Copy link
Owner

dunglas commented May 30, 2024

@jeremyj11 , thanks! This is likely this issue: php/php-src#13648

As a workaround, switching to a FrankenPHP build that uses glibc (like the one we provide in the Debian Docker images) instead of musl should fix the problem.

@dunglas dunglas mentioned this issue May 30, 2024
Open
@dunglas
Copy link
Owner

dunglas commented May 30, 2024

@jeremyj11 is the code calling openssl_pkey_get_private() public, or could you share it?

@jeremyj11
Copy link
Contributor

jeremyj11 commented May 30, 2024 •
edited
Loading

@jeremyj11 is the code calling openssl_pkey_get_private() public, or could you share it?

It's public - a fresh Laravel 11 install with Passport should contain it, it's here:
https://github.com/thephpleague/oauth2-server/blob/master/src/CryptKey.php

I've confirmed the above is what causes the crash, but from searching my vendor directory the following packages also contain it:
Symfony\Component\Mime\Crypto\DkimSigner;
Lcobucci\JWT\Signer\OpenSSL;
Firebase\JWT\JWT;

I'd be happy to create a minimal reproduction git repo for you if you like?

@dunglas
Copy link
Owner

dunglas commented May 30, 2024

@jeremyj11 I think we'll switch to the PHP bug tracker, as @bukka where already investigating a similar issue and has a good knowledge of OpenSSL-related things, but I'm sure a minimal reproducer will help!

@theaungmyatmoe
Copy link

theaungmyatmoe commented May 30, 2024 via email

@jeremyj11
Copy link
Contributor

@jeremyj11 , thanks! This is likely this issue: php/php-src#13648

As a workaround, switching to a FrankenPHP build that uses glibc (like the one we provide in the Debian Docker images) instead of musl should fix the problem.

I compiled frankenphp from sources and it's working fine now 🎉

@withinboredom withinboredom mentioned this issue May 31, 2024
Closed
@dunglas dunglas closed this as not planned Won't fix, can't repro, duplicate, stale Jun 10, 2024
@g-ra g-ra mentioned this issue Jul 31, 2024
Closed
@g-ra
Copy link

g-ra commented Aug 6, 2024

@jeremyj11 , thanks! This is likely this issue: php/php-src#13648
As a workaround, switching to a FrankenPHP build that uses glibc (like the one we provide in the Debian Docker images) instead of musl should fix the problem.

I compiled frankenphp from sources and it's working fine now 🎉

can you sgare examples please
i have same issue #944

@jeremyj11
Copy link
Contributor

@jeremyj11 , thanks! This is likely this issue: php/php-src#13648
As a workaround, switching to a FrankenPHP build that uses glibc (like the one we provide in the Debian Docker images) instead of musl should fix the problem.

I compiled frankenphp from sources and it's working fine now 🎉

can you sgare examples please i have same issue #944

I followed the instruction here https://frankenphp.dev/docs/compile/ for Linux

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@dunglas @emaia @g-ra @m-hm @jeremyj11 @theaungmyatmoe