runtime: segfault in sync.Pool when using a shared runtime #24640
Labels
FrozenDueToAge
NeedsInvestigation
Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Comments
|
The original post from the linked bug report:
|
|
On that bug report, Matt Madison (perhaps @madisongh?) notes:
And later:
|
bcmills
added
the
NeedsInvestigation
bcmills
changed the title
|
(@aclements, @randall77: any ideas? The initial bugreport theorizes that the compiler may be generating invalid code.) |
|
Shouldn't this be tagged for 1.10.2 release? It is a regression after all. |
|
That probably depends on what the root cause turns out to be and how invasive the fix is. But sure, we can tag it for 1.10.2 until we figure that out. |
|
Where is the stack trace? It should be in this bug. I also don't see it in the linked bugzilla bug. |
|
Stack traces from a few sample runs: Example 1: Example 2: Example 3: |
|
Any idea how we can debug this? |
|
Have you run the program under the race detector? |
|
In stack trace 3 above it's clear that a value that should be a pointer to a type structure is actually a string. So memory is being corrupted somewhere. The first guesses are always the race detector or invalid use of cgo. |
halstead
pushed a commit
to openembedded/openembedded-core
that referenced
this issue
Apr 13, 2018
For the time being, there is a serious bug[1] in Go 1.10 when it comes to use the shared runtime support which cases problems in multiple projects. 1. golang/go#24640 It is still unclear if the problem arises from a bug inside the compiler itself or it makes a real problem more visible. Either way, using 1.10 as default seems to be a risk so we are changing back to 1.9 for now. Refs: [YOCTO: #12631] Signed-off-by: Otavio Salvador <otavio@ossystems.com.br> Signed-off-by: Ross Burton <ross.burton@intel.com>
jpuhlman
pushed a commit
to MontaVista-OpenSourceTechnology/poky
that referenced
this issue
Apr 21, 2018
Source: poky MR: 00000 Type: Integration Disposition: Merged from poky ChangeID: f6d8c3f Description: For the time being, there is a serious bug[1] in Go 1.10 when it comes to use the shared runtime support which cases problems in multiple projects. 1. golang/go#24640 It is still unclear if the problem arises from a bug inside the compiler itself or it makes a real problem more visible. Either way, using 1.10 as default seems to be a risk so we are changing back to 1.9 for now. Refs: [YOCTO: #12631] (From OE-Core rev: c5b5055d2dc04317a7a64c150046a6435a6805c2) Signed-off-by: Otavio Salvador <otavio@ossystems.com.br> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Jeremy Puhlman <jpuhlman@mvista.com>
jpuhlman
pushed a commit
to MontaVista-OpenSourceTechnology/poky
that referenced
this issue
Apr 21, 2018
Source: poky MR: 00000 Type: Integration Disposition: Merged from poky ChangeID: f6d8c3f Description: For the time being, there is a serious bug[1] in Go 1.10 when it comes to use the shared runtime support which cases problems in multiple projects. 1. golang/go#24640 It is still unclear if the problem arises from a bug inside the compiler itself or it makes a real problem more visible. Either way, using 1.10 as default seems to be a risk so we are changing back to 1.9 for now. Refs: [YOCTO: #12631] (From OE-Core rev: c5b5055d2dc04317a7a64c150046a6435a6805c2) Signed-off-by: Otavio Salvador <otavio@ossystems.com.br> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Jeremy Puhlman <jpuhlman@mvista.com>
|
@gopherbot please open a backport tracking issue for 1.10. |
|
Backport issue(s) opened: #25037 (for 1.10). Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases. |
ianlancetaylor
added
the
WaitingForInfo
|
Timed out in state WaitingForInfo. Closing. (I am just a bot, though. Please speak up if this is a mistake or you have the requested information.) |
|
This is still a problem with go 1.11. |
|
@jasonkim, we still need more information to debug this. Since it sounds like you can reproduce it, can you provide stack traces and try running under the race detector? |
|
We could only get this going disabling the dynamic linking of runtime. |
|
I tried adding -race to the build/install step and got the following error. What is there to do for this? I'm assuming since my env is an ARM based, the race detector won't work? |
|
Can you try the same program on amd64 with the race detector? Either way, can you provide stack traces? We don't have enough information here to identify the problem. |
|
Here's a couple logs I saw during my testing. These are pretty representative as all the other ones I saw were pretty much the same. It seems to occur when mqtt is connecting (in go routine?) and log happens at/near the same time. Log 1 Log 2 I don't have amd64 machine to test on (in fact we only have 1-2 arm type machines). I am planning to test it on my local env (OS X) but will have to figure out my build and how to run it with race detector. Will report back when I do. |
|
I wasn't able to repro on my OS X (with or without -race), but it's possible that I'm running it differently than how it's run in our deployed env. |
ianlancetaylor
removed
the
WaitingForInfo
|
There's some repro steps in the original ticket. It may help with debugging. |
|
The problem still exists under 1.13.8 under the same conditions. 1.14.1 also fails at the same location, but the stacktrace is less detailed and reports a panic instead of a fatal error |
|
Is it possible to capture an rr recording? https://github.com/mozilla/rr |
This was referenced Nov 18, 2020
|
Obsoleted by #47788 |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
QPlease answer these questions before submitting your issue. Thanks!
What version of Go are you using (
go version)?1.10
Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (
go env)?Reproducible on ARM and x86_64 under Linux
What did you do?
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12631#c10
The text was updated successfully, but these errors were encountered: