Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Rustc endless loop out-of-memory and consequent SIGKILL in generic new type #67690

Closed
dangerousplay opened this issue Dec 28, 2019 · 9 comments · Fixed by #67880
Closed

Rustc endless loop out-of-memory and consequent SIGKILL in generic new type #67690

dangerousplay opened this issue Dec 28, 2019 · 9 comments · Fixed by #67880
Labels
C-bug Category: This is a bug. I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@dangerousplay
Copy link

Hi, i have made some changes for a crate stubborn-io to support a generic type that implements ToSocketAddrs from Tokio, but suddenly the code can't compile because it runs out of memory with 6, 8, 10 GB.

This is the code i'm trying to compile:

use super::io::{StubbornIo, UnderlyingIo};
use std::future::Future;
use std::io;
use tokio::net::ToSocketAddrs;
use std::pin::Pin;
use tokio::net::TcpStream;

impl<A> UnderlyingIo<A> for TcpStream
where A: ToSocketAddrs + Sync + Send + Clone + Unpin + 'static {
    fn establish(addr: A) -> Pin<Box<dyn Future<Output = io::Result<Self>> + Send>> {
        Box::pin(TcpStream::connect(addr))
    }
}

pub type StubbornTcpStream<A: ToSocketAddrs + Sync + Send + Clone + Unpin + 'static> = StubbornIo<TcpStream, A>;

if changed the type A for SocketAddr or String it then compiles and runs fine.

Source: https://github.com/dangerousplay/stubborn-io/blob/64bad09fbb398ceca0add972399cddd04751e378/src/tokio/tcp.rs#L1-L29

It fails to compile on x86_64-unknown-linux-gnu with stable, nightly.

I'm attaching the Heaptrack files for further analysis.
https://gofile.io/?c=Vnar2K
@dangerousplay
Copy link
Author

It seems a problem with regex utilized in the rustc, unfortunately heaptrack memory report is strange, the execution time tracked is wrong besides the graph shows up the memory grow.

I'm thinking ways to better troubleshoot, but the side effects of the issue is hard.

@Mark-Simulacrum
Copy link
Member

It'd be helpful to try and simplify the code, ideally to something that doesn't have dependencies (beyond std or so), or at least as few dependencies as possible.

As a start putting up a github repository with a Cargo.toml and as simple a crate as possible would be helpful for reproducing on our side (i.e., extracting the file you've linked from that crate -- or, if the crate is really necessary, then indicating as such).

The heaptrack profile looks pretty useless unfortunately -- rustc on linux is shipped with jemalloc as the allocator which likely makes heaptrack tracking not really work too well.

@dangerousplay
Copy link
Author

dangerousplay commented Dec 31, 2019 •
edited
Loading

Thank you, sorry for the late response (working) but i get it working on playground.

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=4512361986a733a75f43705ca512a736

I'm now trying to minimize them to focus on the issue.

@Byter09
Copy link

Byter09 commented Dec 31, 2019 •
edited
Loading

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=562a19ca1324c9c57b06149d0e0dd19a

Got it down to 18 lines. Still produces the same error. I have not tested if this goes out-of-memory. But receiving a SIGKILL would signal that.

EDIT:
After taking some more time, I got rid of Tokio and reduced it to 6 lines:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=110e3925bb9e920ccca499b059cf2cc3

Additionally, I advise to rename this issue to something like out-of-memory and consequent SIGKILL in generic new type to make it very clear and concise.

@dangerousplay dangerousplay changed the title Rustc endless loop OOM killer Rustc endless loop out-of-memory and consequent SIGKILL in generic new type Dec 31, 2019
@jyn514
Copy link
Member

jyn514 commented Dec 31, 2019 •
edited
Loading

One liner: pub type T<A: Send + Sync + Clone> = A;.

Strangely, this no longer crashes if you remove any of pub, Send, Sync, or Clone.
There's nothing special about Send or Sync, this also crashes:

#![no_std]
mod a {
    pub trait A {}
}
pub trait B {}
pub trait C {}
pub type T<P: a::A + B + C> = P;

Even weirder, this does not crash: pub type T<P: B + C + a::A> = P;

(I didn't come up with the one-liner, that was @pythondude325)

@jyn514
Copy link
Member

jyn514 commented Dec 31, 2019 •
edited
Loading

I stopped rustc (SIGSTOP) and got a backtrace with gdb, it looks like it's trying to generate infinite errors:

backtrace 
#0  0x00007ff429b08329 in rustc_errors::styled_buffer::StyledBuffer::putc ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#1  0x00007ff429b21fb7 in <rustc_errors::emitter::EmitterWriter as rustc_errors::emitter::Emitter>::emit_diagnostic ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#2  0x00007ff429b372a9 in rustc_errors::HandlerInner::emit_diagnostic ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#3  0x00007ff429b0662b in rustc_errors::diagnostic_builder::DiagnosticBuilder::emit
    ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#4  0x00007ff427eb00ec in <rustc_lint::builtin::TypeAliasBounds as rustc::lint::LateLintPass>::check_item ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#5  0x00007ff427e40ca6 in <rustc_lint::BuiltinCombinedModuleLateLintPass as rustc::lint::LateLintPass>::check_item ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#6  0x00007ff427e6652e in rustc::hir::intravisit::Visitor::visit_nested_item ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#7  0x00007ff427e6850d in rustc::lint::context::late_lint_mod ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#8  0x00007ff427df5c8a in rustc::ty::query::__query_compute::lint_mod ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#9  0x00007ff427e1263b in rustc::ty::query::<impl rustc::ty::query::config::QueryAccessors for rustc::ty::query::queries::lint_mod>::compute ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#10 0x00007ff427dfa04d in rustc::dep_graph::graph::DepGraph::with_task_impl ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#11 0x00007ff427e25174 in rustc::ty::query::plumbing::<impl rustc::ty::context::TyCtxt>::get_query ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin---Type <return> to continue, or q <return> to quit---
/../lib/librustc_driver-63848c3330cc991c.so
#12 0x00007ff427e12c25 in rustc::ty::query::plumbing::<impl rustc::ty::context::TyCtxt>::ensure_query ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#13 0x00007ff427d57876 in rustc::lint::context::check_crate::{{closure}}::{{closure}} ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#14 0x00007ff427d582f6 in rustc::util::common::time ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#15 0x00007ff427d58956 in rustc::util::common::time ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#16 0x00007ff4277aa44a in __rust_maybe_catch_panic ()
    at src/libpanic_unwind/lib.rs:78
#17 0x00007ff427de3f73 in <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#18 0x00007ff4277aa44a in __rust_maybe_catch_panic ()
    at src/libpanic_unwind/lib.rs:78
#19 0x00007ff427d5e4e2 in rustc_interface::passes::analysis::{{closure}} ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#20 0x00007ff427d5df45 in rustc_interface::passes::analysis ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#21 0x00007ff427ce8481 in rustc::ty::query::__query_compute::analysis ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#22 0x00007ff427ceac7d in rustc::dep_graph::graph::DepGraph::with_task_impl ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#23 0x00007ff427c633c7 in rustc::ty::query::plumbing::<impl rustc::ty::context::TyCtxt>::get_query ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#24 0x00007ff427ceda3a in rustc_interface::passes::BoxedGlobalCtxt::access::{{closure}} ()
---Type <return> to continue, or q <return> to quit---
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#25 0x00007ff427daf92a in rustc_interface::passes::create_global_ctxt::{{closure}}
    ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#26 0x00007ff427ced78e in rustc_interface::passes::BoxedGlobalCtxt::enter ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#27 0x00007ff427cbd7c7 in rustc_interface::interface::run_compiler_in_existing_thread_pool ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#28 0x00007ff427c9e112 in std::thread::local::LocalKey<T>::with ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#29 0x00007ff427c9bbce in scoped_tls::ScopedKey<T>::set ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#30 0x00007ff427cca264 in syntax::with_globals ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#31 0x00007ff427ce28d0 in std::sys_common::backtrace::__rust_begin_short_backtrace
    ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#32 0x00007ff4277aa44a in __rust_maybe_catch_panic ()
    at src/libpanic_unwind/lib.rs:78
#33 0x00007ff427ce3809 in core::ops::function::FnOnce::call_once{{vtable-shim}} ()
   from /home/joshua/.local/lib/rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-63848c3330cc991c.so
#34 0x00007ff42777ae2f in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once ()
    at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/liballoc/boxed.rs:942
#35 0x00007ff4277a8e70 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once ()
    at /rustc/73528e339aae0f17a15ffa49a8ac608f50c6cf14/src/liballoc/boxed.rs:942
#36 std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#37 std::sys::unix::thread::Thread::new::thread_start ()
    at src/libstd/sys/unix/thread.rs:79
#38 0x00007ff4275156db in start_thread (arg=0x7ff425bff700) at pthread_create.c:463
---Type <return> to continue, or q <return> to quit---
#39 0x00007ff426e3288f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

@jyn514
Copy link
Member

jyn514 commented Dec 31, 2019

It looks like this is a problem with the error reporter itself: it's affected by the length of the name (!!)

OOM:

pub trait AAAAAAAAAAAAAA {}
pub trait B {}
pub type T<P: AAAAAAAAAAAAAA + B> = P;

No OOM:

pub trait AAAAAAAAAAAAA {}
pub trait B {}
pub type T<P: AAAAAAAAAAAAA + B> = P;

@pythongirl325
Copy link

pythongirl325 commented Dec 31, 2019 •
edited
Loading

I figured out the issue has to do with how the error printer places the annotations, adding a newline after the trait names and before the plus solves the issue.

pub trait AAAA {}
pub trait B {}
pub trait C {}
pub type T<P: AAAA
+ B + C> = P;

@pythongirl325
Copy link

Probably caused by some code in here: https://github.com/rust-lang/rust/blob/master/src/librustc_errors/emitter.rs#L792-L849

@lbonn lbonn mentioned this issue Jan 5, 2020
Merged
@jonas-schievink jonas-schievink added C-bug Category: This is a bug. I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jan 5, 2020
Dylan-DPC-zz pushed a commit to Dylan-DPC-zz/rust that referenced this issue Jan 6, 2020
@Dylan-DPC
Rollup merge of rust-lang#67880 - lbonn:fix/multi-substs, r=petrochenkov
9f1e7b0 
Handle multiple error fix suggestions carefuly

The existing code seems to assume that substitutions spans are disjoint,
which is not always the case.

In the example:

    pub trait AAAA {}
    pub trait B {}
    pub trait C {}
    pub type T<P: AAAA + B + C> = P;

, we get three substituions starting from ':' and ending respectively at
the end of each trait token.

With the former offset calculation, this would cause `underline_start` to
eventually become negative before being converted to `usize`...

The new version may report erroneous results for non perfectly overlapping
substitutions but I don't know if such examples exist. Alternatively, we
could detect these cases and trim out overlapping substitutions.

Fixes rust-lang#67690
@bors bors closed this as completed in 6d9913d Jan 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-bug Category: This is a bug. I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Handle multiple error fix suggestions carefuly lbonn/rust
6 participants
@jonas-schievink @Mark-Simulacrum @pythongirl325 @Byter09 @jyn514 @dangerousplay