Add a way to debug or get panic information from failing doctests

[jrmuizel]

I wasn't able to find a way to do this.

[matklad]

RUST_BACKTRACE=1 cargo test --doc should print the usual backtrace information.

[jrmuizel]

It doesn't seem to for me.

///
/// ```
/// fn bar() {
///     panic!();
/// }
/// fn main() {
///     bar();
/// }
/// ```
pub fn foo() {
}

#[cfg(test)]
mod tests {
    #[test]
    fn it_works() {
        assert_eq!(2 + 2, 4);
    }
}

gives:

$ RUST_BACKTRACE=1 cargo test --doc
   Compiling dod v0.1.0 (file:///private/tmp/dod)
    Finished dev [unoptimized + debuginfo] target(s) in 0.71s
   Doc-tests dod

running 1 test
test src/lib.rs - foo (line 3) ... FAILED

failures:

failures:
    src/lib.rs - foo (line 3)

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out

error: test failed, to rerun pass '--doc'

[matklad]

Hm, it seems to work here:

    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
   Doc-tests foo

running 1 test
test src/lib.rs - foo (line 2) ... FAILED

failures:

---- src/lib.rs - foo (line 2) stdout ----
	thread 'rustc' panicked at 'test executable failed:

thread 'main' panicked at 'explicit panic', src/lib.rs:3:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::sys_common::backtrace::print
             at libstd/sys_common/backtrace.rs:71
             at libstd/sys_common/backtrace.rs:59
   2: std::panicking::default_hook::{{closure}}
             at libstd/panicking.rs:207
   3: std::panicking::default_hook
             at libstd/panicking.rs:223
   4: std::panicking::rust_panic_with_hook
             at libstd/panicking.rs:402
   5: std::panicking::begin_panic
   6: rust_out::bar
   7: rust_out::main
   8: std::rt::lang_start::{{closure}}
   9: std::panicking::try::do_call
             at libstd/rt.rs:59
             at libstd/panicking.rs:306
  10: __rust_maybe_catch_panic
             at libpanic_unwind/lib.rs:102
  11: std::rt::lang_start_internal
             at libstd/panicking.rs:285
             at libstd/panic.rs:361
             at libstd/rt.rs:58
  12: std::rt::lang_start
  13: main
  14: __libc_start_main
  15: _start
             at ../sysdeps/x86_64/start.S:120

', librustdoc/test.rs:341:17
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::sys_common::backtrace::print
             at libstd/sys_common/backtrace.rs:71
             at libstd/sys_common/backtrace.rs:59
   2: std::panicking::default_hook::{{closure}}
             at libstd/panicking.rs:207
   3: std::panicking::default_hook
             at libstd/panicking.rs:217
   4: core::ops::function::Fn::call
   5: std::panicking::rust_panic_with_hook
             at libstd/panicking.rs:403
   6: std::panicking::begin_panic_fmt
             at libstd/panicking.rs:349
   7: rustdoc::test::run_test
             at librustdoc/test.rs:341
   8: syntax::with_globals
             at librustdoc/test.rs:537
             at /cargo/registry/src/github.com-1ecc6299db9ec823/scoped-tls-0.1.1/src/lib.rs:155
             at /checkout/src/libsyntax/lib.rs:100
             at /cargo/registry/src/github.com-1ecc6299db9ec823/scoped-tls-0.1.1/src/lib.rs:155
             at /checkout/src/libsyntax/lib.rs:99


failures:
    src/lib.rs - foo (line 2)

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out

error: test failed, to rerun pass '--doc'

Not sure what would be the steps to debug this...

[jrmuizel]

I'm on macOS. Perhaps you're on Linux?

[matklad]

Yep, I am on linux. The relevant Cargo code is here:

cargo/src/cargo/ops/cargo_test.rs

Line 144 in 5845464

fn run_doc_tests(

.

It might be the case that this is actually an issue with rustdoc and not Cargo per se.

[ehuss]

@jrmuizel which version of rust are you using?

I'm able to repro this on nightly rustc 1.28.0-nightly (2a1c4eec4 2018-06-25) (linux and macos). I can run a bisect to see what changed.

[jrmuizel]

rustc 1.28.0-nightly (523097979 2018-06-18)

[ehuss]

This is rust-lang/rust#51162.

[matklad]

Closing in favor of rust-lang/rust issue!