rustdoc 2015 regression: doc test fails (unresolved import) if it contains a doc comment

[japaric]

STR

Cargo.toml

[package]
name = "foo"
version = "0.1.0"
authors = ["Jorge Aparicio <jorge@japaric.io>"]

[dependencies]

[dev-dependencies]
stm32f30x = "=0.8.0"

src/lib.rs

//! ```
//! // crate: stm32f30x-hal
//! //! An implementation of the `embedded-hal` traits for STM32F30x microcontrollers
//!
//! // device crate
//! extern crate stm32f30x;
//!
//! use stm32f30x::USART1;
//!
//! # fn main() {}
//! ```

$ cargo test
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running target/debug/deps/foo-794f65deaf39282e

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

   Doc-tests foo

running 1 test
test src/lib.rs -  (line 1) ... FAILED

failures:

---- src/lib.rs -  (line 1) stdout ----
error[E0432]: unresolved import `stm32f30x`
 --> src/lib.rs:8:5
  |
9 | use stm32f30x::USART1;
  |     ^^^^^^^^^ maybe a missing `extern crate stm32f30x;`?

thread 'src/lib.rs -  (line 1)' panicked at 'couldn't compile the test', src/librustdoc/test.rs:327:13
note: Run with `RUST_BACKTRACE=1` for a backtrace.


failures:
    src/lib.rs -  (line 1)

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out

error: test failed, to rerun pass '--doc'

$ rustc -V
rustc 1.32.0-nightly (4a45578bc 2018-12-07)

Ways to "fix" the error:

• Add edition = "2018" to Cargo.toml, OR
• Remove the doc comment (//!), OR
• Switch to an older nightly (e.g. nightly-2018-11-01)

[QuietMisdreavus]

I almost have a PR ready for this.

Here's the basic structure of what's happening:

1. Rustdoc scans the doctest to split out crate attributes and crate statements from the doctest. However, the current scan doesn't expect comments in the doctest, so upon encountering the comment at the start of the doctest, it dumps the whole thing into what it calls "everything else".
2. Rustdoc scans the doctest again to look for fn main and extern crate embedded-hal. This is the libsyntax-based scan. However, there is now a //! doc comment at the beginning of this sample, which causes an error. The parsing loop here is expecting to pull out individual items - it doesn't treat the whole sample as a potential block, so the inner attribute causes a parse error. This error is silenced, and the loop exits, claiming to have not found the fn main later on.
3. Because the loop did not find fn main, Rustdoc proceeds to wrap the sample in another one.
4. Upon attempting to compile the completed doctest, the compiler doesn't register the extern crate statement inside the generated main function, so when it tries to resolve the use statement, the resolution fails.
   • However! If the doctest is being compiled under 2018, then the crate is available in the extern prelude! In this situation, the compiler can ultimately ignore the extern crate statement (which is valid despite being unaddressable) and load the path from there.

EDIT: (submitted too early, oops)

So, the solution is to make sure rustdoc can handle comments when splitting up doctests. Like i said, i almost have this ready.

[QuietMisdreavus]

That PR is now live: #56793