Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buffer lexer errors in rustdoc syntax checking #67563

Merged
merged 1 commit into from
Jan 6, 2020

Conversation

euclio
Copy link
Contributor

@euclio euclio commented Dec 23, 2019

The code isn't ideal (I really would like to display the errors inline), but this at least gets us to where we were before #63017.

@JohnTitor JohnTitor added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Dec 23, 2019
@euclio euclio force-pushed the rustdoc-buffer-lexer branch from 1a52ae5 to ac14d81 Compare December 23, 2019 17:37
@Dylan-DPC-zz
Copy link

r? @GuillaumeGomez

@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-12-23T18:00:09.1793362Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-12-23T18:00:09.2070551Z ##[command]git config gc.auto 0
2019-12-23T18:00:09.2133399Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-12-23T18:00:09.2198895Z ##[command]git config --get-all http.proxy
2019-12-23T18:00:09.2350396Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/67563/merge:refs/remotes/pull/67563/merge
---
2019-12-23T19:01:13.4619881Z .................................................................................................... 1600/9427
2019-12-23T19:01:18.1175655Z .................................................................................................... 1700/9427
2019-12-23T19:01:28.6091614Z .....................................................................................i.............. 1800/9427
2019-12-23T19:01:36.2290399Z .................................................................................................... 1900/9427
2019-12-23T19:01:43.3066259Z .......................................................................iiiii........................ 2000/9427
2019-12-23T19:02:04.2509670Z .................................................................................................... 2200/9427
2019-12-23T19:02:06.6742299Z .................................................................................................... 2300/9427
2019-12-23T19:02:09.3400094Z .................................................................................................... 2400/9427
2019-12-23T19:02:22.6603420Z .................................................................................................... 2500/9427
---
2019-12-23T19:05:15.6508630Z .i...............i.................................................................................. 4900/9427
2019-12-23T19:05:25.9363075Z .................................................................................................... 5000/9427
2019-12-23T19:05:31.3909686Z .............................................i...................................................... 5100/9427
2019-12-23T19:05:41.3123835Z .................................................................................................... 5200/9427
2019-12-23T19:05:47.1251634Z ............ii.ii...........i....................................................................... 5300/9427
2019-12-23T19:05:56.3184433Z .................................................................................................... 5500/9427
2019-12-23T19:06:07.5396799Z ..............................................................................................i..... 5600/9427
2019-12-23T19:06:15.6136826Z .................................................................................................... 5700/9427
2019-12-23T19:06:20.8162411Z .................................................................................................... 5800/9427
2019-12-23T19:06:20.8162411Z .................................................................................................... 5800/9427
2019-12-23T19:06:30.4272363Z ..................................................................................ii...i...ii....... 5900/9427
2019-12-23T19:06:53.2309876Z .................................................................................................... 6100/9427
2019-12-23T19:07:00.9544495Z .................................................................................................... 6200/9427
2019-12-23T19:07:08.2375343Z .................................................................................................... 6300/9427
2019-12-23T19:07:08.2375343Z .................................................................................................... 6300/9427
2019-12-23T19:07:21.3754229Z .........i..ii...................................................................................... 6400/9427
2019-12-23T19:07:40.9093994Z .....................................................................................i.............. 6600/9427
2019-12-23T19:07:43.0772868Z .................................................................................................... 6700/9427
2019-12-23T19:07:45.3057233Z .....................................................................................i.............. 6800/9427
2019-12-23T19:07:48.0705526Z .................................................................................................... 6900/9427
---
2019-12-23T19:13:02.1779735Z  finished in 6.514
2019-12-23T19:13:02.2005566Z Check compiletest suite=codegen mode=codegen (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-12-23T19:13:02.3915561Z 
2019-12-23T19:13:02.3915817Z running 166 tests
2019-12-23T19:13:05.5166000Z iiii......i........ii..iiii...i.............................i..i..................i....i............ 100/166
2019-12-23T19:13:07.6423027Z i.i.i...iii..ii.iiiii......................iii............ii......
2019-12-23T19:13:07.6425685Z 
2019-12-23T19:13:07.6427769Z  finished in 5.442
2019-12-23T19:13:07.6643321Z Check compiletest suite=codegen-units mode=codegen-units (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-12-23T19:13:07.8371335Z 
---
2019-12-23T19:13:09.9156979Z  finished in 2.251
2019-12-23T19:13:09.9379190Z Check compiletest suite=assembly mode=assembly (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-12-23T19:13:10.1401640Z 
2019-12-23T19:13:10.1404923Z running 9 tests
2019-12-23T19:13:10.1406794Z iiiiiiiii
2019-12-23T19:13:10.1409136Z 
2019-12-23T19:13:10.1410200Z  finished in 0.202
2019-12-23T19:13:10.1627485Z Check compiletest suite=incremental mode=incremental (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-12-23T19:13:10.3601471Z 
---
2019-12-23T19:13:30.1521938Z  finished in 19.988
2019-12-23T19:13:30.1765833Z Check compiletest suite=debuginfo mode=debuginfo-gdb+lldb (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-12-23T19:13:30.3744218Z 
2019-12-23T19:13:30.3744518Z running 124 tests
2019-12-23T19:13:55.4234114Z .iiiii..ii.....i..i...i..i.i.i..i..i..iii....ii.ii....ii..........iiii..........i.....i..ii.......ii 100/124
2019-12-23T19:13:59.6386910Z .i.iii.....iiiiii.....ii
2019-12-23T19:13:59.6387494Z 
2019-12-23T19:13:59.6387544Z  finished in 29.461
2019-12-23T19:13:59.6397304Z Uplifting stage1 rustc (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-12-23T19:13:59.6397693Z Copying stage2 rustc from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
---
2019-12-23T19:27:39.7186805Z 
2019-12-23T19:27:39.7196157Z    Doc-tests core
2019-12-23T19:27:44.4612387Z 
2019-12-23T19:27:44.4613933Z running 2439 tests
2019-12-23T19:27:53.4668139Z ......iiiii......................................................................................... 100/2439
2019-12-23T19:28:02.3242694Z ..................................................................................ii................ 200/2439
2019-12-23T19:28:23.1760025Z ................i................................................................................... 400/2439
2019-12-23T19:28:23.1760025Z ................i................................................................................... 400/2439
2019-12-23T19:28:32.4817680Z ................................................................i..i..................iiii.......... 500/2439
2019-12-23T19:28:48.7882669Z .................................................................................................... 700/2439
2019-12-23T19:28:57.2248453Z .................................................................................................... 800/2439
2019-12-23T19:29:05.7093823Z .................................................................................................... 900/2439
2019-12-23T19:29:14.3427714Z .................................................................................................... 1000/2439
---
2019-12-23T19:33:02.5082572Z 
2019-12-23T19:33:02.5084245Z running 1003 tests
2019-12-23T19:33:20.5165337Z i................................................................................................... 100/1003
2019-12-23T19:33:30.4834630Z .................................................................................................... 200/1003
2019-12-23T19:33:37.5839494Z ..................iii......i......i...i......i...................................................... 300/1003
2019-12-23T19:33:42.4073534Z .................................................................................................... 400/1003
2019-12-23T19:33:49.1630282Z ..........................................i..i.....................................ii............... 500/1003
2019-12-23T19:34:01.8543835Z .................................................................................................... 700/1003
2019-12-23T19:34:01.8543835Z .................................................................................................... 700/1003
2019-12-23T19:34:08.2397543Z .............................iiii................................................................... 800/1003
2019-12-23T19:34:22.1404269Z .................................................................................................... 900/1003
2019-12-23T19:34:29.0205981Z ...................................................iiii............................................. 1000/1003
2019-12-23T19:34:29.1232398Z test result: ok. 983 passed; 0 failed; 20 ignored; 0 measured; 0 filtered out
2019-12-23T19:34:29.1232436Z 
2019-12-23T19:34:29.1360212Z  finished in 182.729
2019-12-23T19:34:29.1374181Z Testing term stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
---
2019-12-23T19:53:49.5469175Z  finished in 38.711
2019-12-23T19:53:49.5872994Z Check compiletest suite=run-make-fulldeps mode=run-make (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-12-23T19:53:49.8020741Z 
2019-12-23T19:53:49.8021473Z running 206 tests
2019-12-23T19:54:21.8860195Z ....................i...ii....................................................................i..... 100/206
2019-12-23T19:54:53.4097222Z ..................................iiiiii......i............iiii.iii................................. 200/206
2019-12-23T19:54:55.7803331Z test result: ok. 187 passed; 0 failed; 19 ignored; 0 measured; 0 filtered out
2019-12-23T19:54:55.7803380Z 
2019-12-23T19:54:55.7814278Z  finished in 66.194
2019-12-23T19:54:55.8099135Z doc tests for: /checkout/src/doc/rustdoc/src/advanced-features.md
---
2019-12-23T19:55:23.1100188Z 135 
2019-12-23T19:55:23.1100233Z 
2019-12-23T19:55:23.1100260Z 
2019-12-23T19:55:23.1100305Z The actual stderr differed from the expected stderr.
2019-12-23T19:55:23.1101078Z Actual stderr saved to /checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc-ui/invalid-syntax/invalid-syntax.stderr
2019-12-23T19:55:23.1101389Z To update references, rerun the tests and pass the `--bless` flag
2019-12-23T19:55:23.1101650Z To only update this specific test, also pass `--test-args invalid-syntax.rs`
2019-12-23T19:55:23.1101750Z error: 1 errors occurred comparing output.
2019-12-23T19:55:23.1101796Z status: exit code: 0
2019-12-23T19:55:23.1101796Z status: exit code: 0
2019-12-23T19:55:23.1102584Z command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "/checkout/src/test/rustdoc-ui/invalid-syntax.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--error-format" "json" "-Zui-testing" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc-ui/invalid-syntax" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc-ui/invalid-syntax/auxiliary"
2019-12-23T19:55:23.1102925Z ------------------------------------------
2019-12-23T19:55:23.1102977Z 
2019-12-23T19:55:23.1103193Z ------------------------------------------
2019-12-23T19:55:23.1103240Z stderr:
2019-12-23T19:55:23.1103240Z stderr:
2019-12-23T19:55:23.1103451Z ------------------------------------------
2019-12-23T19:55:23.1103518Z warning: could not parse code block as Rust code
2019-12-23T19:55:23.1103755Z   --> /checkout/src/test/rustdoc-ui/invalid-syntax.rs:3:5
2019-12-23T19:55:23.1103805Z    |
2019-12-23T19:55:23.1103866Z LL |   /// ```
2019-12-23T19:55:23.1103909Z    |  _____^
2019-12-23T19:55:23.1104591Z LL | | /// \__________pkt->size___________/          \_result->size_/ \__pkt->size__/
2019-12-23T19:55:23.1104824Z    | |_______^
2019-12-23T19:55:23.1108244Z    |
2019-12-23T19:55:23.1108323Z help: mark blocks that do not contain Rust code as text
2019-12-23T19:55:23.1108371Z    |
---
2019-12-23T19:55:23.1112788Z 
2019-12-23T19:55:23.1112850Z warning: doc comment contains an invalid Rust code block
2019-12-23T19:55:23.1113087Z   --> /checkout/src/test/rustdoc-ui/invalid-syntax.rs:70:1
2019-12-23T19:55:23.1113135Z    |
2019-12-23T19:55:23.1113209Z LL | / #[doc = "```"]
2019-12-23T19:55:23.1113252Z LL | | /// \_
2019-12-23T19:55:23.1113295Z LL | | #[doc = "```"]
2019-12-23T19:55:23.1113396Z    |
2019-12-23T19:55:23.1113585Z    = help: mark blocks that do not contain Rust code as text: ```text
2019-12-23T19:55:23.1113691Z 
2019-12-23T19:55:23.1113753Z warning: Rust code block is empty
---
2019-12-23T19:55:23.1116913Z test result: FAILED. 33 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out
2019-12-23T19:55:23.1116950Z 
2019-12-23T19:55:23.1116975Z 
2019-12-23T19:55:23.1117009Z 
2019-12-23T19:55:23.1118747Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--rustdoc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "--src-base" "/checkout/src/test/rustdoc-ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc-ui" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "ui" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-7/bin/FileCheck" "--host-rustcflags" "-Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "7.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2019-12-23T19:55:23.1119018Z 
2019-12-23T19:55:23.1119052Z 
2019-12-23T19:55:23.1119119Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
2019-12-23T19:55:23.1119172Z Build completed unsuccessfully in 1:48:41
2019-12-23T19:55:23.1119172Z Build completed unsuccessfully in 1:48:41
2019-12-23T19:55:23.1119222Z == clock drift check ==
2019-12-23T19:55:23.1119360Z   local time: Mon Dec 23 19:55:23 UTC 2019
2019-12-23T19:55:23.4018386Z   network time: Mon, 23 Dec 2019 19:55:23 GMT
2019-12-23T19:55:23.4018588Z == end clock drift check ==
2019-12-23T19:55:24.9575475Z 
2019-12-23T19:55:24.9679420Z ##[error]Bash exited with code '1'.
2019-12-23T19:55:24.9724425Z ##[section]Starting: Checkout
2019-12-23T19:55:24.9726464Z ==============================================================================
2019-12-23T19:55:24.9726525Z Task         : Get sources
2019-12-23T19:55:24.9726596Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@euclio euclio force-pushed the rustdoc-buffer-lexer branch from ac14d81 to bfa1929 Compare December 23, 2019 20:42
@GuillaumeGomez
Copy link
Member

It's a bit better, however we lost the context. It now just says "we have this error in this code block" but doesn't provide the line or anything. Can you provide more information for the users?

@euclio
Copy link
Contributor Author

euclio commented Jan 3, 2020

We could provide the line/col number, but doing anything more would be significant work that I would rather explore in a separate PR (I'm not sure that we can translate the spans in the general case).

To be clear, this is fixing a regression (we've reopened #53919) and bringing the output back to where it was in 1.34.0.

@GuillaumeGomez
Copy link
Member

Then please add a TODO in the code and open an issue so we don't lose track of this. Then r=me. Thanks a lot!

@euclio
Copy link
Contributor Author

euclio commented Jan 4, 2020

@GuillaumeGomez Added a TODO and created the issue.

@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2020-01-04T05:51:21.0056570Z ##[command]git remote add origin https://github.com/rust-lang/rust
2020-01-04T05:51:21.0260941Z ##[command]git config gc.auto 0
2020-01-04T05:51:21.0359078Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2020-01-04T05:51:21.0405900Z ##[command]git config --get-all http.proxy
2020-01-04T05:51:21.0552903Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/67563/merge:refs/remotes/pull/67563/merge
---
2020-01-04T05:57:36.9104904Z    Compiling serde_json v1.0.40
2020-01-04T05:57:38.5776655Z    Compiling tidy v0.1.0 (/checkout/src/tools/tidy)
2020-01-04T05:57:48.2512362Z     Finished release [optimized] target(s) in 1m 21s
2020-01-04T05:57:48.2614231Z tidy check
2020-01-04T05:57:49.3433339Z tidy error: /checkout/src/librustdoc/passes/check_code_block_syntax.rs:103: TODO is deprecated; use FIXME
2020-01-04T05:57:51.0887327Z some tidy checks failed
2020-01-04T05:57:51.0888294Z Found 486 error codes
2020-01-04T05:57:51.0888747Z Found 0 error codes with no tests
2020-01-04T05:57:51.0889040Z Done!
2020-01-04T05:57:51.0889040Z Done!
2020-01-04T05:57:51.0899771Z 
2020-01-04T05:57:51.0900165Z 
2020-01-04T05:57:51.0901472Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor"
2020-01-04T05:57:51.0902048Z 
2020-01-04T05:57:51.0902266Z 
2020-01-04T05:57:51.0943007Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
2020-01-04T05:57:51.0943568Z Build completed unsuccessfully in 0:01:32
2020-01-04T05:57:51.0943568Z Build completed unsuccessfully in 0:01:32
2020-01-04T05:57:51.0977916Z == clock drift check ==
2020-01-04T05:57:51.0985976Z   local time: Sat Jan  4 05:57:51 UTC 2020
2020-01-04T05:57:51.2500141Z   network time: Sat, 04 Jan 2020 05:57:51 GMT
2020-01-04T05:57:51.2504714Z == end clock drift check ==
2020-01-04T05:57:52.6459741Z 
2020-01-04T05:57:52.6581066Z ##[error]Bash exited with code '1'.
2020-01-04T05:57:52.6611593Z ##[section]Starting: Checkout
2020-01-04T05:57:52.6613351Z ==============================================================================
2020-01-04T05:57:52.6613413Z Task         : Get sources
2020-01-04T05:57:52.6613482Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors
Copy link
Contributor

bors commented Jan 4, 2020

☔ The latest upstream changes (presumably #67853) made this pull request unmergeable. Please resolve the merge conflicts.

@euclio euclio force-pushed the rustdoc-buffer-lexer branch from 544032c to 0b79474 Compare January 4, 2020 21:57
@euclio euclio force-pushed the rustdoc-buffer-lexer branch from 0b79474 to 1ad928e Compare January 5, 2020 00:21
@euclio
Copy link
Contributor Author

euclio commented Jan 5, 2020

Rebased.

@GuillaumeGomez
Copy link
Member

@bors: r+

@bors
Copy link
Contributor

bors commented Jan 5, 2020

📌 Commit 1ad928e has been approved by GuillaumeGomez

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 5, 2020
@bors
Copy link
Contributor

bors commented Jan 6, 2020

⌛ Testing commit 1ad928e with merge 33640f0...

bors added a commit that referenced this pull request Jan 6, 2020
buffer lexer errors in rustdoc syntax checking

The code isn't ideal (I really would like to display the errors inline), but this at least gets us to where we were before #63017.
@bors
Copy link
Contributor

bors commented Jan 6, 2020

☀️ Test successful - checks-azure
Approved by: GuillaumeGomez
Pushing 33640f0 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Jan 6, 2020
@bors bors merged commit 1ad928e into rust-lang:master Jan 6, 2020
@euclio euclio deleted the rustdoc-buffer-lexer branch January 30, 2020 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants