Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rollup of 5 pull requests #90203

Merged
merged 35 commits into from
Oct 23, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
4b3f82a
Add updated support for example-analyzer
willcrichton May 9, 2021
7831fee
Fix check issue
willcrichton May 30, 2021
2855bf0
Factor scraping and rendering into separate calls to rustdoc
willcrichton Jun 1, 2021
b6338e7
Generate example source files with corresponding links
willcrichton Jun 3, 2021
eea8f0a
Sort examples by size
willcrichton Aug 26, 2021
55bb517
Move highlighting logic from JS to Rust
willcrichton Aug 26, 2021
18edcf8
Reduce blur size, fix example width bug, add better error handling fo…
willcrichton Sep 14, 2021
a1cb194
Add styles for non-white themes
willcrichton Sep 14, 2021
829b1a9
Incorporate jyn's feedback
willcrichton Sep 17, 2021
5c05b3c
Add target crates as inputs to reduce size of intermediates
willcrichton Sep 20, 2021
df5e3a6
Change serialized format to use DefPathHash instead of custom String
willcrichton Sep 21, 2021
25323ec
Move JS into a standalone file
willcrichton Oct 1, 2021
55731bb
Fix lint error, change scrape-examples.js minify call
willcrichton Oct 1, 2021
ed8e12f
Unversioned -> InvocationSpecific
willcrichton Oct 1, 2021
5584c79
Update to latest rustc and rustdoc styles
willcrichton Oct 7, 2021
bb383ed
Move some expansion logic into generation-time, fix section header li…
willcrichton Oct 7, 2021
8b141a2
Add variance constraints for const params
voidc Oct 7, 2021
e22e858
Move more scrape-examples logic from JS to rust
willcrichton Oct 7, 2021
f10dcee
Change handling of spans in scrape examples, add test for highlight d…
willcrichton Oct 8, 2021
9e4958a
Add test for prev/back arrows + examples across multiple files
willcrichton Oct 9, 2021
b1616f3
Add test for ordering of examples, simplify with single scrape.mk file
willcrichton Oct 9, 2021
59b36bc
Add UI test for the variance of types appearing in consts
voidc Oct 12, 2021
a400f10
Bless tests
voidc Oct 12, 2021
24a71cb
Fix local crate not being scraped
willcrichton Oct 13, 2021
02e4d0b
Make all proc-macro back-compat lints deny-by-default
Aaron1011 Aug 15, 2021
8f80d86
Small scrape-example fixes
willcrichton Oct 20, 2021
d1c29c6
Revert def_id addition from clean::Function, add test for
willcrichton Oct 22, 2021
fd5d614
Move def_id logic into render_call_locations
willcrichton Oct 22, 2021
e4aeeca
Reset qualifs when a storage of a local ends
tmiasko Oct 22, 2021
fd25491
Add caveat about changing parallelism and function call overhead
the8472 Oct 23, 2021
dcf9242
Rollup merge of #85833 - willcrichton:example-analyzer, r=jyn514
matthiaskrgr Oct 23, 2021
17c602d
Rollup merge of #88041 - Aaron1011:deny-proc-macro-hack, r=wesleywiser
matthiaskrgr Oct 23, 2021
2b874f0
Rollup merge of #89829 - voidc:assoc-const-variance, r=lcnr
matthiaskrgr Oct 23, 2021
74a0c49
Rollup merge of #90168 - tmiasko:const-qualif-storage, r=matthewjasper
matthiaskrgr Oct 23, 2021
1680359
Rollup merge of #90198 - the8472:available-parallelism-runtime, r=jos…
matthiaskrgr Oct 23, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion compiler/rustc_const_eval/src/transform/check_consts/resolver.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

use rustc_index::bit_set::BitSet;
use rustc_middle::mir::visit::Visitor;
use rustc_middle::mir::{self, BasicBlock, Local, Location};
use rustc_middle::mir::{self, BasicBlock, Local, Location, Statement, StatementKind};

use std::marker::PhantomData;

Expand Down Expand Up @@ -120,6 +120,15 @@ where
self.super_assign(place, rvalue, location);
}

fn visit_statement(&mut self, statement: &Statement<'tcx>, location: Location) {
match statement.kind {
StatementKind::StorageDead(local) => {
self.qualifs_per_local.remove(local);
}
_ => self.super_statement(statement, location),
}
}

fn visit_terminator(&mut self, terminator: &mir::Terminator<'tcx>, location: Location) {
// The effect of assignment to the return place in `TerminatorKind::Call` is not applied
// here; that occurs in `apply_call_return_effect`.
Expand Down
4 changes: 2 additions & 2 deletions compiler/rustc_lint_defs/src/builtin.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1957,7 +1957,7 @@ declare_lint! {
/// [issue #50504]: https://github.com/rust-lang/rust/issues/50504
/// [future-incompatible]: ../index.md#future-incompatible-lints
pub PROC_MACRO_DERIVE_RESOLUTION_FALLBACK,
Warn,
Deny,
"detects proc macro derives using inaccessible names from parent modules",
@future_incompatible = FutureIncompatibleInfo {
reference: "issue #83583 <https://github.com/rust-lang/rust/issues/83583>",
Expand Down Expand Up @@ -3253,7 +3253,7 @@ declare_lint! {
/// [issue #83125]: https://github.com/rust-lang/rust/issues/83125
/// [future-incompatible]: ../index.md#future-incompatible-lints
pub PROC_MACRO_BACK_COMPAT,
Warn,
Deny,
"detects usage of old versions of certain proc-macro crates",
@future_incompatible = FutureIncompatibleInfo {
reference: "issue #83125 <https://github.com/rust-lang/rust/issues/83125>",
Expand Down
30 changes: 25 additions & 5 deletions compiler/rustc_typeck/src/variance/constraints.rs
Original file line number Diff line number Diff line change
Expand Up @@ -223,8 +223,8 @@ impl<'a, 'tcx> ConstraintContext<'a, 'tcx> {
self.add_constraints_from_region(current, lt, variance_i)
}
GenericArgKind::Type(ty) => self.add_constraints_from_ty(current, ty, variance_i),
GenericArgKind::Const(_) => {
// Consts impose no constraints.
GenericArgKind::Const(val) => {
self.add_constraints_from_const(current, val, variance_i)
}
}
}
Expand Down Expand Up @@ -263,7 +263,8 @@ impl<'a, 'tcx> ConstraintContext<'a, 'tcx> {
self.add_constraints_from_mt(current, &ty::TypeAndMut { ty, mutbl }, variance);
}

ty::Array(typ, _) => {
ty::Array(typ, len) => {
self.add_constraints_from_const(current, len, variance);
self.add_constraints_from_ty(current, typ, variance);
}

Expand Down Expand Up @@ -385,13 +386,32 @@ impl<'a, 'tcx> ConstraintContext<'a, 'tcx> {
self.add_constraints_from_region(current, lt, variance_i)
}
GenericArgKind::Type(ty) => self.add_constraints_from_ty(current, ty, variance_i),
GenericArgKind::Const(_) => {
// Consts impose no constraints.
GenericArgKind::Const(val) => {
self.add_constraints_from_const(current, val, variance)
}
}
}
}

/// Adds constraints appropriate for a const expression `val`
/// in a context with ambient variance `variance`
fn add_constraints_from_const(
&mut self,
current: &CurrentItem,
val: &ty::Const<'tcx>,
variance: VarianceTermPtr<'a>,
) {
debug!("add_constraints_from_const(val={:?}, variance={:?})", val, variance);

match &val.val {
ty::ConstKind::Unevaluated(uv) => {
let substs = uv.substs(self.tcx());
self.add_constraints_from_invariant_substs(current, substs, variance);
}
_ => {}
}
}

/// Adds constraints appropriate for a function with signature
/// `sig` appearing in a context with ambient variance `variance`
fn add_constraints_from_sig(
Expand Down
4 changes: 4 additions & 0 deletions library/std/src/thread/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1449,6 +1449,10 @@ fn _assert_sync_and_send() {
/// global state in order to more accurately query the amount of available
/// parallelism.
///
/// Resource limits can be changed during the runtime of a program, therefore the value is
/// not cached and instead recomputed every time this function is called. It should not be
/// called from hot code.
///
/// The value returned by this function should be considered a simplified
/// approximation of the actual amount of parallelism available at any given
/// time. To get a more detailed or precise overview of the amount of
Expand Down
24 changes: 24 additions & 0 deletions src/doc/rustdoc/src/unstable-features.md
Original file line number Diff line number Diff line change
Expand Up @@ -455,3 +455,27 @@ Calculating code examples follows these rules:
* static
* typedef
2. If one of the previously listed items has a code example, then it'll be counted.

### `--with-examples`: include examples of uses of items as documentation

This option, combined with `--scrape-examples-target-crate` and
`--scrape-examples-output-path`, is used to implement the functionality in [RFC
#3123](https://github.com/rust-lang/rfcs/pull/3123). Uses of an item (currently
functions / call-sites) are found in a crate and its reverse-dependencies, and
then the uses are included as documentation for that item. This feature is
intended to be used via `cargo doc --scrape-examples`, but the rustdoc-only
workflow looks like:

```bash
$ rustdoc examples/ex.rs -Z unstable-options \
--extern foobar=target/deps/libfoobar.rmeta \
--scrape-examples-target-crate foobar \
--scrape-examples-output-path output.calls
$ rustdoc src/lib.rs -Z unstable-options --with-examples output.calls
```

First, the library must be checked to generate an `rmeta`. Then a
reverse-dependency like `examples/ex.rs` is given to rustdoc with the target
crate being documented (`foobar`) and a path to output the calls
(`output.calls`). Then, the generated calls file can be passed via
`--with-examples` to the subsequent documentation of `foobar`.
3 changes: 2 additions & 1 deletion src/librustdoc/clean/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2062,7 +2062,8 @@ fn clean_use_statement(
impl Clean<Item> for (&hir::ForeignItem<'_>, Option<Symbol>) {
fn clean(&self, cx: &mut DocContext<'_>) -> Item {
let (item, renamed) = self;
cx.with_param_env(item.def_id.to_def_id(), |cx| {
let def_id = item.def_id.to_def_id();
cx.with_param_env(def_id, |cx| {
let kind = match item.kind {
hir::ForeignItemKind::Fn(ref decl, ref names, ref generics) => {
let abi = cx.tcx.hir().get_foreign_abi(item.hir_id());
Expand Down
13 changes: 13 additions & 0 deletions src/librustdoc/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ use crate::html::render::StylePath;
use crate::html::static_files;
use crate::opts;
use crate::passes::{self, Condition, DefaultPassOption};
use crate::scrape_examples::{AllCallLocations, ScrapeExamplesOptions};
use crate::theme;

#[derive(Clone, Copy, PartialEq, Eq, Debug)]
Expand Down Expand Up @@ -158,6 +159,10 @@ crate struct Options {
crate json_unused_externs: bool,
/// Whether to skip capturing stdout and stderr of tests.
crate nocapture: bool,

/// Configuration for scraping examples from the current crate. If this option is Some(..) then
/// the compiler will scrape examples and not generate documentation.
crate scrape_examples_options: Option<ScrapeExamplesOptions>,
}

impl fmt::Debug for Options {
Expand Down Expand Up @@ -202,6 +207,7 @@ impl fmt::Debug for Options {
.field("run_check", &self.run_check)
.field("no_run", &self.no_run)
.field("nocapture", &self.nocapture)
.field("scrape_examples_options", &self.scrape_examples_options)
.finish()
}
}
Expand Down Expand Up @@ -280,6 +286,7 @@ crate struct RenderOptions {
crate emit: Vec<EmitType>,
/// If `true`, HTML source pages will generate links for items to their definition.
crate generate_link_to_definition: bool,
crate call_locations: AllCallLocations,
}

#[derive(Copy, Clone, Debug, PartialEq, Eq)]
Expand Down Expand Up @@ -671,6 +678,10 @@ impl Options {
return Err(1);
}

let scrape_examples_options = ScrapeExamplesOptions::new(&matches, &diag)?;
let with_examples = matches.opt_strs("with-examples");
let call_locations = crate::scrape_examples::load_call_locations(with_examples, &diag)?;

let (lint_opts, describe_lints, lint_cap) = get_cmd_lint_options(matches, error_format);

Ok(Options {
Expand Down Expand Up @@ -737,10 +748,12 @@ impl Options {
),
emit,
generate_link_to_definition,
call_locations,
},
crate_name,
output_format,
json_unused_externs,
scrape_examples_options,
})
}

Expand Down
75 changes: 64 additions & 11 deletions src/librustdoc/html/highlight.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ use crate::html::render::Context;
use std::collections::VecDeque;
use std::fmt::{Display, Write};

use rustc_data_structures::fx::FxHashMap;
use rustc_lexer::{LiteralKind, TokenKind};
use rustc_span::edition::Edition;
use rustc_span::symbol::Symbol;
Expand All @@ -30,6 +31,10 @@ crate struct ContextInfo<'a, 'b, 'c> {
crate root_path: &'c str,
}

/// Decorations are represented as a map from CSS class to vector of character ranges.
/// Each range will be wrapped in a span with that class.
crate struct DecorationInfo(crate FxHashMap<&'static str, Vec<(u32, u32)>>);

/// Highlights `src`, returning the HTML output.
crate fn render_with_highlighting(
src: &str,
Expand All @@ -40,6 +45,7 @@ crate fn render_with_highlighting(
edition: Edition,
extra_content: Option<Buffer>,
context_info: Option<ContextInfo<'_, '_, '_>>,
decoration_info: Option<DecorationInfo>,
) {
debug!("highlighting: ================\n{}\n==============", src);
if let Some((edition_info, class)) = tooltip {
Expand All @@ -56,7 +62,7 @@ crate fn render_with_highlighting(
}

write_header(out, class, extra_content);
write_code(out, &src, edition, context_info);
write_code(out, &src, edition, context_info, decoration_info);
write_footer(out, playground_button);
}

Expand Down Expand Up @@ -89,17 +95,23 @@ fn write_code(
src: &str,
edition: Edition,
context_info: Option<ContextInfo<'_, '_, '_>>,
decoration_info: Option<DecorationInfo>,
) {
// This replace allows to fix how the code source with DOS backline characters is displayed.
let src = src.replace("\r\n", "\n");
Classifier::new(&src, edition, context_info.as_ref().map(|c| c.file_span).unwrap_or(DUMMY_SP))
.highlight(&mut |highlight| {
match highlight {
Highlight::Token { text, class } => string(out, Escape(text), class, &context_info),
Highlight::EnterSpan { class } => enter_span(out, class),
Highlight::ExitSpan => exit_span(out),
};
});
Classifier::new(
&src,
edition,
context_info.as_ref().map(|c| c.file_span).unwrap_or(DUMMY_SP),
decoration_info,
)
.highlight(&mut |highlight| {
match highlight {
Highlight::Token { text, class } => string(out, Escape(text), class, &context_info),
Highlight::EnterSpan { class } => enter_span(out, class),
Highlight::ExitSpan => exit_span(out),
};
});
}

fn write_footer(out: &mut Buffer, playground_button: Option<&str>) {
Expand Down Expand Up @@ -127,6 +139,7 @@ enum Class {
PreludeTy,
PreludeVal,
QuestionMark,
Decoration(&'static str),
}

impl Class {
Expand All @@ -150,6 +163,7 @@ impl Class {
Class::PreludeTy => "prelude-ty",
Class::PreludeVal => "prelude-val",
Class::QuestionMark => "question-mark",
Class::Decoration(kind) => kind,
}
}

Expand Down Expand Up @@ -248,6 +262,24 @@ impl Iterator for PeekIter<'a> {
}
}

/// Custom spans inserted into the source. Eg --scrape-examples uses this to highlight function calls
struct Decorations {
starts: Vec<(u32, &'static str)>,
ends: Vec<u32>,
}

impl Decorations {
fn new(info: DecorationInfo) -> Self {
let (starts, ends) = info
.0
.into_iter()
.map(|(kind, ranges)| ranges.into_iter().map(move |(lo, hi)| ((lo, kind), hi)))
.flatten()
.unzip();
Decorations { starts, ends }
}
}

/// Processes program tokens, classifying strings of text by highlighting
/// category (`Class`).
struct Classifier<'a> {
Expand All @@ -259,13 +291,20 @@ struct Classifier<'a> {
byte_pos: u32,
file_span: Span,
src: &'a str,
decorations: Option<Decorations>,
}

impl<'a> Classifier<'a> {
/// Takes as argument the source code to HTML-ify, the rust edition to use and the source code
/// file span which will be used later on by the `span_correspondance_map`.
fn new(src: &str, edition: Edition, file_span: Span) -> Classifier<'_> {
fn new(
src: &str,
edition: Edition,
file_span: Span,
decoration_info: Option<DecorationInfo>,
) -> Classifier<'_> {
let tokens = PeekIter::new(TokenIter { src });
let decorations = decoration_info.map(Decorations::new);
Classifier {
tokens,
in_attribute: false,
Expand All @@ -275,6 +314,7 @@ impl<'a> Classifier<'a> {
byte_pos: 0,
file_span,
src,
decorations,
}
}

Expand Down Expand Up @@ -356,6 +396,19 @@ impl<'a> Classifier<'a> {
/// token is used.
fn highlight(mut self, sink: &mut dyn FnMut(Highlight<'a>)) {
loop {
if let Some(decs) = self.decorations.as_mut() {
let byte_pos = self.byte_pos;
let n_starts = decs.starts.iter().filter(|(i, _)| byte_pos >= *i).count();
for (_, kind) in decs.starts.drain(0..n_starts) {
sink(Highlight::EnterSpan { class: Class::Decoration(kind) });
}

let n_ends = decs.ends.iter().filter(|i| byte_pos >= **i).count();
for _ in decs.ends.drain(0..n_ends) {
sink(Highlight::ExitSpan);
}
}

if self
.tokens
.peek()
Expand Down Expand Up @@ -657,7 +710,7 @@ fn string<T: Display>(
// https://github.com/rust-lang/rust/blob/60f1a2fc4b535ead9c85ce085fdce49b1b097531/src/librustdoc/html/render/context.rs#L315-L338
match href {
LinkFromSrc::Local(span) => context
.href_from_span(*span)
.href_from_span(*span, true)
.map(|s| format!("{}{}", context_info.root_path, s)),
LinkFromSrc::External(def_id) => {
format::href_with_root_path(*def_id, context, Some(context_info.root_path))
Expand Down
2 changes: 2 additions & 0 deletions src/librustdoc/html/highlight/fixtures/decorations.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
<span class="example"><span class="kw">let</span> <span class="ident">x</span> <span class="op">=</span> <span class="number">1</span>;</span>
<span class="kw">let</span> <span class="ident">y</span> <span class="op">=</span> <span class="number">2</span>;
Loading