Auto merge of #45368 - kennytm:rollup, r=kennytm

Rollup of 10 pull requests

- Successful merges: #44138, #45082, #45098, #45181, #45217, #45281, #45325, #45326, #45340, #45354
- Failed merges:

.travis.yml

@@ -12,7 +12,7 @@ matrix:
   fast_finish: true
   include:
     # Images used in testing PR and try-build should be run first.
-    - env: IMAGE=x86_64-gnu-llvm-3.7 RUST_BACKTRACE=1
+    - env: IMAGE=x86_64-gnu-llvm-3.9 RUST_BACKTRACE=1
       if: type = pull_request OR branch = auto
 
     - env: IMAGE=dist-x86_64-linux DEPLOY=1

CONTRIBUTING.md

@@ -365,6 +365,116 @@ In order to prepare your PR, you can run the build locally by doing
 there, you may wish to set `submodules = false` in the `config.toml`
 to prevent `x.py` from resetting to the original branch.
 
+#### Breaking Tools Built With The Compiler
+[breaking-tools-built-with-the-compiler]: #breaking-tools-built-with-the-compiler
+
+Rust's build system builds a number of tools that make use of the
+internals of the compiler. This includes clippy,
+[RLS](https://github.com/rust-lang-nursery/rls) and
+[rustfmt](https://github.com/rust-lang-nursery/rustfmt). If these tools
+break because of your changes, you may run into a sort of "chicken and egg"
+problem. These tools rely on the latest compiler to be built so you can't update
+them to reflect your changes to the compiler until those changes are merged into
+the compiler. At the same time, you can't get your changes merged into the compiler
+because the rust-lang/rust build won't pass until those tools build and pass their
+tests.
+
+That means that, in the default state, you can't update the compiler without first
+fixing rustfmt, rls and the other tools that the compiler builds.
+
+Luckily, a feature was [added to Rust's build](https://github.com/rust-lang/rust/pull/45243)
+to make all of this easy to handle. The idea is that you mark the tools as "broken",
+so that the rust-lang/rust build passes without trying to build them, then land the change
+in the compiler, wait for a nightly, and go update the tools that you broke. Once you're done
+and the tools are working again, you go back in the compiler and change the tools back
+from "broken".
+
+This should avoid a bunch of synchronization dances and is also much easier on contributors as
+there's no need to block on rls/rustfmt/other tools changes going upstream.
+
+Here are those same steps in detail:
+
+1. (optional) First, if it doesn't exist already, create a `config.toml` by copying
+   `config.toml.example` in the root directory of the Rust repository.
+   Set `submodules = false` in the `[build]` section. This will prevent `x.py`
+   from resetting to the original branch after you make your changes. If you
+   need to [update any submodules to their latest versions][updating-submodules],
+   see the section of this file about that for more information.
+2. (optional) Run `./x.py test src/tools/rustfmt` (substituting the submodule
+   that broke for `rustfmt`). Fix any errors in the submodule (and possibly others).
+3. (optional) Make commits for your changes and send them to upstream repositories as a PR.
+4. (optional) Maintainers of these submodules will **not** merge the PR. The PR can't be
+   merged because CI will be broken. You'll want to write a message on the PR referencing
+   your change, and how the PR should be merged once your change makes it into a nightly.
+5. Update `src/tools/toolstate.toml` to indicate that the tool in question is "broken",
+   that will disable building it on CI. See the documentation in that file for the exact
+   configuration values you can use.
+6. Commit the changes to `src/tools/toolstate.toml`, **do not update submodules in your commit**,
+   and then update the PR you have for rust-lang/rust.
+7. Wait for your PR to merge.
+8. Wait for a nightly
+9. (optional) Help land your PR on the upstream repository now that your changes are in nightly.
+10. (optional) Send a PR to rust-lang/rust updating the submodule, reverting `src/tools/toolstate.toml` back to a "building" or "testing" state.
+
+#### Updating submodules
+[updating-submodules]: #updating-submodules
+
+These instructions are specific to updating `rustfmt`, however they may apply
+to the other submodules as well. Please help by improving these instructions
+if you find any discrepencies or special cases that need to be addressed.
+
+To update the `rustfmt` submodule, start by running the appropriate
+[`git submodule` command](https://git-scm.com/book/en/v2/Git-Tools-Submodules).
+For example, to update to the latest commit on the remote master branch,
+you may want to run:
+```
+git submodule update --remote src/tools/rustfmt
+```
+If you run `./x.py build` now, and you are lucky, it may just work. If you see
+an error message about patches that did not resolve to any crates, you will need
+to complete a few more steps which are outlined with their rationale below.
+
+*(This error may change in the future to include more information.)*
+```
+error: failed to resolve patches for `https://github.com/rust-lang-nursery/rustfmt`
+
+Caused by:
+  patch for `rustfmt-nightly` in `https://github.com/rust-lang-nursery/rustfmt` did not resolve to any crates
+failed to run: ~/rust/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path ~/rust/src/bootstrap/Cargo.toml
+```
+
+If you haven't used the `[patch]`
+section of `Cargo.toml` before, there is [some relevant documentation about it
+in the cargo docs](http://doc.crates.io/manifest.html#the-patch-section). In
+addition to that, you should read the 
+[Overriding dependencies](http://doc.crates.io/specifying-dependencies.html#overriding-dependencies)
+section of the documentation as well.
+
+Specifically, the following [section in Overriding dependencies](http://doc.crates.io/specifying-dependencies.html#testing-a-bugfix) reveals what the problem is:
+
+> Next up we need to ensure that our lock file is updated to use this new version of uuid so our project uses the locally checked out copy instead of one from crates.io. The way [patch] works is that it'll load the dependency at ../path/to/uuid and then whenever crates.io is queried for versions of uuid it'll also return the local version.
+> 
+> This means that the version number of the local checkout is significant and will affect whether the patch is used. Our manifest declared uuid = "1.0" which means we'll only resolve to >= 1.0.0, < 2.0.0, and Cargo's greedy resolution algorithm also means that we'll resolve to the maximum version within that range. Typically this doesn't matter as the version of the git repository will already be greater or match the maximum version published on crates.io, but it's important to keep this in mind!
+
+This says that when we updated the submodule, the version number in our
+`src/tools/rustfmt/Cargo.toml` changed. The new version is different from
+the version in `Cargo.lock`, so the build can no longer continue.
+
+To resolve this, we need to update `Cargo.lock`. Luckily, cargo provides a
+command to do this easily.
+
+First, go into the `src/` directory since that is where `Cargo.toml` is in
+the rust repository. Then run, `cargo update -p rustfmt-nightly` to solve
+the problem.
+
+```
+$ cd src
+$ cargo update -p rustfmt-nightly
+```
+
+This should change the version listed in `src/Cargo.lock` to the new version you updated
+the submodule to. Running `./x.py build` should work now.
+
 ## Writing Documentation
 [writing-documentation]: #writing-documentation
 

config.toml.example

@@ -35,7 +35,7 @@
 # If an external LLVM root is specified, we automatically check the version by
 # default to make sure it's within the range that we're expecting, but setting
 # this flag will indicate that this version check should not be done.
-#version-check = false
+#version-check = true
 
 # Link libstdc++ statically into the librustc_llvm instead of relying on a
 # dynamic version to be available.

src/bootstrap/config.rs

@@ -299,6 +299,7 @@ impl Config {
         let mut config = Config::default();
         config.llvm_enabled = true;
         config.llvm_optimize = true;
+        config.llvm_version_check = true;
         config.use_jemalloc = true;
         config.backtrace = true;
         config.rust_optimize = true;

src/bootstrap/native.rs

@@ -259,11 +259,14 @@ fn check_llvm_version(build: &Build, llvm_config: &Path) {
 
     let mut cmd = Command::new(llvm_config);
     let version = output(cmd.arg("--version"));
-    if version.starts_with("3.5") || version.starts_with("3.6") ||
-       version.starts_with("3.7") {
-        return
+    let mut parts = version.split('.').take(2)
+        .filter_map(|s| s.parse::<u32>().ok());
+    if let (Some(major), Some(minor)) = (parts.next(), parts.next()) {
+        if major > 3 || (major == 3 && minor >= 9) {
+            return
+        }
     }
-    panic!("\n\nbad LLVM version: {}, need >=3.5\n\n", version)
+    panic!("\n\nbad LLVM version: {}, need >=3.9\n\n", version)
 }
 
 #[derive(Debug, Copy, Clone, PartialEq, Eq, Hash)]

src/ci/docker/x86_64-gnu-llvm-3.7/Dockerfile → src/ci/docker/x86_64-gnu-llvm-3.9/Dockerfile

@@ -11,15 +11,17 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
   cmake \
   sudo \
   gdb \
-  llvm-3.7-tools \
+  llvm-3.9-tools \
   libedit-dev \
   zlib1g-dev \
   xz-utils
 
 COPY scripts/sccache.sh /scripts/
 RUN sh /scripts/sccache.sh
 
+# using llvm-link-shared due to libffi issues -- see #34486
 ENV RUST_CONFIGURE_ARGS \
       --build=x86_64-unknown-linux-gnu \
-      --llvm-root=/usr/lib/llvm-3.7
+      --llvm-root=/usr/lib/llvm-3.9 \
+      --enable-llvm-link-shared
 ENV RUST_CHECK_TARGET check

src/doc/unstable-book/src/language-features/lang-items.md

@@ -227,3 +227,95 @@ A third function, `rust_eh_unwind_resume`, is also needed if the `custom_unwind_
 flag is set in the options of the compilation target. It allows customizing the
 process of resuming unwind at the end of the landing pads. The language item's name
 is `eh_unwind_resume`.
+
+## List of all language items
+
+This is a list of all language items in Rust along with where they are located in
+the source code.
+
+- Primitives
+  - `i8`: `libcore/num/mod.rs`
+  - `i16`: `libcore/num/mod.rs`
+  - `i32`: `libcore/num/mod.rs`
+  - `i64`: `libcore/num/mod.rs`
+  - `i128`: `libcore/num/mod.rs`
+  - `isize`: `libcore/num/mod.rs`
+  - `u8`: `libcore/num/mod.rs`
+  - `u16`: `libcore/num/mod.rs`
+  - `u32`: `libcore/num/mod.rs`
+  - `u64`: `libcore/num/mod.rs`
+  - `u128`: `libcore/num/mod.rs`
+  - `usize`: `libcore/num/mod.rs`
+  - `f32`: `libstd/f32.rs`
+  - `f64`: `libstd/f64.rs`
+  - `char`: `libstd_unicode/char.rs`
+  - `slice`: `liballoc/slice.rs`
+  - `str`: `liballoc/str.rs`
+  - `const_ptr`: `libcore/ptr.rs`
+  - `mut_ptr`: `libcore/ptr.rs`
+  - `unsafe_cell`: `libcore/cell.rs`
+- Runtime
+  - `start`: `libstd/rt.rs`
+  - `eh_personality`: `libpanic_unwind/emcc.rs` (EMCC)
+  - `eh_personality`: `libpanic_unwind/seh64_gnu.rs` (SEH64 GNU)
+  - `eh_personality`: `libpanic_unwind/seh.rs` (SEH)
+  - `eh_unwind_resume`: `libpanic_unwind/seh64_gnu.rs` (SEH64 GNU)
+  - `eh_unwind_resume`: `libpanic_unwind/gcc.rs` (GCC)
+  - `msvc_try_filter`: `libpanic_unwind/seh.rs` (SEH)
+  - `panic`: `libcore/panicking.rs`
+  - `panic_bounds_check`: `libcore/panicking.rs`
+  - `panic_fmt`: `libcore/panicking.rs`
+  - `panic_fmt`: `libstd/panicking.rs`
+- Allocations
+  - `owned_box`: `liballoc/boxed.rs`
+  - `exchange_malloc`: `liballoc/heap.rs`
+  - `box_free`: `liballoc/heap.rs`
+- Operands
+  - `not`: `libcore/ops/bit.rs`
+  - `bitand`: `libcore/ops/bit.rs`
+  - `bitor`: `libcore/ops/bit.rs`
+  - `bitxor`: `libcore/ops/bit.rs`
+  - `shl`: `libcore/ops/bit.rs`
+  - `shr`: `libcore/ops/bit.rs`
+  - `bitand_assign`: `libcore/ops/bit.rs`
+  - `bitor_assign`: `libcore/ops/bit.rs`
+  - `bitxor_assign`: `libcore/ops/bit.rs`
+  - `shl_assign`: `libcore/ops/bit.rs`
+  - `shr_assign`: `libcore/ops/bit.rs`
+  - `deref`: `libcore/ops/deref.rs`
+  - `deref_mut`: `libcore/ops/deref.rs`
+  - `index`: `libcore/ops/index.rs`
+  - `index_mut`: `libcore/ops/index.rs`
+  - `add`: `libcore/ops/arith.rs`
+  - `sub`: `libcore/ops/arith.rs`
+  - `mul`: `libcore/ops/arith.rs`
+  - `div`: `libcore/ops/arith.rs`
+  - `rem`: `libcore/ops/arith.rs`
+  - `neg`: `libcore/ops/arith.rs`
+  - `add_assign`: `libcore/ops/arith.rs`
+  - `sub_assign`: `libcore/ops/arith.rs`
+  - `mul_assign`: `libcore/ops/arith.rs`
+  - `div_assign`: `libcore/ops/arith.rs`
+  - `rem_assign`: `libcore/ops/arith.rs`
+  - `eq`: `libcore/cmp.rs`
+  - `ord`: `libcore/cmp.rs`
+- Functions
+  - `fn`: `libcore/ops/function.rs`
+  - `fn_mut`: `libcore/ops/function.rs`
+  - `fn_once`: `libcore/ops/function.rs`
+  - `generator_state`: `libcore/ops/generator.rs`
+  - `generator`: `libcore/ops/generator.rs`
+- Other
+  - `coerce_unsized`: `libcore/ops/unsize.rs`
+  - `drop`: `libcore/ops/drop.rs`
+  - `drop_in_place`: `libcore/ptr.rs`
+  - `clone`: `libcore/clone.rs`
+  - `copy`: `libcore/marker.rs`
+  - `send`: `libcore/marker.rs`
+  - `sized`: `libcore/marker.rs`
+  - `unsize`: `libcore/marker.rs`
+  - `sync`: `libcore/marker.rs`
+  - `phantom_data`: `libcore/marker.rs`
+  - `freeze`: `libcore/marker.rs`
+  - `debug_trait`: `libcore/fmt/mod.rs`
+  - `non_zero`: `libcore/nonzero.rs`

src/doc/unstable-book/src/library-features/alloc-jemalloc.md

@@ -8,55 +8,6 @@ See also [`alloc_system`](library-features/alloc-system.html).
 
 ------------------------
 
-The compiler currently ships two default allocators: `alloc_system` and
-`alloc_jemalloc` (some targets don't have jemalloc, however). These allocators
-are normal Rust crates and contain an implementation of the routines to
-allocate and deallocate memory. The standard library is not compiled assuming
-either one, and the compiler will decide which allocator is in use at
-compile-time depending on the type of output artifact being produced.
-
-Binaries generated by the compiler will use `alloc_jemalloc` by default (where
-available). In this situation the compiler "controls the world" in the sense of
-it has power over the final link. Primarily this means that the allocator
-decision can be left up the compiler.
-
-Dynamic and static libraries, however, will use `alloc_system` by default. Here
-Rust is typically a 'guest' in another application or another world where it
-cannot authoritatively decide what allocator is in use. As a result it resorts
-back to the standard APIs (e.g. `malloc` and `free`) for acquiring and releasing
-memory.
-
-# Switching Allocators
-
-Although the compiler's default choices may work most of the time, it's often
-necessary to tweak certain aspects. Overriding the compiler's decision about
-which allocator is in use is done simply by linking to the desired allocator:
-
-```rust,no_run
-#![feature(alloc_system)]
-
-extern crate alloc_system;
-
-fn main() {
-    let a = Box::new(4); // Allocates from the system allocator.
-    println!("{}", a);
-}
-```
-
-In this example the binary generated will not link to jemalloc by default but
-instead use the system allocator. Conversely to generate a dynamic library which
-uses jemalloc by default one would write:
-
-```rust,ignore
-#![feature(alloc_jemalloc)]
-#![crate_type = "dylib"]
-
-extern crate alloc_jemalloc;
-
-pub fn foo() {
-    let a = Box::new(4); // Allocates from jemalloc.
-    println!("{}", a);
-}
-# fn main() {}
-```
+This feature has been replaced by [the `jemallocator` crate on crates.io.][jemallocator].
 
+[jemallocator]: https://crates.io/crates/jemallocator