Rollup merge of rust-lang#40554 - nrc:rls-data, r=alexcrichton

Use rls-data crate

This basically pulls out a bunch of data structures used by save-analysis for serialization into an external crate, and pulls that crate in using Rustbuild. The RLS can then share these data structures with the compiler which in some cases will allow more efficient communication between the compiler and the RLS (i.e., without serialisation).

Along the way, I have to pull in rls-span, which is the RLS's way of defining spans (more type-safe than the compiler's built-in way). This is basically just to convert from compiler spans to RLS spans.

I also pull in the crates.io version of rustc-serialize, which is a bit annoying, but seems to be the only way to have serialisable data in an external crate. To make this work, all of the save-analysis crate has to use this version too (cc rust-lang#40527).

Finally I pull in a line from rust-lang#40347 to make the unstable crate checking stuff working.

There are a lot of changes to save-analysis but they are all mechanical and trivial - changing from using `From` to `Into` (because of orphan rules) being the main thing.

r? @alexcrichton

src/bootstrap/bin/rustc.rs

@@ -94,6 +94,13 @@ fn main() {
             cmd.arg("-Cprefer-dynamic");
         }
 
+        // Pass the `rustbuild` feature flag to crates which rustbuild is
+        // building. See the comment in bootstrap/lib.rs where this env var is
+        // set for more details.
+        if env::var_os("RUSTBUILD_UNSTABLE").is_some() {
+            cmd.arg("--cfg").arg("rustbuild");
+        }
+
         // Help the libc crate compile by assisting it in finding the MUSL
         // native libraries.
         if let Some(s) = env::var_os("MUSL_ROOT") {

src/bootstrap/lib.rs

@@ -180,7 +180,7 @@ struct Crate {
 ///
 /// These entries currently correspond to the various output directories of the
 /// build system, with each mod generating output in a different directory.
-#[derive(Clone, Copy)]
+#[derive(Clone, Copy, PartialEq, Eq)]
 pub enum Mode {
     /// This cargo is going to build the standard library, placing output in the
     /// "stageN-std" directory.
@@ -491,14 +491,35 @@ impl Build {
         // For other crates, however, we know that we've already got a standard
         // library up and running, so we can use the normal compiler to compile
         // build scripts in that situation.
-        if let Mode::Libstd = mode {
+        if mode == Mode::Libstd {
             cargo.env("RUSTC_SNAPSHOT", &self.rustc)
                  .env("RUSTC_SNAPSHOT_LIBDIR", self.rustc_snapshot_libdir());
         } else {
             cargo.env("RUSTC_SNAPSHOT", self.compiler_path(compiler))
                  .env("RUSTC_SNAPSHOT_LIBDIR", self.rustc_libdir(compiler));
         }
 
+        // There are two invariants we try must maintain:
+        // * stable crates cannot depend on unstable crates (general Rust rule),
+        // * crates that end up in the sysroot must be unstable (rustbuild rule).
+        //
+        // In order to do enforce the latter, we pass the env var
+        // `RUSTBUILD_UNSTABLE` down the line for any crates which will end up
+        // in the sysroot. We read this in bootstrap/bin/rustc.rs and if it is
+        // set, then we pass the `rustbuild` feature to rustc when building the
+        // the crate.
+        //
+        // In turn, crates that can be used here should recognise the `rustbuild`
+        // feature and opt-in to `rustc_private`.
+        //
+        // We can't always pass `rustbuild` because crates which are outside of
+        // the comipiler, libs, and tests are stable and we don't want to make
+        // their deps unstable (since this would break the first invariant
+        // above).
+        if mode != Mode::Tool {
+            cargo.env("RUSTBUILD_UNSTABLE", "1");
+        }
+
         // Ignore incremental modes except for stage0, since we're
         // not guaranteeing correctness acros builds if the compiler
         // is changing under your feet.`

src/librustc_save_analysis/Cargo.toml

@@ -12,5 +12,8 @@ crate-type = ["dylib"]
 log = { path = "../liblog" }
 rustc = { path = "../librustc" }
 syntax = { path = "../libsyntax" }
-serialize = { path = "../libserialize" }
-syntax_pos = { path = "../libsyntax_pos" }
+syntax_pos = { path = "../libsyntax_pos" }
+rls-data = "0.1"
+rls-span = "0.1"
+# FIXME(#40527) should move rustc serialize out of tree
+rustc-serialize = "0.3"

src/librustc_save_analysis/csv_dumper.rs

@@ -13,6 +13,8 @@ use std::io::Write;
 use super::external_data::*;
 use super::dump::Dump;
 
+use rls_data::{SpanData, CratePreludeData};
+
 pub struct CsvDumper<'b, W: 'b> {
     output: &'b mut W
 }
@@ -429,6 +431,6 @@ fn make_values_str(pairs: &[(&'static str, &str)]) -> String {
 fn span_extent_str(span: SpanData) -> String {
     format!("file_name,\"{}\",file_line,{},file_col,{},byte_start,{},\
              file_line_end,{},file_col_end,{},byte_end,{}",
-             span.file_name, span.line_start, span.column_start, span.byte_start,
-             span.line_end, span.column_end, span.byte_end)
+             span.file_name.to_str().unwrap(), span.line_start.0, span.column_start.0,
+             span.byte_start, span.line_end.0, span.column_end.0, span.byte_end)
 }