Migrate rustc from rustc-serialize to serde

[bstrie]

Cargo is doing it ( rust-lang/cargo#3682 ). Crates.io is doing it ( rust-lang/crates.io#583 ). We want them to think we're cool, so we need to do it too.

[eddyb]

cc @rust-lang/compiler

This is possible, in theory, but it would only work for decoding/encoding JSON, not our cross-crate metadata (which have much crazier requirements than serde is willing, or should handle by default).

We also need to figure out plugin staging, and one proposal at the rustc dev sprint was to go full IPC.

[oli-obk]

cc @dtolnay Rust wants to be cool, too

[bstrie]

@eddyb Would it not make sense to implement a metadata-specific backend for Serde? What sort of requirements are we talking here?

[eddyb]

@bstrie It just doesn't add up. Serde excels in turning arbitrary "tokens" into Rust types, whereas our metadata requires both specializing over the decoder (so it can be seeked in the binary blob), and a lifetime-parametric context to allocate/intern certain resources into.

The metadata format is ours, and we want to experiment with a mode where the format actually matches the in-memory type layouts of the host architecture, for memory-mapping. Serde can't deal with the fact that your deserialization process is actually a relocating GC now.

[oli-obk]

This isn't the main point here, but do I understand this correctly: Rust metadata will only ever have a single (de-)serialization backend, so there's no use in the generic way serde handles backend agnostic (de-)serialization?

[eddyb]

@oli-obk Correct, if there's any flexibility, it wouldn't be anywhere near Serde's level or direction.

[bstrie]

@eddyb Fair enough. I admit that most of my concern is in deprecating rustc-serialize as a crate that's "intended" for community consumption (Crates.io's fourth-most downloaded crate of all time nooooooooooooo). Ideally I'd like to see it reduced to the level of all of the other thousand "librustc_foo" crates in src. Or hey, could we just fold it into librustc_metadata entirely? But I'm getting offtopic here...

[eddyb]

@bstrie No, the plan is to remove the uses of it from the compiler entirely and introduce a "swiss army knife" of reflection-ish tasks (not in the classical sense, more specific to the compiler) - see #36588.

[steveklabnik]

Triage update: not only has this not been done, but we're also not using the one from crates.io.

[eddyb]

Blocked on #49219.

[cuviper]

@eddyb With #49219 merged, I believe proc_macro is no longer a blocker, right?
If so, I'm interested to work on this.

[eddyb]

@cuviper I'd first need to dig out some of my old PRs, I remember having a plan.
But, most importantly, most of the compiler can't use serde, due to the need for a specific context during serialization/deserialization (see above - also, #36588).

[cuviper]

OK - the json part for target specs looks simple enough, at least.

[Xanewok]

@eddyb I'd love to offload you and help support serde in the compiler at least for JSON workloads, if you could help with mentoring me on this. I'd love to get librustc_save_analysis to use serde for save-analysis serialization (and hopefully to easily switch to binary serialization later on).