Implement `-Z embed-source` (DWARFv5 source code embedding extension) #126985

Mrmaxmeier · 2024-06-26T09:34:38Z

Implement rust-lang/compiler-team#764 MCP which adds an unstable flag that exposes LLVM's DWARFv5 source code embedding support.

rustbot · 2024-06-26T09:34:46Z

r? @TaKO8Ki

rustbot has assigned @TaKO8Ki.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

compiler/rustc_session/src/options.rs

src/doc/rustc/src/codegen-options/index.md

compiler/rustc_codegen_llvm/src/debuginfo/metadata.rs

tgross35 · 2024-06-26T10:49:27Z

This should also get a test in tests/debuginfo/ to make sure it is doing what is expected

compiler/rustc_session/messages.ftl

src/doc/unstable-book/src/compiler-flags/embed-source.md

compiler/rustc_session/messages.ftl

Mrmaxmeier · 2024-06-26T13:45:12Z

This should also get a test in tests/debuginfo/ to make sure it is doing what is expected

~~Both LLDB and~~ GDB doesn't support this proposal yet, so there's not a great way to test for it. I guess something similar to the llvm-dwarfdump-based tests in tests/run-make/remap-path-prefix-dwarf might be the way to go.

Edit: Looks like support was added somewhat recently in LLDB, but looking at the DWARF is probably still easier. I didn't find a (non-horrible) way to differentiate between embedded source code and code loaded from disk. (settings set target.source-map will fall back to the original paths if the mapped ones don't exist)

tgross35 · 2024-06-26T17:37:33Z

A run-make test would be great. remap-path-prefix-dwarf still uses a Makefile, note that you should probably use rmake if you are comfortable with it (the tests are migrating away from makefiles to having a rmake.rs instead. You get these crates to do your inspection https://github.com/rust-lang/rust/blob/d7c59370cea68cd17006ec3440a43254fd0eda7d/src/tools/run-make-support/Cargo.toml)

Mrmaxmeier · 2024-06-28T11:51:48Z

I've added a run-make test that depends on a new release of gimli which includes support for the relevant DWARF tags.
I'll update the draft PR once the relevant update is available on crates.io. This PR will fail tidy with invalid source: "git+https://github.com/gimli-rs/[...] in the mean time.

tgross35 · 2024-06-28T11:57:01Z

You can probably ask for a minor release on your Gimli PR, then open a separate PR to bump the versions in this repo once it is released.

run-make-support: update gimli to 0.31.0 This version bump is required for the tests in rust-lang#126985 as suggested [here](rust-lang#126985 (comment)). r? tgross35 (^ that didn't work. cc `@tgross35)` try-job: x86_64-msvc try-job: x86_64-mingw try-job: aarch64-apple try-job: test-various try-job: armhf-gnu try-job: dist-x86_64-linux

…jieyouxu run-make-support: update gimli to 0.31.0 This version bump is required for the tests in rust-lang#126985 as suggested [here](rust-lang#126985 (comment)). r? tgross35 (^ that didn't work. cc `@tgross35)` try-job: x86_64-msvc try-job: x86_64-mingw try-job: aarch64-apple try-job: test-various try-job: armhf-gnu try-job: dist-x86_64-linux

Rollup merge of rust-lang#127805 - Mrmaxmeier:run-make-bump-gimli, r=jieyouxu run-make-support: update gimli to 0.31.0 This version bump is required for the tests in rust-lang#126985 as suggested [here](rust-lang#126985 (comment)). r? tgross35 (^ that didn't work. cc `@tgross35)` try-job: x86_64-msvc try-job: x86_64-mingw try-job: aarch64-apple try-job: test-various try-job: armhf-gnu try-job: dist-x86_64-linux

Mrmaxmeier · 2024-07-17T08:51:55Z

Status update:
The MCP got accepted (cc @davidtwco) and I've added a run-make test for the new functionality.

It turns out that LLVM 17 is not compatible with the way I'm emitting source code with each file. The proposed DWARF v6 spec was only implemented somewhat recently (llvm/llvm-project@87e22bd).

There does not seem to be an easy way to gate -Zembed-source to LLVM 18+, and I'm tempted to just ignore the issue. (llvm_util::get_version is not exposed outside of rustc_codegen_llvm and I'd like to not plumb it though the target features functionality.)

Is rustc going to be used with LLVM 17 for nightly builds anywhere? (I guess people might be able to run into this with RUSTC_BOOTSTRAP=1 on distro builds?)