Support symbolicating zstd-compressed ELF sections

[workingjubilee]

This requires bumping backtrace to rust-lang/backtrace-rs@4f3acf7 which includes support for symbolicating from newer Android APK formats, as well as for using zstd-compressed ELF sections for debuginfo.

It also requires compiling ruzstd, a zstd decompressor written in Rust.

This can combine with rust-lld's support for zstd-compressed debuginfo for overall smaller debuginfo sections.

r? @ghost

[workingjubilee]

Need to know the impacts this has on the stats of our toolchain and binaries before we do this.

@bors try @rust-timer queue

[bors]

⌛ Trying commit 7cd48bd with merge cba0a0a...

[bors]

☀️ Try build successful - checks-actions
Build commit: cba0a0a (cba0a0ad1c06ec505c5d921b20b4eafad8a6159e)

[rust-timer]

Finished benchmarking commit (cba0a0a): comparison URL.

Overall result: ❌ regressions - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.8%	[0.2%, 5.9%]	21
Regressions ❌ (secondary)	1.9%	[0.2%, 15.9%]	58
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	2.8%	[0.2%, 5.9%]	21

Max RSS (memory usage)

Results (primary 1.2%, secondary 1.0%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.2%	[1.2%, 1.2%]	1
Regressions ❌ (secondary)	4.0%	[1.6%, 9.2%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.7%	[-4.5%, -1.5%]	5
All ❌✅ (primary)	1.2%	[1.2%, 1.2%]	1

Cycles

Results (primary 5.5%, secondary 8.5%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	5.5%	[5.4%, 5.8%]	4
Regressions ❌ (secondary)	8.5%	[2.4%, 16.9%]	3
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	5.5%	[5.4%, 5.8%]	4

Binary size

Results (primary 6.0%, secondary 13.9%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	6.0%	[1.0%, 20.2%]	32
Regressions ❌ (secondary)	13.9%	[0.1%, 21.3%]	84
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	6.0%	[1.0%, 20.2%]	32

Bootstrap: 758.794s -> 759.241s (0.06%)
Artifact size: 341.06 MiB -> 342.18 MiB (0.33%)

[workingjubilee]

Mmyeah that's pretty brutal.

[workingjubilee]

@khuey Do you think we can get away with enabling the zstd-compressed debuginfo by default when building the Rust toolchain, at least? Not necessarily every binary.

[khuey]

I'm not sure I understand the question.

[khuey]

Rereading that now that I'm fully awake ...

If the concern here is the artifact size at the end I think we could turn on zstd compression for debug info for the dist rustc binaries. Really people shouldn't need more than backtraces anyways, and since backtrace-rs supports zstd now we should be good I think?

If people are going to do profiling/etc (with other tools that may not support zstd-compressed debug info) I'd assume they'd build from source but maybe I'm wrong about that.

[workingjubilee]

If people are going to do profiling/etc (with other tools that may not support zstd-compressed debug info) I'd assume they'd build from source but maybe I'm wrong about that.

yeah that's what I'm thinking myself.

[workingjubilee]

@bors try @rust-timer queue

[bors]

⌛ Trying commit 6fb058e with merge 7100709...

[rust-log-analyzer]

The job x86_64-gnu-llvm-17 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

------
 > importing cache manifest from ghcr.io/rust-lang/rust-ci-cache:fb97a1d6377f6cf2227825318ca4bbde3889e0c420746f5a03ba46a07e9a862725c26a09b9fc49a0d129ebd75935d3f6cd19acf41cc4267a6846fd4aa574b12c:
------
##[endgroup]
Setting extra environment values for docker:  --env ENABLE_GCC_CODEGEN=1 --env GCC_EXEC_PREFIX=/usr/lib/gcc/
[CI_JOB_NAME=x86_64-gnu-llvm-17]
---
sccache: Starting the server...
##[group]Configure the build
configure: processing command line
configure: 
configure: build.configure-args := ['--build=x86_64-unknown-linux-gnu', '--llvm-root=/usr/lib/llvm-17', '--enable-llvm-link-shared', '--set', 'rust.randomize-layout=true', '--set', 'rust.thin-lto-import-instr-limit=10', '--set', 'change-id=99999999', '--enable-verbose-configure', '--enable-sccache', '--disable-manage-submodules', '--enable-locked-deps', '--enable-cargo-native-static', '--set', 'rust.codegen-units-std=1', '--set', 'dist.compression-profile=balanced', '--dist-compression-formats=xz', '--set', 'rust.lld=false', '--disable-dist-src', '--release-channel=nightly', '--enable-debug-assertions', '--enable-overflow-checks', '--enable-llvm-assertions', '--set', 'rust.verify-llvm-ir', '--set', 'rust.codegen-backends=llvm,cranelift,gcc', '--set', 'llvm.static-libstdcpp', '--enable-new-symbol-mangling']
configure: target.x86_64-unknown-linux-gnu.llvm-config := /usr/lib/llvm-17/bin/llvm-config
configure: llvm.link-shared     := True
configure: rust.randomize-layout := True
configure: rust.thin-lto-import-instr-limit := 10
---
  Downloaded boml v0.3.1
   Compiling boml v0.3.1
   Compiling y v0.1.0 (/checkout/compiler/rustc_codegen_gcc/build_system)
    Finished `release` profile [optimized] target(s) in 3.79s
     Running `/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-codegen/x86_64-unknown-linux-gnu/release/y test --use-system-gcc --use-backend gcc --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/cg_gcc --release --mini-tests --std-tests`
Using system GCC
[BUILD] example
[AOT] mini_core_hello_world
/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/cg_gcc/mini_core_hello_world
abc

[bors]

💔 Test failed - checks-actions

[rust-log-analyzer]

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

[workingjubilee]

uh.
@bors try

[bors]

⌛ Trying commit 6fb058e with merge e7fc2c2...

[bors]

☀️ Try build successful - checks-actions
Build commit: e7fc2c2 (e7fc2c2abd1d8618431f98ae4bd29292fb0bffe2)

[rust-timer]

Finished benchmarking commit (e7fc2c2): comparison URL.

Overall result: ❌ regressions - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.3%	[0.2%, 4.4%]	20
Regressions ❌ (secondary)	1.5%	[0.2%, 12.9%]	60
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	2.3%	[0.2%, 4.4%]	20

Max RSS (memory usage)

Results (primary 4.0%, secondary 0.7%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	4.0%	[1.7%, 10.8%]	7
Regressions ❌ (secondary)	3.6%	[2.2%, 5.1%]	5
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-3.0%	[-4.9%, -1.5%]	4
All ❌✅ (primary)	4.0%	[1.7%, 10.8%]	7

Cycles

Results (primary 5.0%, secondary 6.3%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	5.0%	[3.6%, 10.0%]	5
Regressions ❌ (secondary)	6.3%	[1.8%, 14.2%]	3
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	5.0%	[3.6%, 10.0%]	5

Binary size

Results (primary 2.2%, secondary 10.6%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	4.8%	[0.1%, 19.9%]	29
Regressions ❌ (secondary)	10.6%	[0.1%, 20.2%]	84
Improvements ✅ (primary)	-0.2%	[-0.5%, -0.0%]	30
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	2.2%	[-0.5%, 19.9%]	59

Bootstrap: 759.275s -> 761.352s (0.27%)
Artifact size: 341.29 MiB -> 341.75 MiB (0.14%)

[workingjubilee]

less awful, but still pretty...

[khuey]

The size increases were baked in when using the bindings to libzstd.so was refused months ago.

[workingjubilee]

dynamically linking in libzstd.so would also be unacceptable.

[khuey]

I haven't had much time to look at this but I did take a brief glance and I found this surprising:

khuey@zhadum:~/dev/rust$ readelf --dyn-syms libstd-after.so | grep ruzstd | wc -l
89

It's not limited to ruzstd either

khuey@zhadum:~/dev/rust$ readelf --dyn-syms libstd-before.so | grep miniz | wc -l
3

These compression crates are implementation details of backtrace, right? Why are their symbols leaking out of libstd.so?

Some of these symbols are things like Display::fmt impls for errors in ruzstd. I picked one of them (HuffmanTableError's) and verified that it's not actually called by anything in libstd.so, so if it weren't exported it would presumably be discarded by the linker.

[khuey]

I suppose it's possible that Display::fmt impls are hanging around because they're still referred to by a vtable, but it still doesn't seem like they should be exposed from libstd.so. There are also symbols like ruzstd::blocks::sequence_section::CompressionModes::of_mode that show up that definitely wouldn't be in a vtable.

[tmiasko]

I don't think such considerations are used to limit exported symbols at the moment. It is possible to use those crates, so their public API is exported:

#![feature(rustc_private)]
extern crate ruzstd;
use ruzstd::blocks::sequence_section::*;

fn main() {
    let _ = CompressionModes::of_mode as fn(_) -> _;
}

[khuey]

Just stripping out the Display impls on the error types in ruzstd (which backtrace-rs doesn't even use anyways, since it immediate .ok()s the return values) recovers about 24% of the size increase in libstd.so.

[khuey]

I don't think such considerations are used to limit exported symbols at the moment. It is possible to use those crates, so their public API is exported:

That seems more than a little crazy.

[khuey]

I think we can get the size increase in libstd.so to about 140k, but lower than that seems unachievable.

[workingjubilee]

I mean, I'm most concerned about the helloworld impact.

[workingjubilee]

Basically, right now, even if they don't actually need zstd decoding ability (because the binary contains no such debuginfo sections), they will still compile it in. It's bothersome because if we're creating the binary we definitely know the answer to that question...

[bjorn3]

It's bothersome because if we're creating the binary we definitely know the answer to that question...

Libc or another system library used by the program may use zstd compressed sections. Or it may be compressed by post-processing of the executable.

[workingjubilee]

ah, true, I forgot about libc... bothersome.

[khuey]

I haven't actually tested helloworld locally but I would expect the size impact there to be bounded above by the impact on libstd.so. That generally tracks with the perf results above (helloworld increased by about 84k above, presumably the linker can GC more stuff once libstd is statically linked in).

As bjorn3 pointed out there's really no way to know that we won't use zstd compression since it can be applied to shared libraries or even the binary itself after the fact.