feat!: use stable hash from rustc-stable-hash #14116
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rustbot
added
A-cache-messages
|
This is blocked on releasing rustc-stable-hash to crates.io |
|
From a <1 minute reading of "-Ztrim-paths build a stable cross-platform path for the registry and git sources." My understanding is that the intent here is to use a hash function to create a stable path to a particular set of source files or artifacts. If there is a hash collision then potentially hash(malicous-sources) == hash(trusted-sources) and so malicous-sources could be used instead of trusted-sources, silently. |
Urgau
reviewed
Jun 20, 2024
src/cargo/util/hasher.rs
Outdated
Comment on lines
22
to
24
| fn write(&mut self, bytes: &[u8]) { | ||
| self.0.write(bytes) | ||
| } |
There was a problem hiding this comment.
I'm not sure only forwarding Hasher::write is enough, since the endian-ness handling is done on the individual write_{u,i}{8,16,32,64,128} methods and not forwarding those will bypass that endian-ness handling 1.
I think it's also going to bypass the {u,i}size handling.
Footnotes
-
the default implementation use native endian-ness, instead of a fixed one ↩
There was a problem hiding this comment.
After rust-lang/rustc-stable-hash#6 and rust-lang/rustc-stable-hash#8, I think StableSipHasher128 should be a drop-in replacement that we can use directly.
I am also thinking of blake3 as an alternative, but haven't figured out how to make it play nice with ExtendedHasher.
This helps `-Ztrim-paths` build a stable cross-platform path for the registry and git sources. Sources files then can be found from the same path when debugging. See rust-lang#13171 (comment) A few caveats: * This will invalidate the current downloaded caches. Need to put this in the Cargo CHANGELOG. * As a consequence of changing how `SourceId` is hashed, the global cache tracker is also affected because Cargo writes source identifiers (e.g. `index.crates.io-6f17d22bba15001f`) to SQLite. * https://github.com/rust-lang/cargo/blob/6e236509b2331eef64df844b7bbc8ed352294107/src/cargo/core/global_cache_tracker.rs#L388-L391 * The performance of rustc-stable-hash is slightly worse than the old SipHasher in std on short things like `SourceId`, but for long stuff like fingerprint. See appendix. StableHasher is used in several places (some might not be needed?): * Rebuild detection (fingerprints) * Rustc version, including all the CLI args running `rustc -vV`. * https://github.com/rust-lang/cargo/blob/6e236509b2331eef64df844b7bbc8ed352294107/src/cargo/util/rustc.rs#L326 * https://github.com/rust-lang/cargo/blob/6e236509b2331eef64df844b7bbc8ed352294107/src/cargo/util/rustc.rs#L381 * Build caches * https://github.com/rust-lang/cargo/blob/6e236509b2331eef64df844b7bbc8ed352294107/src/cargo/core/compiler/fingerprint/mod.rs#L1456 * Compute rustc `-C metadata` * stable hash for SourceId * https://github.com/rust-lang/cargo/blob/6e236509b2331eef64df844b7bbc8ed352294107/src/cargo/core/package_id.rs#L207 * Also read and hash contents from custom target JSON file. * https://github.com/rust-lang/cargo/blob/6e236509b2331eef64df844b7bbc8ed352294107/src/cargo/core/compiler/compile_kind.rs#L81-L91 * `UnitInner::dep_hash` * This is to distinguish same units having different features set between normal and build dependencies. * https://github.com/rust-lang/cargo/blob/6e236509b2331eef64df844b7bbc8ed352294107/src/cargo/ops/cargo_compile/mod.rs#L627 * Hash file contents for `cargo package` to verify if files were modified before and after the build. * https://github.com/rust-lang/cargo/blob/6e236509b2331eef64df844b7bbc8ed352294107/src/cargo/ops/cargo_package.rs#L999 * Rusc diagnostics deduplication * https://github.com/rust-lang/cargo/blob/6e236509b2331eef64df844b7bbc8ed352294107/src/cargo/core/compiler/job_queue/mod.rs#L311 * Places using `SourceId` identifier like `registry/src` path, and `-Zscript` target directories. Appendix -------- Benchmark on x86_64-unknown-linux-gnu ``` bench_hasher/RustcStableHasher/URL time: [33.843 ps 33.844 ps 33.845 ps] change: [-0.0167% -0.0049% +0.0072%] (p = 0.44 > 0.05) No change in performance detected. Found 10 outliers among 100 measurements (10.00%) 5 (5.00%) low severe 3 (3.00%) high mild 2 (2.00%) high severe bench_hasher/SipHasher/URL time: [18.954 ns 18.954 ns 18.955 ns] change: [-0.1281% -0.0951% -0.0644%] (p = 0.00 < 0.05) Change within noise threshold. Found 14 outliers among 100 measurements (14.00%) 3 (3.00%) low severe 4 (4.00%) low mild 3 (3.00%) high mild 4 (4.00%) high severe bench_hasher/RustcStableHasher/lorem ipsum time: [659.18 ns 659.20 ns 659.22 ns] change: [-0.0192% -0.0062% +0.0068%] (p = 0.34 > 0.05) No change in performance detected. Found 12 outliers among 100 measurements (12.00%) 4 (4.00%) low severe 3 (3.00%) low mild 3 (3.00%) high mild 2 (2.00%) high severe bench_hasher/SipHasher/lorem ipsum time: [1.2006 µs 1.2008 µs 1.2010 µs] change: [+0.0117% +0.0467% +0.0808%] (p = 0.01 < 0.05) Change within noise threshold. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high mild ```
weihanglo
commented
Jul 9, 2024
| // The hash value depends on endianness and bit-width, so we only run this test on | ||
| // little-endian 64-bit CPUs (such as x86-64 and ARM64) where it matches the | ||
| // well-known value. | ||
| // The hash value should be stable across platforms, and doesn't depend on |
There was a problem hiding this comment.
This is something we need to fix, if the goal is a fully cross-platform.
|
For information, |
|
Status update: Had some chats with Urgau, here is data from benchmark blake3/blake2s on rustc
While in Cargo hash is not a dominant factor of performance it still plays a role. A Will need more real world benchmarks for like |
|
☔ The latest upstream changes (presumably #14334) made this pull request unmergeable. Please resolve the merge conflicts. |
8 tasks
|
Got some more data. This hash numbers matches what I expected (10-20 hash per units) Number and size of hash inputs in a
|
|
The benchmark result of rustc-stable-hash, blake3, and the original siphash. We have 5 kinds of inputs.
And according to the benchmark result below, rustc-stable-hash and the original is pretty much at the same scale for small input (<=200 chars), whereas our integration of blake3 is worse and 3x slower. I think if we don't consider cryptographic hash, switching to rustc-stable-hash is clearly a no-brainer. Blake3 is surprisingly bad, and Benchmark on aarch64-apple-darwin
Benchmark on x86_64-unknown-linux-gnu
criterion benchmark script
#![allow(deprecated)]
use std::hash::Hash as _;
use std::hash::Hasher;
use criterion::criterion_group;
use criterion::criterion_main;
use criterion::BenchmarkId;
use criterion::Criterion;
use rustc_stable_hash::StableSipHasher128 as RustcStableHasher;
pub struct SipHasher(std::hash::SipHasher);
impl SipHasher {
pub fn new() -> SipHasher {
SipHasher(std::hash::SipHasher::new())
}
}
impl Hasher for SipHasher {
fn finish(&self) -> u64 {
self.0.finish()
}
fn write(&mut self, bytes: &[u8]) {
self.0.write(bytes)
}
}
pub struct Blake3Hasher {
state: blake3::Hasher,
}
impl Blake3Hasher {
pub fn new() -> Blake3Hasher {
Blake3Hasher {
state: Default::default(),
}
}
}
impl rustc_stable_hash::ExtendedHasher for Blake3Hasher {
type Hash = blake3::Hash;
#[inline]
fn finish(self) -> Self::Hash {
self.state.finalize()
}
}
impl Hasher for Blake3Hasher {
#[inline]
fn write(&mut self, bytes: &[u8]) {
self.state.update(bytes);
}
#[inline]
fn finish(&self) -> u64 {
let hash = self.state.finalize();
let [a0, a1, a2, a3, a4, a5, a6, a7, b0, b1, b2, b3, b4, b5, b6, b7, c0, c1, c2, c3, c4, c5, c6, c7, d0, d1, d2, d3, d4, d5, d6, d7] =
*hash.as_bytes();
let p0 = u64::from_ne_bytes([a0, a1, a2, a3, a4, a5, a6, a7]);
let p1 = u64::from_ne_bytes([b0, b1, b2, b3, b4, b5, b6, b7]);
let p2 = u64::from_ne_bytes([c0, c1, c2, c3, c4, c5, c6, c7]);
let p3 = u64::from_ne_bytes([d0, d1, d2, d3, d4, d5, d6, d7]);
p0.wrapping_mul(3)
.wrapping_add(p1)
.wrapping_add(p2)
.wrapping_mul(p3)
.to_le()
}
}
const INPUTS: &[(&'static str, &'static str)] = &[
("URL", "registry+https://github.com/rust-lang/crates.io-index"),
("100 chars", "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec sem odio, consectetur ac velit ac, he"),
("200 chars", "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec sem odio, consectetur ac velit ac, hendrerit pulvinar nisl. Aenean auctor felis non accumsan porta. Nullam purus diam, aliquam nec dui vi"),
("300 chars", "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec sem odio, consectetur ac velit ac, hendrerit pulvinar nisl. Aenean auctor felis non accumsan porta. Nullam purus diam, aliquam nec dui vitae, iaculis fermentum eros. Nunc laoreet lectus nec malesuada tristique. Quisque venenatis vehicula"),
("lorem ipsum (3222 chars)", "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec sem odio, consectetur ac velit ac, hendrerit pulvinar nisl. Aenean auctor felis non accumsan porta. Nullam purus diam, aliquam nec dui vitae, iaculis fermentum eros. Nunc laoreet lectus nec malesuada tristique. Quisque venenatis vehicula lacus sed auctor. In libero sapien, auctor vulputate tellus ut, scelerisque feugiat neque. Sed feugiat nulla vel lorem tincidunt viverra. Proin blandit pretium sapien id imperdiet. Sed elementum, ligula quis porttitor consectetur, augue ligula consectetur erat, at congue massa tortor in odio. Morbi sit amet tincidunt libero, eu rutrum felis. Integer rhoncus tortor et erat congue venenatis. Proin ac ante sit amet urna tincidunt ullamcorper. Vestibulum nec tincidunt neque. Vestibulum venenatis, libero et blandit pretium, risus nibh efficitur libero, vel condimentum tortor nulla non sapien. Morbi ac dapibus est. Duis justo arcu, laoreet lacinia luctus mollis, placerat non augue. Interdum et malesuada fames ac ante ipsum primis in faucibus. Fusce vestibulum eu tellus in pellentesque. Nam efficitur mattis turpis. Vestibulum a condimentum purus. Suspendisse eget augue scelerisque sem dignissim ornare vitae in augue. Vestibulum porta rhoncus sapien, non luctus nisi vehicula in. Etiam cursus tortor turpis, eu imperdiet purus facilisis ut. Nullam vestibulum erat ex, sit amet commodo est fermentum eleifend. Donec pulvinar imperdiet urna, egestas ultricies mi pulvinar at. Maecenas velit dui, iaculis at egestas eu, consequat sit amet nisl. Ut eu leo ultricies, porttitor ante eu, ultrices massa. Nam commodo, nunc ut mollis egestas, lectus ex eleifend nisl, vitae mollis metus quam vitae sapien. Curabitur eu nulla massa. Vivamus sodales turpis et lorem placerat, ac dignissim nulla luctus. In placerat eleifend orci, dapibus varius felis tincidunt sed. Nulla suscipit mauris condimentum ipsum finibus, ac mattis sapien aliquet. Cras feugiat elementum augue, viverra lacinia ante congue et. Sed et bibendum sem. Aenean pretium tellus eget velit commodo pretium a sit amet velit. Curabitur vitae est vitae nulla venenatis tristique in a eros. In scelerisque lectus et luctus mattis. Cras ac purus ac purus tempor molestie vitae vitae felis. Quisque volutpat elementum felis vitae mollis. Pellentesque finibus quam eget vestibulum tempus. Praesent quis massa eget ligula ultrices lobortis. Ut pellentesque, mi ac finibus sagittis, dui felis tempor dui, ac commodo mauris massa nec dolor. Cras congue, lectus vitae luctus faucibus, massa mauris malesuada elit, et facilisis turpis odio non justo. Proin volutpat turpis quis ante interdum pellentesque. Morbi faucibus, erat vel elementum aliquet, odio leo eleifend magna, sagittis semper lorem mauris nec arcu. Curabitur lacinia sagittis ante mollis facilisis. Fusce ultrices tellus sed justo rhoncus varius ut eu justo. Sed a est purus. Sed nec mi laoreet, consequat justo nec, sodales augue. Nullam posuere ipsum et velit aliquam blandit a quis metus. Aliquam id eros non magna suscipit bibendum. Curabitur porta auctor sapien, a molestie nisl. Donec neque leo, consequat vitae velit sit amet, aliquam elementum purus. Donec sit amet congue mi. Etiam at magna nunc."),
];
fn bench_hasher(c: &mut Criterion) {
let mut group = c.benchmark_group("bench_hasher");
group.sample_size(1000);
for (name, input) in INPUTS {
let id = BenchmarkId::new("RustcStableHasher", name);
group.bench_with_input(id, input, |b, input| {
b.iter(|| {
let mut hasher = RustcStableHasher::new();
input.hash(&mut hasher);
_ = Hasher::finish(&hasher);
})
});
let id = BenchmarkId::new("Blake3Hasher", name);
group.bench_with_input(id, input, |b, input| {
b.iter(|| {
let mut hasher = Blake3Hasher::new();
input.hash(&mut hasher);
_ = Hasher::finish(&hasher);
})
});
let id = BenchmarkId::new("SipHasher", name);
group.bench_with_input(id, input, |b, input| {
b.iter(|| {
let mut hasher = SipHasher::new();
input.hash(&mut hasher);
_ = Hasher::finish(&hasher);
})
});
}
group.finish();
}
criterion_group!(benches, bench_hasher);
criterion_main!(benches); |
|
Here is a benchmark a bit closer the real world project — it captures and generates exactly same length of random bytes for each of input bytes being hashed. So the time spent is close to the totoal time spent in stable-hashing when running Summary: The original siphash and rustc-stable-hash are still on par with each others, while blake3 is 2x to 3x slower. However, the total time spent is under 1.5ms for blake3, and under 500µs for others. 4638 of hash computations are included in a run. This may suggest that it has no significant difference on the user side if we even switch to blake3. Benchmark on aarch64-apple-darwin
Benchmark on x86_64-unknown-linux-gnu
criterion benchmark script
#![allow(deprecated)]
use std::hash::Hash as _;
use std::hash::Hasher;
use criterion::criterion_group;
use criterion::criterion_main;
use criterion::BenchmarkId;
use criterion::Criterion;
use rand::RngCore as _;
use rustc_stable_hash::StableSipHasher128 as RustcStableHasher;
/// All lengths of input bytes being hash from `cargo build`ing cargo itself.
const HASHES: [u64; 4638] = [
110, 94, 471, 40, 53, 110, 94, 0, 0, 16, 0, 8, 0, 0, 0, 48, 0, 8, 8, 0, 0, 0, 0, 8, 8, 8, 0,
24, 8, 8, 0, 0, 40, 0, 0, 0, 8, 16, 0, 0, 24, 8, 8, 16, 16, 16, 16, 16, 8, 8, 24, 8, 24, 16,
16, 24, 8, 24, 32, 24, 0, 8, 8, 0, 8, 0, 8, 0, 16, 8, 24, 24, 16, 0, 0, 0, 0, 8, 40, 0, 8, 24,
16, 0, 16, 0, 0, 48, 48, 0, 0, 16, 16, 40, 16, 8, 16, 0, 0, 8, 8, 8, 16, 8, 0, 24, 16, 0, 40,
0, 8, 56, 8, 8, 0, 8, 8, 0, 8, 8, 0, 24, 8, 0, 16, 0, 32, 24, 0, 0, 8, 0, 8, 32, 24, 0, 24, 0,
8, 24, 104, 0, 8, 16, 0, 16, 24, 0, 8, 8, 8, 8, 16, 8, 8, 0, 8, 8, 0, 32, 0, 16, 16, 8, 40, 32,
0, 0, 8, 0, 0, 8, 8, 16, 24, 64, 0, 0, 8, 40, 8, 0, 0, 32, 0, 0, 16, 32, 8, 0, 16, 16, 16, 32,
8, 32, 48, 8, 16, 0, 8, 24, 16, 32, 40, 40, 32, 0, 32, 24, 48, 0, 8, 8, 16, 0, 8, 8, 0, 16, 0,
8, 0, 32, 16, 16, 0, 112, 0, 40, 32, 24, 0, 8, 0, 72, 0, 32, 8, 8, 48, 40, 24, 40, 24, 24, 16,
96, 104, 16, 104, 40, 40, 72, 32, 56, 40, 0, 8, 56, 72, 160, 56, 88, 96, 8, 32, 104, 64, 0, 0,
96, 104, 32, 88, 24, 80, 56, 48, 56, 360, 0, 8, 8, 0, 8, 8, 8, 16, 8, 8, 16, 48, 0, 8, 0, 0, 8,
0, 8, 0, 24, 0, 8, 24, 0, 32, 16, 24, 8, 8, 16, 48, 104, 16, 16, 48, 8, 32, 32, 104, 0, 0, 16,
24, 0, 0, 8, 24, 8, 8, 48, 0, 32, 8, 24, 0, 0, 16, 0, 8, 8, 0, 8, 0, 8, 16, 24, 80, 24, 8, 8,
8, 0, 568, 576, 298, 306, 330, 298, 319, 298, 294, 320, 356, 292, 291, 304, 302, 288, 288, 290,
289, 300, 298, 294, 309, 300, 291, 284, 304, 336, 296, 295, 294, 318, 292, 282, 306, 281, 296,
332, 304, 294, 251, 353, 302, 293, 311, 286, 352, 360, 301, 308, 283, 300, 367, 250, 303, 302,
313, 317, 316, 304, 303, 298, 329, 309, 385, 328, 336, 280, 302, 278, 306, 305, 354, 283, 282,
324, 306, 288, 329, 288, 290, 371, 335, 292, 343, 351, 371, 366, 348, 285, 304, 296, 293, 292,
314, 320, 330, 329, 413, 626, 314, 282, 338, 290, 294, 340, 298, 302, 298, 316, 307, 280, 279,
294, 292, 343, 304, 290, 318, 289, 381, 316, 296, 306, 294, 341, 340, 377, 312, 304, 332, 313,
326, 344, 369, 298, 297, 318, 284, 318, 324, 286, 296, 287, 304, 318, 310, 312, 312, 292, 287,
278, 293, 314, 306, 351, 316, 315, 358, 334, 302, 322, 328, 315, 300, 323, 326, 314, 310, 353,
288, 284, 306, 406, 366, 298, 355, 294, 306, 312, 348, 302, 322, 329, 322, 309, 375, 294, 356,
310, 284, 313, 365, 310, 326, 393, 384, 322, 343, 320, 319, 338, 294, 326, 324, 304, 298, 311,
302, 292, 325, 290, 289, 294, 336, 318, 358, 290, 500, 280, 328, 320, 314, 306, 313, 294, 372,
306, 326, 298, 294, 350, 344, 308, 336, 312, 322, 312, 388, 390, 302, 396, 332, 326, 374, 320,
352, 332, 290, 300, 350, 368, 450, 352, 395, 382, 316, 370, 396, 362, 288, 280, 421, 390, 352,
467, 334, 392, 352, 342, 354, 803, 280, 294, 298, 290, 306, 287, 278, 333, 318, 304, 324, 330,
292, 324, 288, 306, 294, 290, 327, 294, 306, 294, 308, 328, 294, 362, 298, 312, 288, 302, 312,
381, 468, 302, 333, 410, 300, 392, 509, 487, 288, 301, 314, 357, 320, 326, 296, 391, 366, 387,
358, 304, 288, 306, 316, 310, 304, 294, 409, 333, 296, 288, 304, 294, 304, 312, 333, 528, 324,
289, 280, 294, 310, 806, 798, 2350, 126, 209, 72, 88, 0, 87, 120, 290, 209, 78, 95, 0, 93,
2229, 157, 209, 82, 99, 0, 97, 87, 120, 161, 209, 74, 90, 0, 89, 2350, 133, 209, 78, 94, 0, 93,
2350, 105, 209, 78, 94, 0, 93, 2350, 101, 209, 76, 92, 0, 91, 2312, 151, 209, 85, 102, 0, 100,
89, 2350, 180, 209, 73, 90, 0, 88, 87, 120, 155, 209, 67, 87, 0, 86, 209, 86, 120, 155, 209,
71, 88, 0, 86, 120, 161, 209, 71, 88, 0, 86, 120, 157, 209, 73, 89, 0, 88, 120, 210, 209, 73,
89, 0, 88, 120, 179, 209, 67, 86, 0, 84, 209, 84, 120, 179, 209, 69, 87, 0, 84, 120, 209, 209,
74, 91, 0, 89, 84, 120, 318, 209, 70, 86, 0, 85, 120, 309, 209, 67, 84, 0, 82, 120, 145, 209,
67, 86, 0, 86, 209, 86, 120, 330, 209, 71, 87, 0, 86, 120, 175, 209, 81, 97, 0, 96, 120, 145,
209, 71, 87, 0, 86, 120, 196, 209, 73, 89, 0, 88, 86, 120, 196, 209, 67, 89, 0, 89, 209, 89,
120, 196, 209, 74, 90, 0, 89, 82, 120, 216, 209, 75, 92, 0, 90, 120, 210, 209, 70, 87, 0, 85,
164, 259, 209, 67, 89, 0, 88, 209, 88, 164, 259, 209, 73, 90, 0, 88, 120, 366, 209, 71, 88, 0,
86, 86, 84, 120, 212, 209, 73, 90, 0, 88, 84, 120, 448, 209, 68, 85, 0, 83, 386, 146, 209, 67,
16, 0, 6, 82, 90, 120, 151, 209, 67, 100, 0, 98, 82, 120, 195, 209, 67, 102, 0, 100, 209, 100,
82, 90, 85, 164, 259, 209, 67, 89, 0, 88, 209, 88, 209, 98, 82, 90, 120, 184, 209, 67, 98, 0,
96, 82, 90, 120, 215, 209, 67, 91, 0, 91, 88, 82, 90, 85, 120, 171, 209, 66, 96, 0, 91, 209,
91, 209, 91, 88, 91, 209, 96, 209, 6, 86, 120, 179, 209, 67, 86, 0, 84, 209, 84, 120, 179, 209,
69, 87, 0, 84, 120, 210, 209, 67, 87, 0, 85, 209, 85, 120, 275, 209, 67, 92, 0, 91, 209, 91,
120, 221, 209, 78, 95, 0, 93, 120, 275, 209, 76, 93, 0, 91, 91, 120, 130, 209, 70, 87, 0, 85,
91, 85, 93, 120, 124, 209, 68, 85, 0, 83, 120, 223, 209, 77, 95, 0, 92, 120, 210, 209, 70, 88,
0, 85, 120, 141, 209, 69, 86, 0, 84, 120, 278, 209, 71, 87, 0, 86, 120, 137, 209, 68, 85, 0,
83, 85, 120, 175, 209, 67, 92, 0, 90, 209, 90, 120, 175, 209, 75, 93, 0, 90, 120, 116, 209, 67,
90, 0, 89, 209, 89, 91, 85, 83, 120, 149, 209, 79, 96, 0, 94, 120, 116, 209, 74, 91, 0, 89,
120, 337, 209, 73, 89, 0, 88, 85, 120, 116, 209, 73, 90, 0, 88, 84, 249, 314, 209, 73, 89, 0,
88, 88, 85, 120, 199, 209, 74, 90, 0, 89, 2201, 251, 209, 69, 86, 0, 84, 386, 156, 209, 81, 43,
0, 17, 17, 120, 194, 209, 73, 89, 0, 88, 120, 187, 209, 84, 100, 0, 99, 84, 120, 187, 209, 80,
97, 0, 95, 99, 84, 99, 84, 683, 245, 209, 87, 104, 0, 102, 683, 237, 209, 83, 99, 0, 98, 386,
169, 209, 96, 58, 0, 32, 85, 386, 147, 209, 79, 37, 0, 15, 86, 95, 86, 84, 120, 212, 209, 73,
90, 0, 88, 120, 167, 209, 68, 84, 0, 83, 120, 181, 209, 67, 96, 0, 95, 209, 95, 120, 181, 209,
80, 97, 0, 95, 120, 175, 209, 80, 97, 0, 95, 95, 120, 175, 209, 80, 96, 0, 95, 86, 120, 188,
209, 77, 93, 0, 92, 86, 92, 86, 120, 184, 209, 77, 93, 0, 92, 120, 219, 209, 79, 95, 0, 94,
120, 191, 209, 69, 85, 0, 84, 120, 132, 209, 68, 85, 0, 83, 94, 92, 120, 412, 209, 72, 89, 0,
87, 83, 86, 94, 120, 226, 209, 74, 90, 0, 89, 89, 120, 173, 209, 72, 88, 0, 87, 120, 290, 209,
71, 88, 0, 86, 84, 120, 209, 209, 74, 91, 0, 89, 84, 89, 86, 84, 120, 261, 209, 76, 93, 0, 91,
120, 173, 209, 78, 94, 0, 93, 120, 204, 209, 67, 94, 0, 93, 209, 93, 120, 389, 209, 66, 92, 0,
87, 209, 87, 120, 389, 209, 72, 89, 0, 87, 120, 383, 209, 72, 88, 0, 87, 120, 204, 209, 78, 95,
0, 93, 120, 131, 209, 77, 94, 0, 92, 120, 312, 209, 74, 90, 0, 89, 93, 87, 120, 119, 209, 78,
94, 0, 93, 120, 263, 209, 71, 87, 0, 86, 120, 164, 209, 71, 88, 0, 86, 120, 201, 209, 69, 86,
0, 84, 120, 170, 209, 77, 93, 0, 92, 86, 120, 147, 209, 73, 89, 0, 88, 88, 84, 120, 161, 209,
70, 86, 0, 85, 84, 884, 208, 209, 67, 88, 0, 86, 209, 86, 884, 208, 209, 71, 89, 0, 86, 120,
306, 209, 73, 90, 0, 88, 1133, 147, 209, 81, 98, 0, 96, 91, 85, 83, 120, 259, 209, 83, 100, 0,
98, 120, 152, 209, 74, 91, 0, 89, 120, 168, 209, 77, 94, 0, 92, 120, 190, 209, 72, 89, 0, 87,
87, 386, 144, 209, 75, 33, 0, 11, 120, 162, 209, 67, 87, 0, 86, 209, 86, 85, 120, 162, 209, 71,
88, 0, 86, 85, 85, 120, 135, 209, 71, 87, 0, 86, 120, 149, 209, 77, 93, 0, 92, 85, 86, 120,
171, 209, 79, 95, 0, 94, 120, 139, 209, 72, 88, 0, 87, 120, 177, 209, 67, 91, 0, 90, 209, 90,
120, 177, 209, 75, 92, 0, 90, 120, 188, 209, 78, 95, 0, 93, 85, 120, 103, 209, 76, 92, 0, 91,
89, 85, 85, 2350, 127, 209, 78, 94, 0, 93, 85, 2229, 166, 209, 78, 94, 0, 93, 120, 105, 209,
75, 91, 0, 90, 93, 120, 176, 209, 67, 86, 0, 85, 209, 85, 86, 89, 120, 158, 209, 73, 90, 0, 88,
120, 176, 209, 70, 87, 0, 85, 120, 182, 209, 79, 96, 0, 94, 120, 152, 209, 74, 91, 0, 89, 142,
127, 209, 73, 89, 0, 88, 85, 93, 93, 86, 2199, 123, 209, 71, 88, 0, 86, 2229, 176, 209, 74, 92,
0, 89, 2229, 351, 209, 69, 86, 0, 84, 120, 347, 209, 76, 92, 0, 91, 120, 121, 209, 81, 97, 0,
96, 120, 185, 209, 80, 96, 0, 95, 120, 153, 209, 77, 94, 0, 92, 120, 134, 209, 79, 95, 0, 94,
120, 151, 209, 72, 88, 0, 87, 120, 387, 209, 86, 103, 0, 101, 120, 160, 209, 69, 85, 0, 84, 96,
120, 145, 209, 68, 84, 0, 83, 386, 136, 209, 83, 41, 0, 19, 88, 87, 2271, 102, 209, 73, 89, 0,
88, 120, 318, 209, 71, 88, 0, 86, 86, 120, 157, 209, 78, 94, 0, 93, 2258, 139, 209, 77, 94, 0,
92, 2258, 139, 209, 69, 86, 0, 84, 84, 88, 120, 156, 209, 78, 94, 0, 93, 85, 2258, 121, 209,
78, 95, 0, 93, 120, 278, 209, 71, 87, 0, 86, 120, 153, 209, 80, 96, 0, 95, 120, 143, 209, 68,
84, 0, 83, 91, 85, 83, 120, 160, 209, 87, 103, 0, 102, 120, 205, 209, 76, 92, 0, 91, 120, 204,
209, 67, 85, 0, 84, 98, 209, 84, 98, 84, 84, 100, 120, 195, 209, 79, 103, 0, 100, 84, 88, 164,
259, 209, 73, 90, 0, 88, 120, 151, 209, 73, 98, 0, 98, 84, 84, 120, 287, 209, 72, 88, 0, 87,
120, 204, 209, 69, 86, 0, 84, 96, 85, 90, 89, 83, 386, 142, 209, 74, 29, 0, 10, 84, 98, 88, 89,
88, 120, 215, 209, 71, 87, 0, 86, 168, 316, 209, 76, 92, 0, 91, 120, 366, 209, 71, 88, 0, 86,
88, 84, 84, 96, 84, 91, 88, 84, 91, 120, 171, 209, 76, 93, 0, 91, 120, 215, 209, 76, 89, 0, 91,
88, 91, 120, 184, 209, 76, 96, 0, 96, 83, 83, 120, 310, 209, 69, 86, 0, 84, 84, 84, 83, 83,
120, 249, 209, 74, 91, 0, 89, 84, 84, 84, 120, 235, 209, 69, 86, 0, 84, 89, 2532, 181, 209, 73,
89, 0, 88, 84, 88, 101, 2532, 184, 209, 74, 91, 0, 89, 84, 89, 86, 2532, 147, 209, 74, 91, 0,
89, 84, 88, 84, 120, 175, 209, 70, 86, 0, 85, 89, 95, 120, 205, 209, 82, 99, 0, 97, 86, 85,
120, 177, 209, 75, 91, 0, 90, 89, 2532, 183, 209, 73, 90, 0, 88, 2532, 199, 209, 74, 91, 0, 89,
89, 84, 89, 87, 120, 190, 209, 67, 89, 0, 88, 209, 88, 120, 346, 209, 75, 91, 0, 90, 120, 190,
209, 73, 90, 0, 88, 86, 84, 120, 172, 209, 67, 96, 0, 96, 209, 96, 120, 178, 209, 73, 90, 0,
88, 120, 172, 209, 81, 97, 0, 96, 120, 195, 209, 76, 93, 0, 91, 83, 91, 120, 191, 209, 72, 89,
0, 87, 120, 173, 209, 74, 90, 0, 89, 89, 87, 2532, 193, 209, 77, 94, 0, 92, 84, 89, 359, 143,
209, 69, 85, 0, 84, 89, 89, 2532, 186, 209, 73, 91, 0, 88, 2532, 180, 209, 73, 90, 0, 88, 88,
84, 89, 89, 2532, 194, 209, 74, 91, 0, 89, 89, 120, 199, 209, 82, 98, 0, 97, 120, 132, 209, 72,
88, 0, 87, 88, 89, 120, 165, 209, 76, 92, 0, 91, 2532, 179, 209, 79, 96, 0, 94, 84, 88, 89,
120, 174, 209, 76, 92, 0, 91, 2532, 190, 209, 76, 93, 0, 91, 84, 89, 2532, 206, 209, 74, 90, 0,
89, 92, 88, 84, 120, 196, 209, 72, 88, 0, 87, 89, 2532, 211, 209, 80, 97, 0, 95, 84, 88, 84,
88, 84, 89, 2532, 185, 209, 81, 98, 0, 96, 92, 88, 88, 89, 92, 88, 92, 89, 2532, 181, 209, 71,
88, 0, 86, 88, 86, 84, 89, 91, 88, 2532, 185, 209, 77, 94, 0, 92, 89, 89, 2532, 155, 209, 73,
90, 0, 88, 84, 89, 88, 92, 88, 88, 89, 91, 2532, 201, 209, 78, 94, 0, 93, 89, 84, 89, 2532,
173, 209, 77, 93, 0, 92, 84, 88, 89, 86, 2532, 187, 209, 75, 92, 0, 90, 88, 92, 89, 92, 87, 89,
86, 2532, 151, 209, 72, 89, 0, 87, 88, 84, 2532, 181, 209, 72, 89, 0, 87, 86, 89, 88, 89, 91,
86, 2532, 163, 209, 75, 92, 0, 90, 84, 91, 96, 88, 91, 96, 91, 86, 89, 2532, 187, 209, 75, 91,
0, 90, 87, 89, 84, 92, 88, 89, 83, 2532, 200, 209, 72, 89, 0, 87, 89, 2532, 191, 209, 80, 97,
0, 95, 88, 84, 88, 90, 89, 2532, 168, 209, 73, 90, 0, 88, 84, 84, 86, 88, 88, 87, 87, 89, 2532,
189, 209, 77, 94, 0, 92, 86, 84, 88, 88, 89, 91, 2532, 175, 209, 75, 92, 0, 90, 88, 84, 88,
120, 137, 209, 68, 81, 0, 83, 89, 2532, 204, 209, 75, 92, 0, 90, 92, 86, 88, 88, 90, 88, 95,
88, 88, 93, 90, 95, 88, 88, 93, 90, 88, 89, 2532, 177, 209, 76, 93, 0, 91, 88, 89, 2532, 150,
209, 77, 94, 0, 92, 89, 92, 89, 84, 84, 87, 86, 88, 89, 2532, 209, 209, 74, 91, 0, 89, 90, 88,
88, 84, 94, 96, 88, 88, 89, 2532, 176, 209, 77, 93, 0, 92, 89, 89, 84, 94, 92, 86, 88, 88, 90,
89, 90, 88, 92, 2532, 199, 209, 77, 94, 0, 92, 89, 2532, 179, 209, 72, 88, 0, 87, 92, 92, 84,
86, 120, 195, 209, 76, 93, 0, 91, 94, 91, 88, 90, 84, 90, 89, 89, 2532, 194, 209, 88, 105, 0,
103, 88, 89, 89, 89, 88, 89, 2532, 175, 209, 75, 92, 0, 90, 86, 88, 88, 93, 90, 89, 88, 88, 95,
88, 88, 90, 91, 88, 89, 2532, 186, 209, 78, 95, 0, 93, 90, 120, 133, 209, 73, 89, 0, 88, 88,
92, 86, 88, 93, 90, 120, 183, 209, 69, 85, 0, 84, 89, 92, 88, 93, 90, 88, 92, 87, 91, 88, 89,
2532, 167, 209, 73, 90, 0, 88, 88, 89, 91, 88, 89, 2532, 158, 209, 72, 89, 0, 87, 88, 88, 92,
90, 84, 95, 88, 92, 88, 86, 84, 84, 91, 95, 92, 84, 90, 89, 89, 2532, 197, 209, 79, 96, 0, 94,
89, 87, 87, 89, 2532, 200, 209, 78, 95, 0, 93, 89, 91, 85, 83, 120, 162, 209, 76, 93, 0, 91,
89, 86, 2532, 181, 209, 77, 94, 0, 92, 87, 84, 88, 84, 95, 88, 88, 90, 91, 89, 2532, 218, 209,
77, 94, 0, 92, 92, 88, 89, 2532, 188, 209, 76, 93, 0, 91, 92, 91, 87, 84, 90, 88, 92, 91, 87,
89, 2532, 173, 209, 78, 95, 0, 93, 92, 89, 93, 92, 87, 89, 92, 92, 89, 87, 88, 89, 2532, 169,
209, 68, 85, 0, 83, 120, 196, 209, 69, 85, 0, 84, 83, 86, 120, 161, 209, 69, 86, 0, 84, 84, 86,
120, 226, 209, 74, 90, 0, 89, 120, 211, 209, 74, 90, 0, 89, 86, 87, 120, 122, 209, 72, 88, 0,
87, 93, 120, 185, 209, 67, 86, 0, 85, 209, 85, 86, 84, 120, 178, 209, 74, 91, 0, 89, 120, 226,
209, 74, 90, 0, 89, 89, 120, 190, 209, 77, 93, 0, 92, 87, 87, 120, 133, 209, 77, 93, 0, 92, 87,
120, 185, 209, 70, 87, 0, 85, 88, 120, 171, 209, 71, 88, 0, 86, 120, 153, 209, 74, 91, 0, 89,
89, 120, 189, 209, 73, 89, 0, 88, 84, 96, 86, 120, 161, 209, 71, 87, 0, 86, 83, 120, 245, 209,
72, 88, 0, 87, 120, 225, 209, 74, 90, 0, 89, 89, 120, 226, 209, 80, 96, 0, 95, 89, 120, 182,
209, 76, 92, 0, 91, 86, 87, 120, 134, 209, 70, 87, 0, 85, 89, 120, 321, 209, 73, 89, 0, 88,
120, 449, 209, 76, 92, 0, 91, 87, 120, 307, 209, 68, 84, 0, 83, 86, 120, 339, 209, 73, 89, 0,
88, 93, 89, 86, 87, 120, 328, 209, 78, 94, 0, 93, 86, 89, 86, 120, 216, 209, 67, 84, 0, 82, 93,
82, 89, 86, 120, 201, 209, 70, 87, 0, 85, 84, 120, 187, 209, 69, 86, 0, 84, 91, 83, 83, 120,
234, 209, 69, 85, 0, 84, 120, 313, 209, 70, 87, 0, 85, 89, 88, 83, 93, 85, 86, 87, 120, 294,
209, 69, 85, 0, 84, 86, 87, 120, 310, 209, 79, 96, 0, 94, 84, 86, 120, 269, 209, 72, 88, 0, 87,
86, 89, 120, 181, 209, 74, 90, 0, 89, 84, 120, 334, 209, 70, 87, 0, 85, 94, 94, 120, 301, 209,
75, 92, 0, 90, 84, 120, 351, 209, 69, 86, 0, 84, 89, 92, 86, 94, 92, 120, 309, 209, 70, 87, 0,
85, 85, 90, 84, 86, 84, 87, 120, 147, 209, 73, 89, 0, 88, 120, 165, 209, 73, 89, 0, 88, 84,
120, 162, 209, 75, 92, 0, 90, 89, 120, 224, 209, 76, 92, 0, 91, 89, 120, 221, 209, 69, 85, 0,
84, 85, 88, 120, 144, 209, 82, 98, 0, 97, 120, 153, 209, 92, 108, 0, 107, 89, 120, 179, 209,
73, 89, 0, 88, 82, 90, 85, 120, 135, 209, 67, 95, 0, 94, 209, 94, 120, 135, 209, 79, 96, 0, 94,
88, 120, 122, 209, 73, 90, 0, 88, 120, 160, 209, 75, 91, 0, 90, 85, 90, 89, 87, 386, 242, 209,
72, 30, 0, 8, 89, 86, 85, 94, 85, 120, 167, 209, 78, 95, 0, 93, 90, 86, 91, 86, 120, 111, 209,
69, 86, 0, 84, 92, 120, 163, 209, 84, 100, 0, 99, 120, 147, 209, 81, 97, 0, 96, 88, 84, 120,
448, 209, 68, 85, 0, 83, 88, 89, 84, 84, 89, 87, 90, 92, 120, 159, 209, 77, 94, 0, 92, 120,
219, 209, 79, 96, 0, 94, 120, 180, 209, 73, 89, 0, 88, 120, 158, 209, 73, 89, 0, 88, 120, 261,
209, 77, 94, 0, 92, 89, 85, 120, 180, 209, 76, 92, 0, 91, 120, 171, 209, 77, 93, 0, 92, 88, 86,
89, 120, 144, 209, 77, 93, 0, 92, 87, 92, 83, 89, 92, 120, 183, 209, 76, 92, 0, 91, 120, 252,
209, 83, 100, 0, 98, 120, 182, 209, 79, 95, 0, 94, 98, 93, 120, 151, 209, 67, 87, 0, 87, 209,
87, 120, 151, 209, 72, 88, 0, 87, 120, 290, 209, 78, 94, 0, 93, 83, 87, 386, 146, 209, 70, 22,
0, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 97, 88, 87, 86, 86,
86, 88, 6, 6, 17, 32, 15, 11, 19, 84, 93, 91, 10, 84, 98, 88, 86, 84, 89, 83, 84, 83, 84, 84,
89, 89, 86, 85, 88, 89, 89, 88, 84, 96, 86, 86, 87, 88, 88, 84, 85, 88, 90, 8, 89, 86, 85, 94,
93, 90, 84, 92, 99, 96, 83, 88, 89, 84, 84, 89, 87, 94, 98, 87, 93, 83, 87, 386, 146, 209, 54,
26, 0, 8, 204, 204, 187, 187, 171, 171, 165, 165, 286, 286, 314, 314, 155, 155, 173, 173, 171,
171, 297, 297, 258, 258, 327, 327, 311, 311, 585, 585, 181, 181, 159, 159, 196, 196, 327, 327,
275, 275, 207, 207, 195, 195, 468, 468, 301, 301, 195, 195, 227, 227, 231, 231, 205, 205, 224,
224, 324, 324, 233, 233, 309, 230, 230, 210, 210, 309, 257, 257, 340, 340, 230, 230, 200, 200,
185, 185, 185, 185, 266, 266, 133, 133, 251, 251, 163, 163, 222, 222, 207, 207, 268, 268, 321,
321, 177, 177, 354, 354, 186, 186, 163, 163, 244, 244, 196, 196, 352, 352, 315, 315, 303, 303,
198, 198, 345, 345, 254, 254, 225, 225, 472, 472, 239, 239, 196, 196, 153, 153, 777, 777, 405,
405, 715, 715, 239, 239, 379, 379, 410, 410, 187, 187, 279, 279, 275, 275, 201, 201, 465, 465,
397, 397, 263, 263, 201, 201, 133, 133, 292, 292, 227, 227, 206, 206, 1102, 279, 262, 262, 279,
295, 295, 4537, 4537, 347, 347, 806, 806, 304, 304, 244, 244, 316, 316, 259, 259, 263, 263,
182, 182, 200, 200, 537, 537, 479, 479, 272, 272, 475, 475, 167, 167, 214, 214, 542, 542, 235,
235, 188, 188, 1102, 310, 310, 323, 323, 198, 198, 322, 322, 340, 340, 199, 199, 369, 262, 262,
201, 201, 170, 170, 369, 302, 302, 340, 340, 474, 474, 301, 301, 182, 207, 207, 182, 180, 180,
468, 468, 310, 310, 246, 246, 186, 186, 4754, 4754, 266, 266, 231, 231, 189, 189, 359, 359,
373, 373, 286, 286, 282, 282, 415, 415, 339, 339, 218, 218, 190, 190, 192, 192, 239, 239, 271,
271, 177, 177, 328, 328, 196, 196, 191, 191, 287, 287, 422, 422, 244, 244, 267, 267, 308, 308,
240, 240, 284, 284, 755, 755, 192, 192, 223, 223, 163, 163, 161, 161, 167, 167, 201, 201, 171,
171, 203, 203, 183, 183, 165, 165, 668, 668, 294, 294, 404, 404, 4537, 4537, 202, 202, 319,
319, 331, 331, 359, 359, 205, 205, 348, 348, 298, 298, 195, 195, 238, 238, 207, 207, 157, 157,
238, 238, 161, 280, 161, 280, 821, 821, 161, 161, 188, 188, 153, 153, 235, 235, 735, 735, 207,
207, 228, 228, 275, 275, 157, 157, 197, 197, 445, 288, 193, 193, 204, 204, 273, 273, 224, 224,
445, 288, 341, 341, 305, 305, 256, 256, 580, 580, 244, 244, 417, 417, 320, 320, 253, 253, 268,
268, 208, 208, 217, 217, 1016, 1016, 247, 247, 449, 449, 352, 352, 360, 360, 245, 245, 293,
293, 323, 323, 313, 313, 544, 364, 364, 446, 446, 436, 436, 425, 425, 421, 421, 208, 208, 544,
544, 544, 362, 362, 817, 817, 537, 344, 344, 231, 231, 267, 267, 1138, 1138, 606, 606, 591,
591, 169, 169, 328, 328, 537, 708, 533, 533, 385, 385, 396, 396, 238, 238, 477, 477, 266, 708,
266, 328, 328, 560, 560, 229, 229, 538, 538, 216, 216, 380, 380, 359, 359, 280, 280, 219, 219,
213, 213, 291, 291, 260, 260, 962, 962, 132, 132, 270, 270, 196, 196, 997, 997, 302, 302, 351,
351, 395, 395, 682, 682, 569, 569, 487, 487, 523, 523, 250, 250, 353, 353, 1405, 1405, 539,
539, 687, 687, 485, 485, 356, 356, 596, 596, 358, 358, 537, 537, 972, 972, 319, 319, 285, 285,
392, 392, 266, 266, 221, 221, 213, 213, 183, 161, 161, 183, 209, 209, 169, 169, 202, 202, 480,
480, 376, 376, 222, 222, 204, 204, 185, 185, 255, 255, 197, 197, 171, 171, 195, 195, 210, 210,
204, 204, 179, 179, 293, 293, 323, 323, 485, 485, 250, 250, 271, 271, 277, 277, 314, 314, 267,
267, 253, 253, 204, 204, 380, 380, 763, 763, 194, 194, 946, 946, 411, 411, 333, 333, 473, 473,
497, 497, 262, 262, 314, 314, 405, 405, 198, 198, 163, 163, 196, 196, 216, 216, 177, 177, 208,
208, 188, 188, 169, 169, 183, 183, 219, 219, 269, 269, 153, 153, 217, 217, 318, 318, 2727,
2727, 2965, 2965, 493, 493, 3030, 3030, 502, 265, 265, 502, 2116, 2116, 2139, 2139,
];
pub struct SipHasher(std::hash::SipHasher);
impl SipHasher {
pub fn new() -> SipHasher {
SipHasher(std::hash::SipHasher::new())
}
}
impl Hasher for SipHasher {
fn finish(&self) -> u64 {
self.0.finish()
}
fn write(&mut self, bytes: &[u8]) {
self.0.write(bytes)
}
}
pub struct Blake3Hasher {
state: blake3::Hasher,
}
impl Blake3Hasher {
pub fn new() -> Blake3Hasher {
Blake3Hasher {
state: Default::default(),
}
}
}
impl rustc_stable_hash::ExtendedHasher for Blake3Hasher {
type Hash = blake3::Hash;
#[inline]
fn finish(self) -> Self::Hash {
self.state.finalize()
}
}
impl Hasher for Blake3Hasher {
#[inline]
fn write(&mut self, bytes: &[u8]) {
self.state.update(bytes);
}
#[inline]
fn finish(&self) -> u64 {
let hash = self.state.finalize();
let [a0, a1, a2, a3, a4, a5, a6, a7, b0, b1, b2, b3, b4, b5, b6, b7, c0, c1, c2, c3, c4, c5, c6, c7, d0, d1, d2, d3, d4, d5, d6, d7] =
*hash.as_bytes();
let p0 = u64::from_ne_bytes([a0, a1, a2, a3, a4, a5, a6, a7]);
let p1 = u64::from_ne_bytes([b0, b1, b2, b3, b4, b5, b6, b7]);
let p2 = u64::from_ne_bytes([c0, c1, c2, c3, c4, c5, c6, c7]);
let p3 = u64::from_ne_bytes([d0, d1, d2, d3, d4, d5, d6, d7]);
p0.wrapping_mul(3)
.wrapping_add(p1)
.wrapping_add(p2)
.wrapping_mul(p3)
.to_le()
}
}
fn bench_hasher(c: &mut Criterion) {
let mut group = c.benchmark_group("bench_hasher");
group.sample_size(1000);
let id = BenchmarkId::new("RustcStableHasher", "cargo");
let input: Vec<Vec<u8>> = HASHES
.iter()
.map(|length| {
let mut bytes = vec![0; *length as usize];
rand::thread_rng().fill_bytes(&mut bytes);
bytes
})
.collect();
group.bench_with_input(id, &input, |b, input| {
b.iter(|| {
for bytes in input {
let mut hasher = RustcStableHasher::new();
bytes.hash(&mut hasher);
_ = Hasher::finish(&hasher);
}
})
});
let id = BenchmarkId::new("Blake3Hasher", "cargo");
group.bench_with_input(id, &input, |b, input| {
b.iter(|| {
for bytes in input {
let mut hasher = Blake3Hasher::new();
bytes.hash(&mut hasher);
_ = Hasher::finish(&hasher);
}
})
});
let id = BenchmarkId::new("SipHasher", "cargo");
group.bench_with_input(id, &input, |b, input| {
b.iter(|| {
for bytes in input {
let mut hasher = SipHasher::new();
bytes.hash(&mut hasher);
_ = Hasher::finish(&hasher);
}
})
});
group.finish();
}
criterion_group!(benches, bench_hasher);
criterion_main!(benches); |
|
Side note: while Cargo never guarantees to generate a stable hash for source URL for cache purposes, it is still better if people can have one single index cache for all platforms. See #14795 |
github-merge-queue bot
pushed a commit
that referenced
this pull request
Dec 11, 2024
### What does this PR try to resolve? This helps `-Ztrim-paths` build a stable cross-platform path for the registry and git sources. Sources files then can be found from the same path when debugging. It also helps cache registry index all at once for all platforms, for example the use case in #14795 (despite they should use `cargo vendor` instead IMO). Some caveats: * Newer cargo will need to re-download files for global caches (index files, git/registry sources). The old cache is still kept and used when running with older cargoes. * Absolute paths on windows iarenot really covered by the "cross-platform" hash, because path prefix components like `C:` are always there. That means hashes of some sources kind, like local registry and local path, are not going to be real cross-platform stable. #### Security concern There might be hash collisions if you have two registries under the same domain. This won't happen to crates.io, as the infra would have to intentionally put another registry on index.crates.io to collide. We don't consider this is an actual threat model, so we are not going to use any cryptographically secure hash algorithm like BLAKE3. At least, the current unstable SipHash isn't in a better situation. We might switch to a cryptographic secure one when needed. See also <#13171 (comment)> ### How should we test and review this PR? We have an FCP in <#14795 (comment)>. This PR implements the proposal, The path-length concern in <#14795 (comment)> is automatically addressed because we don't need cryptographically secure hash for now. ### Additional information See more information and benchmark results in <#14116>.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR try to resolve?
This helps
-Ztrim-pathsbuild a stable cross-platform path for theregistry and git sources. Sources files then can be found from the same
path when debugging.
See #13171 (comment)
How should we test and review this PR?
There are a few caveats, and we should do an FCP before merge:
Need to put this in the Cargo CHANGELOG.
SourceIdis hashed, the global cachetracker is also affected because Cargo writes source identifiers (e.g.
index.crates.io-6f17d22bba15001f) to SQLite.SipHasher in std on short things like
SourceId, but for long stufflike fingerprint. See Additional information.
StableHasher is used in several places. We should consider if there is a need for cryptographyic hash (see #13171 (comment)).
rustc -vV.-C metadataUnitInner::dep_hashcargo packageto verify if files were modified before and after the build.SourceIdidentifier likeregistry/srcpath,and
-Zscripttarget directories.Additional information
Benchmark on x86_64-unknown-linux-gnu
Benchmark on aarch64-apple-darwin
Criterion benchmark script