1.55.0 clippy checksum failure

[Jake-Shadle]

Problem
Trying to install clippy 1.55.0 results in this..

info: syncing channel updates for '1.55.0-x86_64-unknown-linux-gnu'
info: latest update on 2021-09-09, rust version 1.55.0 (c8dfcfe04 2021-09-06)
info: component 'rust-std' for target 'wasm32-unknown-unknown' is up to date
info: downloading component 'clippy'
error: component download failed for clippy-x86_64-unknown-linux-gnu

Caused by:
    checksum failed for 
'https://static.rust-lang.org/dist/2021-09-09/clippy-1.55.0-x86_64-unknown-linux-gnu.tar.xz',
 expected: 
'1aa42d51b80e2bdadac5128bf66f84b9b1c6ca6262995a1bf34f438a55a0bdf8', 
calculated: 
'a874416a52124064214819d5db7f27ae618246474a20bc2fd344052a39500275'

Steps

1. Have a rustup-toolchain.toml file with

[toolchain]
channel = "1.55.0"
components = [ "rustfmt", "clippy" ]
targets = [ "wasm32-unknown-unknown" ]

2. Run any cargo command on a linux host

Possible Solution(s)
This only started happening within the past few hours for something that has worked fine for the past 20 days, so feels like the tarball at the location has actually been changed for some reason but the corresponding metadata checksum was not?

Notes
Output of rustup --version:

rustup 1.24.3 (ce5817a94 2021-05-31)
info: This is the version for the rustup toolchain manager, not the rustc compiler.
info: syncing channel updates for '1.55.0-x86_64-unknown-linux-gnu'
info: latest update on 2021-09-09, rust version 1.55.0 (c8dfcfe04 2021-09-06)
info: component 'rust-std' for target 'wasm32-unknown-unknown' is up to date
info: downloading component 'clippy'

Output of rustup show:

Default host: x86_64-unknown-linux-gnu
rustup home:  /usr/local/rustup

info: syncing channel updates for '1.55.0-x86_64-unknown-linux-gnu'
info: latest update on 2021-09-09, rust version 1.55.0 (c8dfcfe04 2021-09-06)
info: component 'rust-std' for target 'wasm32-unknown-unknown' is up to date
info: downloading component 'clippy'
info: retrying download for 'https://static.rust-lang.org/dist/2021-09-09/clippy-1.55.0-x86_64-unknown-linux-gnu.tar.xz'
(error: component download failed for clippy-x86_64-unknown-linux-gnu, checksum failed for 'https://static.rust-lang.org/dist/2021-09-09/clippy-1.55.0-x86_64-unknown-linux-gnu.tar.xz', expected: '1aa42d51b80e2bdadac5128bf66f84b9b1c6ca6262995a1bf34f438a55a0bdf8', calculated: 'a874416a52124064214819d5db7f27ae618246474a20bc2fd344052a39500275')

[Jake-Shadle]

Oh and was reading #524 but these failures are in containers that have a completely empty /tmp directory so unfortunately can't nuke a cache.

[rswarbrick]

Looks like this might be a CDN configuration issue. If you download the file with a browser and then with curl/wget, you get different SHA sums:

> sha256sum clippy-1.55.0-x86_64-unknown-linux-gnu.tar.xz.*
1aa42d51b80e2bdadac5128bf66f84b9b1c6ca6262995a1bf34f438a55a0bdf8  clippy-1.55.0-x86_64-unknown-linux-gnu.tar.xz.browser
a874416a52124064214819d5db7f27ae618246474a20bc2fd344052a39500275  clippy-1.55.0-x86_64-unknown-linux-gnu.tar.xz.wget

The files are the same size. Hexdumping and diffing shows:

> diff browser.hex wget.hex
62255,62258c62255,62258
< 000f32e0  e6 f5 f5 05 63 e7 e7 a0  89 49 c7 63 47 e6 7e e5  |....c....I.cG.~.|
< 000f32f0  73 94 2c 56 10 9a 45 f8  71 42 03 39 13 63 6c 70  |s.,V..E.qB.9.clp|
< 000f3300  53 a8 ca eb 4e c1 54 03  c2 25 b8 e1 85 e8 a4 5b  |S...N.T..%.....[|
< 000f3310  63 85 15 69 08 0a f7 4c  fb 66 3e 02 f7 52 e9 d7  |c..i...L.f>..R..|
---
> 000f32e0  e6 f5 f5 05 63 e7 e7 a0  89 49 c7 63 47 e6 74 2d  |....c....I.cG.t-|
> 000f32f0  31 03 65 6c 62 09 61 6d  61 7a 6f 6e 61 77 73 03  |1.elb.amazonaws.|
> 000f3300  63 6f 6d 00 00 01 00 01  00 00 29 10 00 00 00 80  |com.......).....|
> 000f3310  00 00 00 69 08 0a f7 4c  fb 66 3e 02 f7 52 e9 d7  |...i...L.f>..R..|

(looks like some ASCII in the xz file metadata?)

My guess is that this can be solved with a CDN refresh.

[Jake-Shadle]

Ahh interesting, I didn't try downloading via the browser, just curl to confirm that I got the same "incorrect" hash locally.

[rswarbrick]

Oh, and the headers look like they're getting different things.

Browser:

Wget:

HTTP request sent, awaiting response... 
  HTTP/1.1 200 OK
  Content-Type: application/x-tar
  Content-Length: 2538660
  Connection: keep-alive
  Date: Wed, 29 Sep 2021 01:07:16 GMT
  Last-Modified: Thu, 09 Sep 2021 14:00:42 GMT
  ETag: "0f28b82ff4f46ba0d7b8735f53cd4371"
  Cache-Control: public
  x-amz-version-id: nXJKl4p0Zg5fL3WGo_WcLzaGw0xkMyyt
  Accept-Ranges: bytes
  Server: AmazonS3
  X-Edge-Origin-Shield-Skipped: 0
  X-Cache: Hit from cloudfront
  Via: 1.1 67ef3abac0a476e3c8690ff0f09febb8.cloudfront.net (CloudFront)
  X-Amz-Cf-Pop: LHR62-C3
  X-Amz-Cf-Id: 2Zg3XGH0_dwVjSn0b9fX4sokfO5qGG0PxnuE2RC5jgI0oh9pJSw_JQ==
  Age: 28274

[imphil]

It's requesting a compressed response (--compressed) that makes the difference.

Right checksum (what the browser does in curl terms):

curl 'https://static.rust-lang.org/dist/2021-09-09/clippy-1.55.0-x86_64-unknown-linux-gnu.tar.xz' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' --compressed --verbose | sha256sum

Wrong checksum (what normal curl does):

curl 'https://static.rust-lang.org/dist/2021-09-09/clippy-1.55.0-x86_64-unknown-linux-gnu.tar.xz' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8'  --verbose | sha256sum

[kinnison]

@pietroalbini @Mark-Simulacrum I think this is an infra issue?

[Mec-iS]

Just a small addition:

For me while trying to install rustup from scratch with stable toolchain is

error: component download failed for clippy-x86_64-unknown-linux-gnu: checksum failed for 'https://static.rust-lang.org/dist/2021-09-09/clippy-1.55.0-x86_64-unknown-linux-gnu.tar.xz', expected: '1aa42d51b80e2bdadac5128bf66f84b9b1c6ca6262995a1bf34f438a55a0bdf8', calculated: 'a874416a52124064214819d5db7f27ae618246474a20bc2fd344052a39500275'

It works on nightly

[pietroalbini]

Hey all, I'm running a full CDN invalidation so things should get back to normal soon.

If this happens again please ping me and @Mark-Simulacrum, and provide the contents of the X-Amz-Cf-Id header. This seems to be an issue specific to individual CDN POPs, and that ID will give AWS support the information they need to troubleshoot the issue.

[aicantar]

Things are working now apparently. Just ran rustup-init and everything went fine, no errors.

[imphil]

@pietroalbini In case it helps after the fact: These were the headers we got this (UK) morning.

Bad: x-amz-cf-id: C-PLqYyZSyCBg3OJug_9JCAvzbJz1k_8wcapJSjhUH7sWBDHjsxEOg==
Good: x-amz-cf-id: vIK8fU5nuvRz2sh0jfypbn_UeCvDnclKqX44SscE835l4Z2BD20CvQ==

POP in all cases x-amz-cf-pop: MAN50-C3

[Jake-Shadle]

Yes, the CDN invalidation seems to have worked and can now install clippy again, thanks for the quick turnaround!