Unable to download files, certificate verify failed

[nightlark]

Problem
Rustup has no option to disable certificate checking. The place I work at has networking hardware in place that basically breaks SSL. I've tried adding --no-check-certificate and --insecure to the curl/wget commands in the "easy" rustup-init.sh script, but then the rustup command fails due to the same SSL certificate validation errors. A no-nonsense get.

Steps

1. Find a company that intercepts SSL connections.
2. Try rustup-init.sh with curl and wget commands modified to not check executable. Fail because rustup checks the certificate.
3. Try installing using apt-get rustc. Fail because rustup needs to get run (eg rustup install stable) to install a toolchain, and there is no way to bypass the SSL certificate check.

Possible Solution(s)
Add a command line argument that bypasses the SSL certificate check. I've seen the other issues saying to add certificates to the ca cert folder, but there is no certificate available to add to the cert store.

Notes
Adding a command line argument to disable checking certificates is the simplest solution for a user in a restrictive corporate network that just wants a way to get a functional install of rust. It should not be as painful as it currently is; this is a useability bug, that makes for a very bad user experience when your computer is on a corporate network.

[nightlark]

Okay, I stumbled upon an environment variable, RUSTUP_USE_CURL, that appears to not do the SSL certificate checking. It would've been much nicer if a reliable workaround for corporate environments were mentioned somewhere on the install page for rust.

[kinnison]

We have been around on this topic before. Currently we have no intention to add to rustup a way to ignore certificates.

That RUSTUP_USE_CURL=1 helped you suggests that your system has appropriate certificates installed onto it, and that our reqwest based backend is not managing to find them. It is certainly not a workaround to disable certificate checking and if it currently does ignore bad certificates then that is a bug which needs to be fixed.

Do you have problems with your browser every time you visit an https webpage, or does your computer have a stored CA certificate for your company's MITM device?

[nightlark]

It causes problems on enough websites using https that it’s the first thing to blame when I see an error with ssl certificates, though other https webpages work.

Mentioning in a FAQ on the installation page the curl backend as a workaround if there are certificate errors using the default reqwest backend would’ve been nice. If it’s because it does a better job finding certificates, then as a user I don’t care that I’m using a backend that is deprecated — I just want some quick troubleshooting options to try with the installer so that I can get a working install of rust.

I was kind of surprised that installing with apt also required fetching a toolchain (ssl cert issues too) to get a working copy of the rust installer — I was hoping the apt package would be an easy alternative (basically the same as the offline installer, but with apt for updates).

[kinnison]

See, I wouldn't want to mention that in the readme explicitly because we're hoping to remove the cURL backend over time, so encouraging people to use it would be counterproductive. What we need to do is determine where the certificates are being stored and thusly why reqwest isn't finding them.

Regarding the use of apt you can install rustc and cargo via apt install rustc cargo and then just use them; but anything expecting to be able to interact with rustup (e.g. vscode-rls) will fail, as will anything expecting the rust-toolchain file to work.

[kinnison]

The version of cargo and rustc you get via apt will be up to your distribution, not us.

[nightlark]

Actually, it looks like I might have been wrong about RUSTUP_USE_CURL=1 working; I noticed today that running rustc is still giving the no default toolchain configured error, and running rustup install stable or rustup default stable give errors regarding verifying the SSL certificate (with both the reqwest and curl backends) due to it being "self-signed".

The copy of rustc I'm using was installed with apt install rustc cargo -- even trying to print help/version gives the toolchain error; is there some setup command that is needed to tell rustc to ignore the lack of a default toolchain? The top search results all say to use rustup, which isn't working due to the certificate issues (I found a list of certs at work and installed all of them -- none of them seem to be the one used by the MITM device).

[kinnison]

So it sounds like you're expecting /usr/bin to be in your PATH before wherever rustup is (probably ~/.cargo/bin) but in reality the latter is first, so when you run rustc it's picking up rustup's proxy.

Either uninstall rustup (rustup self uninstall) move the PATH around, or you can add your system-installed toolchain for now rustup toolchain link system /usr && rustup toolchain default system and see if that lets you progress.

Regarding the insecure-SSL stuff, we have another issue where we're thinking about this (#1542 and the associated PR #1624) but we're not inclined to support breaking the SSL security constraints unless Cargo opts to do so first.

[Bharath-KKB]

I had the same issue today. Adding Zscaler Root CA cert (used by rust sites) solved the issue.

[Aashay221999]

@Bharath-KKB Hey, can you explain what all you did?

[Bharath-KKB]

@Bharath-KKB Hey, can you explain what all you did?

The exact steps depend on the OS you are using so follow the instructions for your OS. Basically what you need to do is save the Zscaler Root CA cert (from say your browser on windows) and add it to your truststore. For eg. to add on ubuntu/debian systems:
Copy your CA to dir /usr/local/share/ca-certificates/
Use command: sudo cp zscalerroot.crt /usr/local/share/ca-certificates/zscalerroot.crt
Update the CA store: sudo update-ca-certificates

I have not done these for a few months (I only do these when I need to upgrade software). But you need to look out if the intermediate CA certs are valid.. Usually the root CA cert is valid for longer periods(few years typically) but the intermediate certs issuing these certs for me are valid for 2 or 3 weeks . Somehow the cert chains don't work so had to import these regularly.

[lquinn2015]

I really struggled with actually getting the certificate. My advise for figuring this out is to to go to an https://google.com on your corp network on chrome and click the little lock next to the URL. Click the "connection is secure" tab -> "certificate is valid" -> details. The top certificate is your companies self-signed certificate chrome will export it correctly (I tried for hours with openssl to extract this certificate but failed.) Verify for the Certificate manually. Than export the highest most certificate.

Now I was using ubuntu WSL so you can do the following assuming you cert is call company_cert.cer

openssl x509 -in company_cert.cer -inform der -text -out company_cert.crt  ; this is text verification
# sudo cp company_cert.crt /users/local/shared/ca-certificates/
# sudo update-ca-certificates 

After that rustup should work. This Is mostly a SSL problem not a rustup problem but the information on actually find the cert and than also installing it via ca-certs is not in one place. If someone knows how to extract the certificate via openssl that would be a better fix because I assume you have chrome

[ilyagr]

I have a similar problem on Debian stable. rustup update worked fine until recently, but now it fails. RUSTUP_USE_CURL=1 rustup update works.

I'm not sure if it's connected, but one thing I remember doing recently that could be related is sudo apt install firefox-esr.

I put a few more details in https://users.rust-lang.org/t/rustup-update-fails-with-a-certificate-error-but-curl-works/92551/1.

[rbtcollins]

@ilyagr do you have the certificate correctly in place? see e.g. #1856 (comment)

What version of rustup do you have?

[ilyagr]

@ilyagr do you have the certificate correctly in place? see e.g. #1856 (comment)

I can try that (exporting a certificate from my browser). It's a confusing operation. Also, is it relevant for non-corporate networks?

When I follow those instructions to find a root CA I find something generic. In firefox for google.com, I have a "GTS Root R1" certificate and a "GlobalSign Root CA" certificate.

As I said, rustup was working fine before and curl just works.

What version of rustup do you have?

rustup 1.25.2 (17db695 2023-02-01)

This is running on Debian stable on Chrome OS. (I'm not sure if the certificate store in Chrome OS's linux is connected to the certificate score in Chrome OS itself)

Update: One thing I noticed that might be relevant is that the certificate on static.rust-lang.org was issued on Thursday, February 23, 2023, and so is newer than my version of rustup.

I can also copy the exact message from the forum post I linked:

$ rustup update
info: syncing channel updates for 'beta-x86_64-unknown-linux-gnu'
error: could not download file from 'https://static.rust-lang.org/dist/channel-rust-beta.toml.sha256' to '/home/ilyagr/.local/app/rustup/tmp/n8ste_t87j1jy11u_file'
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
error: could not download file from 'https://static.rust-lang.org/dist/channel-rust-nightly.toml.sha256' to '/home/ilyagr/.local/app/rustup/tmp/zdes5clkoev1tz5o_file'
info: syncing channel updates for '1.64-x86_64-unknown-linux-gnu'
error: could not download file from 'https://static.rust-lang.org/dist/channel-rust-1.64.toml.sha256' to '/home/ilyagr/.local/app/rustup/tmp/h8sfjn1fqih4kpcr_file'
info: checking for self-updates
error: could not download file from 'https://static.rust-lang.org/rustup/release-stable.toml' to '/tmp/rustup-updateWTZmyb/release-stable.toml':
  failed to make network request: error sending request for url (https://static.rust-lang.org/rustup/release-stable.toml):
    error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1921:
      (unable to get local issuer certificate): error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1921:
      (unable to get local issuer certificate): error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1921:
      (unable to get local issuer certificate): error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1921:

$ rustup --version
rustup 1.25.2 (17db695f1 2023-02-01)
info: This is the version for the rustup toolchain manager, not the rustc compiler.
info: The currently active `rustc` version is `rustc 1.69.0-beta.5 (734f69d95 2023-03-31)`
$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

$ curl -L https://static.rust-lang.org/rustup/release-stable.toml
schema-version = '1'
version = '1.25.2'

$ curl -L https://static.rust-lang.org/dist/channel-rust-beta.toml.sha256
3af143d9455fb5005b8aab886cb9e9a51e3bdba626276cc24c365347abe5b4b8  channel-rust-beta.toml

[ilyagr]

It seems that updating to the recently released rustup 1.26 via RUSTUP_USE_CURL=1 rustup self update fixed the problem for me. 🎉

That is, now rustup works even without RUSTUP_USE_CURL=1.

I wouldn't be surprised if the problem reoccurs the next time the certificate for static.rust-lang.org gets updated again, though. Or, perhaps, there was some more substantial fix in rustup 1.26, I'm unsure.

[jsturtevant]

I ran into this problem today on rustup 1.26 (worked fine yesterday), using RUSTUP_USE_CURL=1 made everything work but there were no updates to rustup.

Uninstalling rustup (rustup self uninstall) and then re-installing has fixed the issue for me

[ayoubachak]

In windows 11, I got this error when running rustup-init.exe:

1) Proceed with standard installation (default - just press enter)
2) Customize installation
3) Cancel installation
>

info: profile set to 'default'
info: default host triple is x86_64-pc-windows-msvc
info: syncing channel updates for 'stable-x86_64-pc-windows-msvc'
info: latest update on 2024-04-09, rust version 1.77.2 (25ef9e3d8 2024-04-09)
info: downloading component 'cargo'
info: downloading component 'clippy'
info: downloading component 'rust-docs'
error: component download failed for rust-docs-x86_64-pc-windows-msvc: error decoding response body: operation timed out

I knew it was most likely an SSL error under the hood, but setting this environment variable solved the issue :

set RUSTUP_HTTPS_REDIRECT=http

Just execute it in the terminal, and run the rustup-init.exe from the same terminal session.

[rami3l]

@ayoubachak As for Rustup v1.27.0, the default download/TLS backend combination is reqwest/native-tls.

Does setting RUSTUP_USE_RUSTLS=1 (which means using reqwest/rustls) change anything on your side?
Also, does setting RUSTUP_USE_CURL=1 (which means using libcurl) change anything?

In fact, we are currently investigating issues regarding different download/TLS backends:

At least for some of the issues, there's the suggestion that native-tls might actually be at fault, such that reqwest + rustls might actually do better than the default reqwest + native-tls option.

Originally posted by @djc in #3788 (comment)

[ayoubachak]

@ayoubachak As for Rustup v1.27.0, the default download/TLS backend combination is reqwest/native-tls.

Does setting RUSTUP_USE_RUSTLS=1 (which means using reqwest/rustls) change anything on your side? Also, does setting RUSTUP_USE_CURL=1 (which means using libcurl) change anything?

In fact, we are currently investigating issues regarding different download/TLS backends:

At least for some of the issues, there's the suggestion that native-tls might actually be at fault, such that reqwest + rustls might actually do better than the default reqwest + native-tls option.

Originally posted by @djc in #3788 (comment)

I didn't try changing RUSTUP_USE_RUSTLS or RUSTUP_USE_CURL , they could have worked, but since it was a 1 time issue and the setting of RUSTUP_HTTPS_REDIRECT=http worked for me on windows, I thought why not share it just in case.
I'm on a corporate laptop btw, we're using ZScaler.

[rami3l]

I didn't try changing RUSTUP_USE_RUSTLS or RUSTUP_USE_CURL , they could have worked, but since it was a 1 time issue and the setting of RUSTUP_HTTPS_REDIRECT=http worked for me on windows, I thought why not share it just in case. I'm on a corporate laptop btw, we're using ZScaler.

@ayoubachak So you mean this issue is not quite reproducible? That's the very problem that we are facing right now 🤦‍♀️ Thanks anyway!