rustup: Bring up rustup rustc/cargo language installer

[Grommish]

Signed-off-by: Donald Hoskins grommish@gmail.com

Maintainer: me
Compile tested: None
Run tested: None

Description:
Work in progress. Input/Changes would be exceedingly helpful.

[Grommish]

This passed my ARCH test, but would need to be tested by others. I will be sending up a Suricata 5.0.3 package that can be used for testing, as it requires rust to compile.

test package (suricata 5.0.3) PR can be found: #13170

@adschm - I would appreciate your formatting oversight on these, please :)

[adschm]

@Grommish I don't do packages, but there might be many people around that are familiar with the requirements for package feed I'm sure.

[BKPepe]

I didn't look at it much, some quick notes about it. I didn't do compile neither run test. Still thinking that this does not look right because it should be done similar way as go or python in OpenWrt.

Take a look at this comment #11336 (comment) I think @jefferyto should have word on this as well.

[jefferyto]

If my understanding is correct, rustup downloads and installs prebuilt binaries. I don't really like this approach because AFAIK:

• The files it downloads are not cached (offline installation is also not possible)
• It doesn't validate any signatures of the files it downloads
• We have to trust that the binaries they built are safe

[jefferyto]

Perhaps you can save $(filter $(ARCH)-%, $(RUSTC_ARCH_TARGETS)) to a variable so you don't have to repeat it multiple times.

[Grommish]

If my understanding is correct, rustup downloads and installs prebuilt binaries. I don't really like this approach because AFAIK:

* The files it downloads are not cached (offline installation is also not possible)

* It doesn't validate any signatures of the files it downloads

* We have to trust that the binaries they built are safe

I'm all for building from source, when possible. Rustup does validate signatures, afaik, but yes, if there is a better way, I'm all for it

[detly]

I would think the appropriate build-from-source method would be to compile Rust toolchain + Cargo as host packages, and use Cargo for all other rust packages. I'm not sure how bootstrapping would work though, since rustc is in Rust and there is not a frontend for GCC. This is where I got totally stuck; if I need Rust code in OpenWrt I depend on Cargo being installed on the host machine, which is not the standard required for general OpenWrt packaging.

[jefferyto]

I would try looking at / replicating https://github.com/dtolnay/bootstrap.

Be aware that the complete bootstrap uses a lot of disk space. Bootstrapping up through rustc 1.40 requires 120 GB.

This is a concern, but perhaps after compiling a version X+1, version X can be discarded, this might save disk space. This process might take a long time though.

I would also suggest looking at how other distros are handling this, although I'm not sure they are doing a complete bootstrap from source either.

[detly]

This is also an interesting issue: Rust #48707 mainly because of the links to other info on bootstrapping strategies (including the mrustc approach @jefferyto posted).

[Grommish]

I'm looking at the mrustc package. I think I can shrink the footprint. It creates every version of rustc from 1.29 thru 1.45, and probably doesn't need to. However, it also relies on a debian-type system to work and tries to setup a sandbox vm to build in, which may or may not be a deal-breaker. I don't seem to need it under Ubuntu, but not everyone will be using ubuntu to build with. I'll report back when i see what is actually needed

[Grommish]

Update:

I can't even get mrustc to compile outside of OpenWrt; I shiver at the thought of trying to get it done inside the buildroot.
That being said, from what I can see, it builds out each version of rust from 1.29 thru 1.45, IN ORDER, to use the previous version to build the next. This is where the absurd footprint size comes from. I cannot see anyone giving the OK to balloon 160gb PER TARGET on the build servers.. Others can look further into mrustc, but I'm considering it DOA..

rustc from source:
I'm workign on getting rustc from source to build inside the buildroot, and it's slow going. That being said, I need to point out that the rustc source downloads the precompiled 1.44.0 in order to build 1.45... This leads back to the whole "what if they poison the upstream"..

If someone manages to poison the rust repo, then it'll affect anything compiled with rust.. It would NOT effect OpenWrt itself, since at this point, rust is only used to compile packages inside OpenWrt. Until the feasibility of something like mrustc being practical, building from source is the next best thing. Of course, if we use a precompiled binary to build the new version from source, why not use the precompiled totally in rustup..

Since until rust is used in the compilation of OpenWrt, it poses no security threat as rustup is a host-only package. Any package compiled with it would be open to the kind of attack mrustc seeks to mitigate, but the threat is low and any breach of that sort would be widespread news, allowing whoever to outdate the rustup package and any packages that uses it (currently, i think only my suricata5 package uses rust) would just be recompiled with the new version of the toolchain.

[Grommish]

Since mrustc is not an option, and the current rustc/cargo source code not only downloads a pre-compiled binary to compile the latest version, but also has an issue that keeps it from being usable (rust-lang/rust#75841) in it's present form. Because of this, I'm requesting to push on with rustup.

[Grommish]

rustup in now abandoned, as it is ill-equipped to deal with the OpenWrt environment.

New PR for Rust-from-source is #13916

[Grommish]

For anyone in the future finding this because a search for openwrt and rustup, let me save you some time.

As of this comment, rustup is not -msoft-float, and there for cannot be used by Openwrt MUSL.

Check the status of #13916 for status on rust lang being available in OpenWrt.