cargo install --reinstall?

[netvl]

Currently cargo install won't overwrite existing package if asked to install one. However, this is very inconvenient when you're testing your own binaries (after each change you need to execute two commands instead of one to get a fresh version in PATH) or if you need a binary which frequently changes upstream.

What do you think of adding a --reinstall key to cargo install command? When used, it would be equivalent to cargo uninstall <args> && cargo install <args without --reinstall>, or something like it. I remember it was discussed in the RFC on cargo install, but I didn't manage to participate in that discussion; maybe it is worth to lay out reasons why it wasn't added eventually and probably reconsider that decision, if there was one.

[alexcrichton]

Yeah the discussion on the RFC ended up reaching a point that behaving akin to "make install" was the best way forward for cargo install for now. It's certainly always possible to add this backwards-compatibly, although I think perhaps it would be cargo install --update instead of --reinstall?

For now this could probably easily be a new cargo reinstall subcommand, however.

[colin-kiegel]

It would also be nice to have something like cargo install --update-all to update all locally installed (and outdated) packages in one go.

[oconnor663]

Is there any reason not to have cargo install perform an update by default, if one is available? I think that's how most Linux package managers work. apt-get install vim will update vim, for example.

[jonas-schievink]

For my CI needs it would be nice to have an option that installs a package if it isn't installed, and otherwise performs an update to the newest version of the package (or does nothing in case it's up-to-date). That way I could more easily make Travis cache the .cargo directory and save precious compile time for cargo installed stuff.

[alexcrichton]

@jonas-schievink interesting! I suspect that would be lumped into whatever "install the most recent version" logic would be added as part of this (if added)

@oconnor663 there was some pushback on the RFC, but opinions may have perhaps changed since then.

[alexcrichton]

Now that we have a PR for this (thanks @gkoz!) let's try to drill into some of the initial pushback and see if the landscape has changed here. I believe the main objection to a flag like this is that it was pushing cargo install a bit too much towards being a package manager, which is something it's explicitly avoiding trying to do.

The RFC originally proposed a --update flag to blanket update all binaries (or update one specific binary), but the flag proposed in this issue and #2405 is somewhat different where it just allows reinstalling or replacing existing binaries.

I'm gonna highlight and cc some of comments on the original RFC that pushed back against functionality like that, but I'm curious if thoughts have changed since? Does usage of cargo install mean that your opinion has changed about --update, or does --reinstall (or --replaced) as proposed here seem bad?

• RFC: Add cargo install rfcs#1200 (comment) - @Stebalien
• RFC: Add cargo install rfcs#1200 (comment) - @jarcane
• RFC: Add cargo install rfcs#1200 (comment) - @eternaleye

Curious what others think!

[jarcane]

My main comment would merely be confusion as to what the difference between --update and --reinstall would be, semantically speaking. Perhaps "update" might mean an in-place update of any changed/recompiled binaries, while "reinstall" would mean explicitly uninstalling the old binaries before reinstalling new ones?

[CamJN]

I'd argue that adding --install crossed the line from dependency/build manager to package manager; and brought along all the associated expectations.

[eternaleye]

@CamJN
I'm not so sure it did. After all, cargo install installs:

• (mostly) static binaries
• no dependencies - installs only the binaries of the current crate
• no libraries

As a result, it fundamentally doesn't support any kind of system-wide dependency handling, package conflict management, etc. That was a design decision explicitly so as to avoid crossing that line.

@alexcrichton
I'd argue that --reinstall is problematic for the same reasons I mentioned having issues with (some implementations of) --uninstall in my earlier comment that you linked.

If you're not doing it based on what you would install, you need to install metadata, which has a number of pitfalls (which I brought up in the other thread, IIRC). If you're doing it based on what you would install, but not from the exact same repository, revision, and build config as the original, you may generate a different list of things you would install, and then fail to remove files that were installed.

I would, however, support a proposal to add an --uninstall that:

1. Is run from the exact source dir the installed binaries were built from
2. Works by generating a list of would-install files and then removing them
3. Preferably checks its work, possibly by examining the config hash that I know is in libs and I hope is in executables

From there, it would be possible to --uninstall the old, followed by installing the new. Managing the process of keeping the old source around is then an exercise for the user (as would be needed on, say, LFS when using Autotools' make install and make uninstall, which my suggestion mimics) or a proper package manager.

EDIT: Agh, forgot that the metadata thing actually happened, and does get installed. In that case, so long as this doesn't require new metadata, I have no objections as at least it doesn't make it worse. However, the fact that the metadata gets written in-place to a single file (.crates.toml), rather than per-crate files, is still very hostile to package managers, and was one of the pitfalls I explicitly mentioned 😞

[gkoz]

I should probably add that --replace proposed in #2405 is not meant as a substitute for --update: it doesn't care about versions at all and --update might still need --replace for cases of two crates installing identically named binaries.

If you're doing it based on what you would install, but not from the exact same repository, revision, and build config as the original, you may generate a different list of things you would install, and then fail to remove files that were installed.

So the proposed implementation removes the old crate completely if neither --bin nor --example is supplied.

[gkoz]

I would, however, support a proposal to add an --uninstall that:

This does not compute in the presence of cargo uninstall.

[oconnor663]

I don't use make install very much, so correct me if this is mistaken: doesn't make install usually overwrite existing files by default? It sounds like the consensus in the original RFC might've been for something different than what actually got implemented (a make install that does not overwrite existing).

My main question is, is there much of a use case for a non-overwriting cargo install? I admit there's going to be a selection bias here, since people happy with the current behavior aren't going to find this issue, but does anyone currently rely on the non-overwriting default behavior? It's possible we don't need a flag at all.

(I'm not really thinking about the case where two different packages install conflicting files. My guess is that's rare enough / complicated enough that it's fine to require manual intervention.)

[eternaleye]

@gkoz: Yeah, see edit. It's been long enough I'd gone fuzzy on some of the details, including that one of my worst-cases (single-file metadata shared among all packages) came to pass.

[Stebalien]

My original objection was with the global metadata file. I wanted cargo install to be a drop in replacement for make install: install some binaries in $prefix/bin, optionally install some data files in $prefix/share/crate, don't update any global databases, and don't bother tracking anything. This way, installs of non-conflicting packages would be composable and cargo install could be used by packagers. Given that we now track existing packages and their installed files anyways, I don't see any reason not to use this information.

[CamJN]

I know it doesn't support those things right now, and honestly I'm glad of it. However I'd be willing to bet that if you asked a bunch of programmers what they would expect cargo to do considering it has a cargo install function you'd get a lot of descriptions of a package manager, for better or worse.

So while limiting this functionality is a good thing from a technical point of view, I think that there are already expectations that cargo will grow to become a full package manager. People might even sympathize with the complexity of what they really want, but I expect a lot of "can I just have this one small super-useful feature" requests. Or people that come looking for functionality they expect and get turned-off if they are told it purposefully doesn't exist. I'm not sure of the histories of rubygems and npm but I imagine they had well scoped goals at the beginning too.

There might be something you could do to manage expectations when someone uses cargo install, or maybe rename it to something like build-and-move-into-path.

[eternaleye]

@Stebalien
Note that $prefix is insufficient for it to be usable by packagers - the distinction between $prefix and $destdir is essential as well.

[Stebalien]

@eternaleye good point.

[alexcrichton]

@jarcane

Ah yeah sorry to clarify I was thinking that we'd add one of --replace/--reinstall/--uninstall today, not necessarily all at once! We may not even end up needing all of them.

@CamJN

I agree with @eternaleye that we've currently avoided becoming a full-blown package manager, and the intention of cargo install is just to easily share Rust programs, so I don't think that will change any time soon. I suspect that with proper documentation and discipline we can continue along this trajectory, but if we're leaning too much in one direction that'd be good to know!

@eternaleye, @Stebalien

Oh right! I realize now that I forgot to add the flag that indicates "don't generate a global manifest". I believe that assuages the concerns you had for packaging?

In terms of what we have today, though, it sounds like you're on board that if there's a manifest we may as well use it? @gkoz brings up a good point that --replace is different than --update which is something to be considered as well.

@oconnor663

I'd personally prefer that a tool warned me somehow that a binary was going to be overwritten, I suspect it could be easy to bite someone by accident if we just overwrote existing files (even if we mentioned we were doing so)

[mattico]

I agree that we don't want cargo to become a complete package manager, but having install without uninstall or --update is just silly. In my experience, many of the binaries I've installed don't work with new rustc nightlies (rustfmt, clippy, racer). What this means is instead of my rust update process being rustup update cargo install --update, it has to be rustup update rm which racer `rm `which cargo-clippy ... cargo install racer ...

Of course I could script this, and I have, but having it built in would be a huge quality of life improvement for me.

Or maybe I'm missing something that you naysayers use to make this process easier ;)

[eternaleye]

I use multirust, which puts the binaries in version-based locations, such as .multirust/toolchains/beta/cargo/bin/cargo-lichking - I've had no difficulties at all.

EDIT:
Also, @alexcrichton - a flag to not generate any manifest is better than the current state of affairs, but better still would be to have per-crate manifest files that can be merged from $destdir to $root like any other file, rather than a single manifest in $root that is updated on install.

I find it somewhat ironic that Cargo - being a dependency manager for Rust - was so quick to reach for shared mutable state, and it's decidedly suboptimal.

If manifest files were non-colliding and per-crate, there would be no need for a flag to disable manifests - any more than such a flag is needed with Haskell.

To explain my terms somewhat:

• $root describes the "top-of-namespace" - for system-wide installs this is /, for user installs this really ought to have been ~/.local/ (...but instead Cargo rolled its own)
• $prefix, $bindir, $datarootdir, etc. describe the desired layout of files relative to $root in the final install. A common set of values for system installs would be /usr, $prefix/bin, and $prefix/share respectively. These are well-documented by autotools, among others.
• $destdir is a "staging" directory - files are written here in the same structure as they would be written to $root, so as to allow package managers to wrap up the artifacts in a tidy bundle with a bow on it, which can - in theory - be deployed by simple file copy operations from $destdir to $root

A global manifest file violates this by both reading from and writing to $root, a read-modify-write operation that is impossible to reconcile with concurrency, reproducibility, etc. However, per-crate files that are "drop-in" additions to a shared directory are perfectly fine. They do not depend on $root, and fit perfectly into the $destdir model - a model all package managers I am aware of rely on.

Meanwhile, eliminating the manifests altogether means accepting that cargo install --list cannot be aware of system-installed utilities, which is again suboptimal.

In my opinion, manifests aren't the problem - they're just data. Making that data both shared and mutable, on the other hand...

[netvl]

@mattico, note that there is uninstall. However, this does not really make up for the absence of --update/--reinstall.

[eternaleye]

@alexcrichton Should this really have been closed? c.f. #2405 (comment):

[...] [This ticket is] pretty orthogonal from the --update flag.

[janhohenheim]

@alexcrichton Are there any plans on merging cargo-update's functionality into cargo?
When I try to introduce Rust to my acquaintances, this issue always comes up sooner or later, and I have a hard time telling them that manually running cargo install --force for every outdated crate is the official way to go.

[alexcrichton]

Perhaps eventually! At this point it mostly just needs someone to do the legwork to get it done.

[illicitonion]

FWIW I put together a tiny crate which just wraps cargo install called cargo-ensure-installed which attempts to pin installed binaries at specific versions, rather than keep them as up to date as they can be.

[gibix]

I think this ca be closed by multiple discussions

[eddyp]

Edition 2018 is here, and this issue would be a very nice addition.

As suggested, a cargo install --update-all option which would make sure all packages are to the latest version.

Also, the default behaviour of not updating an older version of a binary package (e.g. cargo-binutils) and, instead, suggesting force is not user friendly at all. I would say force should be reserved only for some --reinstall option, as a last resort to bring the system to a sane state (assuming an abnormal state was detected).

[ehuss]

Pinging anyone following this issue, I have a proposal to add upgrade functionality to cargo install at #6667. If you have any comments, I would appreciate if you can leave some over on that issue.

[elichai]

Any updates on this? seems like such a basic feature to have cargo install --update A that installs A only if there are new updates.

[ehuss]

@elichai There is a nightly-only flag that implements this, see #6797.

[emilioplatzer]

So. It is no longer a missing feature?. Look at https://rust-cli.github.io/book/tutorial/packaging.html

[webern]

So. It is no longer a missing feature?. Look at https://rust-cli.github.io/book/tutorial/packaging.html

I don't see anything at that link that solves the problem of needing cargo install --upgrade something.

[fsevenm]

Now 2021, I still can't find a way to update.

[Eh2406]

If the package is already installed, Cargo will reinstall it if the installed version does not appear to be up-to-date.

from https://doc.rust-lang.org/cargo/commands/cargo-install.html

[fsevenm]

Oh, thanks for the information. I missed that one.

[eddyp]

TBH, I am confused. I really do not understand how is the "cargo install --update-all" scenario addressed in any way, even after looking at other tickets, including the stabilization one.

I simply don't get how can one go from the existing cargo install --list to something with the meaning of cargo install --update-all.

@ehuss was your proposal from #6667 and implementation from #6798 ever targeting that? Or was the stabilization #7560 not covering that part?

[Nemo157]

That seems like it needs a separate issue, it wasn't the main point of this issue.

For now on unix you can easily turn cargo install --list into an --update-all (no guarantees given that this will remain working in the future, cargo install --list output is probably not intended to be machine-parseable, does not work with crates like cargo-lock that require --features to be specified):

cargo install --list | grep -v '^ ' | cut -d' ' -f 1 | xargs cargo install

[eddyp]

For now on unix you can easily turn cargo install --list into an --update-all (no guarantees given that this will remain working in the future, cargo install --list output is probably not intended to be machine-parseable, does not work with crates like cargo-lock that require --features to be specified):

cargo install --list | grep -v '^ ' | cut -d' ' -f 1 | xargs cargo install

I'm not saying I'm not able to apply some bash-fu to do this myself, that was not an issue, the point was to make such a thing work for every system, since a tool can end up in the system via cargo install, it should be possible to have an equivalent of apt upgrade in cargo to keep the tools and binaries installed via cargo up to date.

For instance, on a system I see there are some tools I have, but it's not clear if they are up to date:

C:\>cargo install --list
cargo-add v0.2.0:
    cargo-add.exe
cargo-asm v0.1.16:
    cargo-asm.exe
    cargo-llvm-ir.exe
cargo-audit v0.11.2:
    cargo-audit.exe
cargo-binutils v0.1.6:
    cargo-nm.exe
    cargo-objcopy.exe
    cargo-objdump.exe
    cargo-profdata.exe
    cargo-readobj.exe
    cargo-size.exe
    cargo-strip.exe
cargo-bloat v0.9.3:
    cargo-bloat.exe
cargo-deny v0.6.6:
    cargo-deny.exe
....

I was thinking since the --update-all idea was the second comment in the issue and has 128 likes, this was a main factor in the implementation choices and was just adding to the initial proposal.

And the way I was thinking it would have worked was something like this (NOT AN ACTUAL LISTING):

C:\>cargo install --update-all
cargo-add v0.2.0: up to date
cargo-asm v0.1.16: up to date
cargo-audit v0.11.2: outdated, installing v0.14.1
cargo-binutils v0.1.6: outdated, instaling v0.3.3
cargo-bloat v0.9.3: outdated, installing v0.10.0
cargo-deny v0.6.6: outdated, installing v0.9.1
...

I'll look for a dedicated issue asking the same thing, and create one, if there isn't one.