ci: persist Rust caches for Pip package build

[wchargin]

Summary:
The build-data-server-pip job runs on two platforms on every push, and
each run takes about 5 minutes. The vast majority of this time (85%) is
spent building all dependencies from scratch. Since macOS minutes are
priced at 10× the rate of Linux minutes and this is our only
macOS job, we may as well spend a modicum of effort to improve this.

The docs at https://github.com/actions/cache say that a repository can
have up to 5 GB of caches. The cache for our other Rust job is ~200 MB,
and we don’t have any other caches, so adding two more of those caches
seems okay. The caches are also invalidated infrequently: only when we
update Rust dependencies or update the workflow file itself.

Adding a cache should speed up these builds. We already do this for the
Rust lint job, and it works well. This is a separate cache because the
job builds in release mode rather than debug mode.

Rust builds generally have good hermeticity properties, so this should
be fairly safe. If we want to be extra safe, we could turn this off for
builds that will be published in an actual stable release.

Test Plan:
Note that the second run of the checks for this PR spends less time in
the “Build” and “Test” phases. It appears to still spend about a minute
total building, even though it hits cache. I’m not sure why—this is not
the case for normal, local, incremental builds—but this is still an
improvement of about 65%, so I’m okay with it for now.

wchargin-branch: ci-cache-rust-pip

[wchargin]

(Unassigning just because I haven’t finished the test plan yet.)

[nfelt]

I don't quite understand this comment. Could you elaborate slightly?

Update: read the other version of this comment which mentions cargo-raze so I assume this has something to do with raze generating these artifacts? Slightly more elaboration would still be useful IMO.

[wchargin]

Elaborated. This is needed only if you cargo install a binary, like
cargo-raze or hyperfine or something. Upstream documentation issue:
rust-lang/cargo#8841

We don’t actually need it for this job. I kept it to reduce headache in
case we later update it to cargo install. Clarified in comment.

[nfelt]

This also includes tensorboard/data/server/target/ which isn't the Cargo home directory (as I understand it) - maybe broaden description slightly?

[wchargin]

True, yes. Renamed in both cases. (I think “Cargo artifacts” is probably
fine, since it’s Cargo (not rustc) that sets up this structure. But
happy to entertain other suggestions.)

[wchargin]

thanks!

[wchargin]

True, yes. Renamed in both cases. (I think “Cargo artifacts” is probably
fine, since it’s Cargo (not rustc) that sets up this structure. But
happy to entertain other suggestions.)

[wchargin]

Elaborated. This is needed only if you cargo install a binary, like
cargo-raze or hyperfine or something. Upstream documentation issue:
rust-lang/cargo#8841

We don’t actually need it for this job. I kept it to reduce headache in
case we later update it to cargo install. Clarified in comment.

[nfelt]

Ack, thanks for explaining.

[wchargin]

I’ve just realized that this caching behavior is a little weird, albeit
benign. The cache key incorporates the Cargo.lock file and the CI
workflow file, not the actual Rust code. The cached data includes the
built dependencies, which depend only on Cargo.lock, and the cached
builds for our crates, rustboard_core and rustboard, which very
much depend on our code. So, on successive builds, the cache key may be
unchanged yet the cached outputs may differ. My understanding is that
when this happens, the cache is not updated.

I think that this is fine. Any change to the Rust code recompiles the
entire crate, so there’s no finer granularity possible. And it’s okay
that there’s not a functional dependency from the cache key to the exact
cache output, because any of the potential outputs for a given cache
key is an acceptable input for that cache key.

I’m inclined to take no action; just noting for posterity.