Align dates in rustup toolchains with those in rustc -V

[SimonSapin]

Note to future readers: this PR was reverted after being merged, because it broke the Nightly release process.

---

rustup update tells me that the current Nightly is rustc 1.24.0-nightly (cfba0d446 2017-12-05). This is the output of rustc --version. It’s tempting to think that I can pin to this version by specifying a nightly-2017-12-05 toolchain, but that actually refers to yesterday’s Nightly, rust version 1.24.0-nightly (8503b3ff8 2017-12-04). This discrepancy also makes statements like “I can reproduce bug X in Nightly 2017-12-05” ambiguous.

The reason is that rustc -V contains the commit date of whatever happened to be the latest commit in master when a Nightly was published, but the archive URL (and so the toolchain names) contains the date that is current at that time. Since publication is started at midnight UTC, the latest commit is almost always "yesterday" and the dates end up off by one.

(This is technically a race condition but one that commits can never win since testing a PR takes a couple hours. Bors makes a merge commit before testing, and (if successful) pushes to master afterwards. So the latest commit in master is always at least a couple hours old.)

This PR changes the archive URL / toolchain name date to that of "10 minutes ago" instead of "now", soon after Cron starts this program at midnight UTC. That way, the date will most often match that the latest commit. (It can still be off if nothing was merged the day before, but given PR traffic on rust-lang/rust that’s relatively rare.)

This change should probably not be deployed without precaution since at the transition we’d end up with two nightlies with the same date and same URLs. The second one would probably overwrite the first, which is unexpected for dated archive URLs that have been so far immutable. Instead, we may want to skip one nightly (maybe by temporarily disabling the crontab) so that for example nightly-2017-12-07 is published at the end of that day (soon after midnight) and nightly-2017-12-08 after the end of that day, about 48 hours later.

[alexcrichton]

Talked with @SimonSapin and this seemed like a good idea! I figured we should add a comment as to why this argument is being passed (probably referencing this PR) and then I'll get around to deploying this when I get a chance.

[SimonSapin]

Added a code comment.

[alexcrichton]

I'm reverting this as nightlies are currently broken. Turns out this happens as well.

[SimonSapin]

curl -s https://static.rust-lang.org/dist/2017-12-20/channel-rust-nightly.toml | egrep 'version|date' has consistent dates which is what I wanted to achieve, but I don’t understand what happened or how that six-months-old line is relevant.

[alexcrichton]

The manifest was reuploaded after I found the breakage. The problem was that the all artifacts were uploaded to one date and the manifest listed another date. The manifest at https://static.rust-lang.org/dist/2017-12-19/channel-rust-nightly.toml pointed to nonexistent artifacts when it was the only manifest. At that time all artifacts were also uploaded under 2017-12-19

[SimonSapin]

So will future toolchain dates stay consistent with commit dates, or is that today only because of the manual fix up? If so, what else needs to be changed?

[alexcrichton]

The problem is that the current date is loaded in two locations and the two dates are supposed to be equal, with this PR, however, the dates were different. I believe the fix would be to ensure that both dates loaded are the same.

At this point though I'm getting very wary of these changes. I (apparently) don't have the time to properly evaluate, test, and deploy them. It's ending up causing a lot of churn trying to fix everything as a fire rather than preemptively.

[SimonSapin]

The off-by-one dates are a “paper cut” and I thought fixing it would be nice to have, but ultimately it’s not very important. Sorry I did not anticipate that this would be more complicated than a one-line change.

Hopefully in time more people will be involved in taking care of all this infrastructure. We can revisit this then.

[SimonSapin]

@Mark-Simulacrum Is this something that the new Release team could take over?

[Mark-Simulacrum]

Maybe. I see central-station as probably mostly infra team maintained, with perhaps overall direction provided by release team. I don't think we have a clear distinction of responsibilities that's been agreed on by all parties, but that's how we see it right now.

[RalfJung]

The off-by-one dates are a “paper cut” and I thought fixing it would be nice to have, but ultimately it’s not very important.

I agree it's not very important, but what is somewhat relevant is, given a nightly installed via rustup, to determine "how can I install this exact same version on another system"? rustup update prints that information when the update is being installed, but that doesn't help when that terminal has already been closed.

The most convenient way to fix this would be to make the rustc --version and the toolchain version match, but any other way to print the toolchain version would also help (at least when people know about it).

[fintelia]

I'd argue that --version should really be the only command to ask tools about their version information, and that if there are two relevant dates we should just include both in the output. Something like:

rustc 1.28.0-nightly (41affd03e 2018-06-04). Compiled on YYYY-MM-DD. 

[SimonSapin]

@RalfJung I think nobody is saying that there is nothing to fix. It probably takes someone from the Infra team to take the time to make all the pieces fall together to do this without breaking things. I’ve filed rust-lang/rust#51764 so that it’s hopefully more on the team’s radar than a merged and reverted PR.

@fintelia The date already shown by rustc --version is that of the merge commit, which is created just before the CI jobs are started to compile and test a PR before merging. So it is pretty much already “compiled on”. Later Nightly is created by taking the output of those CI jobs for the most recently-merged PR, and the date of that instant is the one that rustup knows about. We can’t include that second date after the fact in the rustc build, since the build is already done.