[Feat]: Publish nightly pre-compiled binaries, make releases atomic

[Veetaha]

Summary

This is based on the PR comment #284 (comment)

We could have a mutable master pre-release that we update every time a new commit is merged to master. For example, like it is done here.
Then the master installation script would install the true latest dev version of marker pre-compiled binaries, and it can use the latest toolchain version we have in the repo.

For example, if someone reported a bug, and we fixed it in master, we closed the issue. Then we can ask the user who reported the issue to check if it works for them by installing the master build of marker binaries and using a git dependency in their marker_api, and if the problem still exists they can reopen the issue after that testing.

Release atomicity

The challenge I see here is the atomicity of the release. A similar problem actually exists in our current stable release flow. However, it is not that big because a stable release is a big and rather rare event, and we also never mutate an existing release, we create a new one. It may happen that someone who uses a sliding v0.x tag in their github action can temporarily observe the state when the release git tag was put, but no pre-compiled binaries are available yet.

That isn't a big problem, and should be alleviated by #274 although not fully solved. In that issue, I suggested using a long retry policy for 404 errors, which will make everyone who wants to install the new sliding-tag release wait until the binaries are built. That's not ideal because once every marker release unlucky people will see an additional ~5-6 minutes of latency in their CI jobs although they will eventually succeed.

The real solution to that problem for stable releases is to move the sliding tags only after the GitHub release is built and is ready for consumption.

A bigger problem will be in the mutable master release because it observes a mid-release state much more often.

The way to solve this problem that I suggest is inspired by the transaction log concept.

1. First we need to do all the non-atomic actions, which are creating a new GitHub release, and building pre-compiled binaries.
2. While all that happens the GitHub release must be viewed as a temporary pre-release (not the latest one).
3. When the binaries are built and the release is ready we mark it as the latest one, and move the sliding tags to this one.

The users will use the previous release in steps 1 and 2, and once step 3 is done they will use the new ready-made release immediately.

In case of a master release we'll need to create a temporary release with a different name e.g. with tmp in its name, and then delete the previous master release and remove the commit hash from the release name once finished.

[xFrednet]

I like the idea of "nightly" builds, but dislike the spam that they would create, if we have them in the main repo. For example, I was subscribed to the releases of Rust-Analyzer. Since they publish a nightly version every day, it meant that I got a notification every day, that just wasn't interesting to me. This caused me to unsubscribe again, since GitHub sadly has no filtering for this kind of thing. I'm now watching the https://github.com/rust-lang/rust-analyzer to test if prereleases are included in the notifications.

One option could be, to have the prereleases/nightly builds in a dedicated repo or probably preferable use a different channel to inform users of new releases. A small note in the CLI or a post on reddit¹ for every release could be an option.

How would you recommend that we handle crates.io? Should we publish a dev/nightly version on there as well, or ask users to use git with something like a nightly tag?

Footnotes

1. Just as a side note, I delayed the announcement on Reddit as I have exams coming up and there hasn't been any time to take care of that. It's still on the todo list, for some time during this release, probably in one or two weeks. ↩

[Veetaha]

How would you recommend that we handle crates.io? Should we publish a dev/nightly version on there as well, or ask users to use git with something like a nightly tag?

I've been thinking of doing releases on every push to master. Although, that would be quite racy. For example, if someone decides to try master and it happens so that we merge a new commit to master that isn't released yet at that moment, they will observe a drift.

A scheduled per-24-hour will not suffer from that problem.

A nightly git tag should be fine. The one who wants to try the nightly version of the API crate will need to use a git dependency. I can't imagine a scenario where we publish a new version of the crates to crates.io every day. Not sure if this is fine with their ToC and I'd view it as abuse.

This caused me to unsubscribe again, since GitHub sadly has no filtering for this kind of thing

Yeah, I didn't think about that. According at the workflows in rust-analyzer they have a single nightly pre-release that they mutate every day. Hopefuly, editing a release doesn't cause a notification, but it needs to be checked.

Unfortunately, the current github action that we use for creating a release with a changelog entry in it is quite limited, but it's rather simple and could be implemented without a github action.

[xFrednet]

Yeah, I didn't think about that. According at the workflows in rust-analyzer they have a single nightly pre-release that they mutate every day. Hopefuly, editing a release doesn't cause a notification, but it needs to be checked.

I got a new notification tonight, about the nightly release. But it might be GH subscribed to me to nightly release itself, meaning that this notification was not a new release but just an update to an old one. I've unsubbed from the specific nightly release. Let's see what'll happen now :)

[Veetaha]

Is there an option to unsubscribe from a specific release? I don't see it

[xFrednet]

I unsubscribed in the notifications overview. So maybe, or it might be a dummy, for UI/UX reasons ^^

[xFrednet]

Today I got another notification about the new nightly version, so it seems like there is no way to avoid that -.-