Consider an Async utility to encapsulate double-check registration

[djspiewak]

A relatively common pattern in lock-free structures is something like the following:

1. Check a condition. If true, then return a value
2. Register a callback for when the condition becomes true
3. Check condition again. If true, unregister the callback and return the value
4. If false, asynchronously suspend

The problem with doing this using async/async_ is there are two race conditions here. First, at step 3, the callback may be invoked at the same moment you try to short-circuit. In order to resolve this race, you must consider whether your callback condition is idempotent. If it is not, you have a problem and may lose data. (note that the callback will drop subsequent invocations, so it is idempotent in a sense, but subsequent data passed to the callback will be lost) This is not really something that can be directly resolved in this type of structure though, since the callback does not return a Boolean. If you need to artificially construct idempotency in this scenario, it is recommended that your registration itself contain an AtomicBoolean which can be subjected to a CAS prior to the invocation of the callback.

The second race condition, which is even subtler, is the fact that the conventional way of achieving this when the condition at step 3 is true (manually invoking the callback within the async block) creates a "gap" in cancelability. If the async registration is canceled somewhere in these steps, and the condition is true in step 3, that cancelation will be observed at the end of the registration block. This cancelation observation is quite intuitive in the "condition false" case, where we asynchronously suspend (step 4), and will generally be handled correctly (often by unregistering the callback). However, in the case where we manually completed and we already have a value, this type of thing can create deadlocks where none are expected.

The solution to this, at present, is to simply use cont directly. Since you then have control over the get action (the async suspension), you can branch around it and only suspend if the condition in step 3 is false. If the condition in step 3 is true, then you don't need to sequence get at all, which means you never suspend the fiber, and in turn also means that your cancelation properties remain air-tight. I didn't really think too much about this pattern until working on the Queue reworks, where it came up constantly, and now that I know to look for it, I see it all over the existing Cats Effect codebase (e.g. take a look at Dispatcher's use of async_). I think we should codify this pattern into something more usable than cont (though still built on it, obviously).

I'm thinking of a function signature something like the following:

// come up with a good name
def bippy[A](k: (Either[Throwable, A] => Unit) => F[Either[Option[F[Unit]], A]]): F[A]

In a sense, this generalizes the existing async a bit. If you produce a Right, the get won't be sequenced and the result is simply immediately returned. If you produce a Left then it behaves exactly like async. In fact, async itself should be reimplemented in terms of this slight generalization.