[hailtop] safely use chunks

[danking]

If we hit an exception and exit the iterator early, then we are no longer iterating. We need to record that fact so that we can retry transient errors.

[daniel-goldstein]

Is this idempotent? How does this prevent a double write on retrying?

[danking]

Hmm, idempotency is a bit hard to talk about here. This change makes it impossible to not "cleanup" the chunks iterator if you hit an exception midway through the chunks iterator. In particular, this now works:

try:
    with chunks(...) as data:
        raise ValueError()
except ValueError:
    pass
with chunks(...) as data:
    ... use data ...

In the current code, that does not work. The second call to chunks raises an error unless chunks is empty.

---

But you're probably asking about the code that uses chunks? In the Google case it is idempotent: lines 206-215 construct a new request before iterating chunks. The PUT request includes the specific range of bytes we want to write to, so even if we partially succeeded with a previous PUT, this subsequent PUT should overwrite (or, more likely, error). In practice, I don't think we can partially succeed. I think either we write fully or we terminate the connection early and google drops the data.

Summary: I think Google is fine.

As for Azure, we use a randomly generated block_id. If we error while inside stage_block that block_id is never added to self.block_ids. As a result, we can safely make a second attempt to upload the block with a new id.

x

[daniel-goldstein]

Thanks for the explanation, looks good