incoming / outgoing / sent block hooks called again for each block on restart

[dirkmc]

There are a couple of bugs open against lotus whereby the "transferred" field increases by the already sent/received data size each time a transfer is restarted: filecoin-project/lotus#5225 filecoin-project/lotus#5133

I believe the root cause is that when a data transfer is restarted, graphsync triggers hooks for all blocks that have already been sent or received for

• RegisterIncomingBlockHook
• RegisterOutgoingBlockHook
• RegisterBlockSentListener

go-data-transfer also watches for these events in order to detect whether a reconnect has occurred.

I think it would make most sense for these events to only be triggered when the block has actually been sent / received over the network.
Alternatively the event could still fire, but the event data could include a flag indicating whether the block was sent / received over the network.

[hannahhoward]

Hi @dirkmc,

From graphsync's point of view, these are two different and unrelated requests, with the second having a list of CID's not to send on the wire. I'm not sure what can be done in this case -- I think the architecture is correct for go-graphsync.

Two things you can do is keep a running total of bytes sent (since that is in the BlockData struct) and check the discrepency between BlockSize & BlockSizeOnWire (which will be different if the block was not sent on the wire)

[dirkmc]

It seems like graphsync is resending some blocks with a non-zero block.BlockSizeOnWire() after a restart. I wrote a test to capture the issue:
filecoin-project/go-data-transfer#140

[dirkmc]

Closing as the fix is at the go-data-transfer layer: filecoin-project/go-data-transfer#140