IPC: Abstract common consensus functions and consensus interface

[adlrocha]

Related Issues

consensus-shipyard/lotus#1

Proposed Changes

This PR extends the consensus interface in order to prepare it for the implementation of other consensus algorithms. It also extracts from FilecoinEC common logic that will be shared by all consensus implementations.

• Extracted block checks and validation to common functions.
• Extracted ValidatedPubSub as a common function.
• Abstracted block signature and verification, and the validation of blocks and their headers.
• Extracted compute_state as common code for all consensus algorithms.
• Abstracted the application of rewards.

Integration of the interface

To illustrate how this interface can be integrated for the implementation of alternative consensus algorithm, I will share how I imagine it to work and be implemented for Mir:

• CreateBlock runs the block proposal logic for Mir in Lotus. In CreateBlock Lotus validators pull unverified messages from the mempool, and send them to Mir for ordering. Once the ordered batch is output by Mir, a new Filecoin block is assembled and broadcast through GossipSub to the network.
• Validators are not subscribed to the block gossipsub topic, as they already receive all the information about new blocks through Mir.
• Other Lotus nodes in the network will receive updates about new blocks being proposed by Mir through the Gossipsub block topic. When a new block is received, GossipSub triggers ValidateBlockHeader before delivering the block to the node. If the validation succeeds, the block is passed to the syncer which runs ValidateBlock to perform a full block validation to determine if the block is valid and can be executed. As part of these verifications, VerifyBlockSignature may be called.
• Finally, the execution of the block triggers the consensus-specific RewardFunc that parametrized how should block rewards be distributed among validators.

[magik6k]

Looks mostly good, just 1 small comment + CI is failing gen checks

[magik6k]

Looks good, just needs conflicts resolved

[adlrocha]

Conflicts should be fixed. CI is not happy but maybe it is a flaky test? It passed the last time it ran. Let me know if you need anything else from my side 🙏

[adlrocha]

@magik6k @jennijuju, are we still planning to get this merged? We want to make consensus-shipyard/lotus its own repo instead of an "official fork" of Lotus so we can run our own Circle CI pipelines and we are blocking this until this PR is merged (to avoid the corresponding overhead of re-spawning this PR). Thanks 🙏

[jennijuju]

@magik6k @jennijuju, are we still planning to get this merged? We want to make consensus-shipyard/lotus its own repo instead of an "official fork" of Lotus so we can run our own Circle CI pipelines and we are blocking this until this PR is merged (to avoid the corresponding overhead of re-spawning this PR). Thanks 🙏

@adlrocha seems like magik is okay with this PR, i just need to wait for either @magik6k / @arajasek to get this a final pass and put a ✅ on it, both of them are still OOO until later this week. We also have to prioritize on some blockers for nv18 & confirm branch management for nv18 release before we can work on landing this in master. Can we target next Friday, Jan 13th - does it work for your project?

[adlrocha]

Can we target next Friday, Jan 13th - does it work for your project?

Sure, no problem. I just wanted an estimate so we can pipeline the work. Thanks 🙏