Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Clique Support #1032

Merged
merged 50 commits into from
Jan 28, 2021
Merged
Changes from 1 commit
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
05b0efa
common -> clique: added consensus config to chain json files, new Com…
holgerd77 Jan 6, 2021
9572564
common: renamed test files to conform with the *.spec.ts filename con…
holgerd77 Jan 6, 2021
d6d7c30
block -> clique: added isEpochTransition() method to Block and BlockH…
holgerd77 Jan 6, 2021
1712049
block: do not allow passing in uncle header on a PoA network
holgerd77 Jan 6, 2021
63c2362
block -> clique: added header timestamp validation checks for clique
holgerd77 Jan 6, 2021
a017648
block -> clique: added extraData length checks for block header, adde…
holgerd77 Jan 6, 2021
7cd888d
block -> clique: added helper functions cliqueExtraVanity(), cliqueEx…
holgerd77 Jan 9, 2021
0873bb7
block -> clique: added checks for coinbase and mixHash
holgerd77 Jan 9, 2021
2df16e0
block -> clique: adopted hash() method for clique blocks, added DRAFT…
holgerd77 Jan 9, 2021
a28851b
blockchain -> clique: data structure preparations for signer lists an…
holgerd77 Jan 10, 2021
0aa3f29
blockchain -> clique: draft signer integration
holgerd77 Jan 10, 2021
29c11e6
block -> clique: moved extra data checks from constructor to validate…
holgerd77 Jan 11, 2021
1557f5c
blockchain -> clique: init blockchain with the state of active signer…
holgerd77 Jan 11, 2021
396c896
blockchain -> clique: added block signature verification and activate…
holgerd77 Jan 11, 2021
b1bf363
blockchain, block: added basic voting structure, voting test setup fr…
holgerd77 Jan 11, 2021
8b82e3a
blockchain -> clique: do not check total difficulty on poa chain when…
holgerd77 Jan 11, 2021
31160ac
blockchain -> clique: added signer voting logic
holgerd77 Jan 12, 2021
222fbfb
blockchain, block -> clique: additional voting test cases, v value fi…
holgerd77 Jan 12, 2021
b6b8bed
block -> clique: fixed BlockHeader.hash() function, moved clique test…
holgerd77 Jan 12, 2021
f9c6157
blockchain -> clique: reset votes on epoch transition blocks, added m…
holgerd77 Jan 12, 2021
4828d1d
vm -> clique: use signer address for the COINBASE opcode for transact…
holgerd77 Jan 12, 2021
ee6902c
only calculate td for ethash consensus
ryanio Jan 19, 2021
8bfafd5
catch clique db gets to return empty on NotFoundError
ryanio Jan 19, 2021
cac505b
remove need for CLIQUE_EMPTY_BENEFICIARY (can use `Address.isZero` in…
ryanio Jan 19, 2021
014e22d
add helper: cliqueSaveGenesisSigners
ryanio Jan 19, 2021
7c8f722
rename `_checkClique` to `_requireClique` to be more self descriptive
ryanio Jan 19, 2021
0641bb8
move DBTarget enums for clique to bottom of list to not disrupt prior…
ryanio Jan 19, 2021
726102b
blockheader: add helper method `cliqueHash()`
ryanio Jan 20, 2021
f57febd
validate extraData, re-enable timestamp checks
ryanio Jan 20, 2021
a6f8ea7
rename cliqueSignatureToAddress() -> cliqueSigner(): Address
ryanio Jan 20, 2021
33d5894
use Address in clique types
ryanio Jan 20, 2021
4aea0d5
lint:fix
ryanio Jan 20, 2021
0411971
move DBSetTd inside if(ethash)
ryanio Jan 22, 2021
74618be
use non-clique chain for mocked test
ryanio Jan 22, 2021
00dc55a
use exported clique nonce constants
ryanio Jan 22, 2021
6d723ce
add block poa header tests:
ryanio Jan 22, 2021
edab558
valiidate checkpoint signers toward active signers
ryanio Jan 22, 2021
fee1381
trim clique signer states and latest votes to CLIQUE_SIGNER_HISTORY_B…
ryanio Jan 22, 2021
def4177
validate goerli poa block (additional fix: pass options correctly to …
ryanio Jan 22, 2021
ba4bb83
lint:fix
ryanio Jan 23, 2021
48da84e
add comments with clique type annotations
ryanio Jan 26, 2021
d69444f
add validation for clique difficulty
ryanio Jan 26, 2021
2ad875f
update param for initWithSigners to use Signer[] to dedupe code in te…
ryanio Jan 26, 2021
9ec8583
fix typo
ryanio Jan 27, 2021
3f983e4
improve clique vote calculations to pass test cases in EIP-225:
ryanio Jan 27, 2021
d9e5558
* add CliqueLatestBlockSigners for snapshot for calculating error "re…
ryanio Jan 27, 2021
6cbebb3
move clique constants to dedicated file
ryanio Jan 27, 2021
8c81c69
clique: add reorg logic and tests
ryanio Jan 28, 2021
141159c
update package-lock
ryanio Jan 28, 2021
6721321
fix vm test (use pow chain instead of poa chain, was throwing on sign…
ryanio Jan 28, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 24 additions & 8 deletions packages/blockchain/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -393,16 +393,23 @@ export default class Blockchain implements BlockchainInterface {
await this.cliqueUpdateVotes()
}

private async cliqueUpdateSignerStates(signerState?: CliqueSignerState) {
private async cliqueUpdateSignerStates(signerState: CliqueSignerState) {
const dbOps: DBOp[] = []
if (signerState) {
this._cliqueLatestSignerStates.push(signerState)
const formatted = this._cliqueLatestSignerStates.map((state) => [
state[0].toBuffer(),
state[1].map((a) => a.toBuffer()),
])
dbOps.push(DBOp.set(DBTarget.CliqueSignerStates, rlp.encode(formatted)))
this._cliqueLatestSignerStates.push(signerState)

// trim length to CLIQUE_SIGNER_HISTORY_BLOCK_LIMIT
const length = this._cliqueLatestSignerStates.length
const limit = this.CLIQUE_SIGNER_HISTORY_BLOCK_LIMIT
if (length > limit) {
this._cliqueLatestSignerStates = this._cliqueLatestSignerStates.slice(length - limit, length)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My idea for CLIQUE_SIGNER_HISTORY_BLOCK_LIMIT was to have a defined period of blocks (therefore the name) and delete the old signer states once these number of blocks have passed. Not sure, this version of just keeping some number of signer states should work as well - I guess - but then I would rename the constant.

Also maybe 100 might be a bit too much then? But not sure, maybe that's ok, shouldn't be too much extra data.

}

const formatted = this._cliqueLatestSignerStates.map((state) => [
state[0].toBuffer(),
state[1].map((a) => a.toBuffer()),
])
dbOps.push(DBOp.set(DBTarget.CliqueSignerStates, rlp.encode(formatted)))

await this.dbManager.batch(dbOps)
}

Expand Down Expand Up @@ -460,12 +467,21 @@ export default class Blockchain implements BlockchainInterface {
}
}
}

// trim length to CLIQUE_SIGNER_HISTORY_BLOCK_LIMIT
const length = this._cliqueLatestVotes.length
const limit = this.CLIQUE_SIGNER_HISTORY_BLOCK_LIMIT
if (length > limit) {
this._cliqueLatestVotes = this._cliqueLatestVotes.slice(length - limit, length)
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was my initial thought as well to trim both the signer states AND the votes (sorry, just looked it up, also still had this as a comment in along CLIQUE_SIGNER_HISTORY_BLOCK_LIMIT), but the votes actually can't/shouldn't be discarded, these can always still be relevant for vote count (not even directly after an epoch transition block since there can incidentally be a reorg directly upon/around an epoch transition block).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I guess for the votes we might want to define a safe-number-of-blocks waiting periods (longest anticipated reorg block counts) and then discard the votes from before the epoch transition.

Copy link
Contributor

@ryanio ryanio Jan 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right now all votes are being cleared on epoch transitions:

if (this._common.consensusAlgorithm() === 'clique') {
// Reset all votes on epoch transition blocks
if (header.cliqueIsEpochTransition()) {
this._cliqueLatestVotes = []
await this.cliqueUpdateVotes()

which is correct as per this part of the spec:

Attack vector: Spamming signer

... The solution is to place a moving window of W blocks after which votes are considered stale. We’ll call this an epoch.

Keeping in mind reorgs would be important though as you mention, but I have to think more about how we would manage that.

Perhaps we could keep some history of stale votes, but the vote counting logic in cliqueUpdateVotes should only count votes from the last active epoch in case of this edge scenario? starts getting a bit tricky 🤔

(edit: I added this to the new follow up PR)

Copy link
Member Author

@holgerd77 holgerd77 Jan 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't actually think it's too tricky: if there is a reorg, then all vote and signer state entries with block numbers higher than the reorg block number will be deleted, the signers from the last singer state stack item will be become the new active signers. Everything else from that moment on then continues (restarts) as before.

If I don't get this wrong, votes need to be kept relatively long in this scenario, so i think "EPOCH_LENGTH + REORG_SAFE_PERIOD" block numbers - so something like 30050, since if a reorg is happening from let's say 30020 -> 29950 and there is voting taking place in these last 50 blocks before the epoch transition, theoretically all existing votes from this 0-29999 epoch might be relevant for the pre-epoch voting. Does this make sense?

For Signer States I would think - all states from the last REORG_SAFE_PERIOD blocks + minimally one state - should be enough if I am not forgetting something.

But I think we can also generally be a bit more generous here - e.g. take a somewhat longer REORG_SAFE_PERIOD of 100 or something or keep more signer states, nothing lost here, this doesn't take that much space and eventually the date might be interesting for analytical purposes.


const dbOps: DBOp[] = []
const formatted = this._cliqueLatestVotes.map((v) => [
v[0].toBuffer(),
[v[1][0].toBuffer(), v[1][0].toBuffer(), v[1][2]],
])
dbOps.push(DBOp.set(DBTarget.CliqueVotes, rlp.encode(formatted)))

await this.dbManager.batch(dbOps)
}

Expand Down