Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Approve multiple candidates with a single signature #7554

Open
wants to merge 62 commits into
base: sandreim/vrf_modulo_comapct_assignment
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
08c2381
Approve multiple candidates with a single signature
alexggh Jul 27, 2023
4901fee
Add Runtime configuration for max_approval_coalesce_count/wait_millis
alexggh Jul 28, 2023
1e3b511
Add zombienet for approval-coalescing
alexggh Jul 28, 2023
cf68a10
Refactor timers to address review comments
alexggh Jul 31, 2023
fd4b906
Make wait time for caching relative to no-show
alexggh Aug 1, 2023
2271c71
Enforce target candidate is part of the signature
alexggh Aug 2, 2023
df410ff
Merge remote-tracking branch 'origin/sandreim/vrf_modulo_comapct_assi…
alexggh Aug 2, 2023
f0f05c6
Fixup runtime configuration after rebase
alexggh Aug 2, 2023
76a10c6
Merge remote-tracking branch 'origin/sandreim/vrf_modulo_comapct_assi…
alexggh Aug 2, 2023
d31d1d0
Fixup build for unittests
alexggh Aug 2, 2023
6dfaecb
Fix bugs discovered during zombie-testing and unittest
alexggh Aug 3, 2023
1a2a5ec
Refactor the implementation a bit
alexggh Aug 3, 2023
81eb942
Make test better
alexggh Aug 8, 2023
ff0b35d
approval-voting: fix migration from v1 to v2
alexggh Aug 8, 2023
2570c1e
Fix-up bugs in assignnments_coalescing of tranche0
alexggh Aug 8, 2023
305a43a
Enable v2 network protocol
alexggh Aug 8, 2023
5bb1e91
Fixup test builds
alexggh Aug 8, 2023
a83cfb7
Enable assignments v2 computing
alexggh Aug 8, 2023
98705e2
Update test params
alexggh Aug 10, 2023
3f28a5d
Merge remote-tracking branch 'origin/sandreim/vrf_modulo_comapct_assi…
alexggh Aug 10, 2023
aae2a4a
Fixup logic for restarting the node
alexggh Aug 10, 2023
435d46a
Add metric to measure impact of coalescing of approvals
alexggh Aug 10, 2023
9016644
Add extra logs to improve debugability in versi
alexggh Aug 11, 2023
1662e14
Fixup sending approval before assignment
alexggh Aug 11, 2023
af57831
Count needed approvals by more than one third
alexggh Aug 15, 2023
be65f42
Add a few more metrics to understand versi behaviour
alexggh Aug 15, 2023
3ef84bf
Don't send assignments to peers that are not part of the authorities …
alexggh Aug 17, 2023
1dbcb90
Count all observe no-shows
alexggh Aug 17, 2023
3ac044e
Fix warning
alexggh Aug 17, 2023
2cb6756
Minor fixes for review
alexggh Aug 17, 2023
0429549
Fix approval-distribution tests
alexggh Aug 18, 2023
76823a4
Address initial review feedback
alexggh Aug 18, 2023
deff64f
Add zombient test 0006-approval-voting-coalescing to pipeline
alexggh Aug 18, 2023
62a57b5
Update implementers guide
alexggh Aug 18, 2023
f38697e
Fix importing of approval out-of-view
alexggh Aug 19, 2023
903cef2
PVF worker: Prevent access to env vars (#7330)
mrcnski Aug 21, 2023
ea027a8
Companion for substrate#14776 (#7648)
skunert Aug 21, 2023
0bbe0a7
PVF worker: random fixes (#7649)
mrcnski Aug 21, 2023
7bdf7dc
Merge remote-tracking branch 'origin/sandreim/vrf_modulo_comapct_assi…
alexggh Aug 22, 2023
e39c003
Add BEEFY capabilities to Westend and Kusama (#7591)
acatangiu Aug 22, 2023
8ce1716
Bump actions/setup-node from 3.8.0 to 3.8.1 (#7639)
dependabot[bot] Aug 22, 2023
3264a00
Merge remote-tracking branch 'origin/sandreim/vrf_modulo_comapct_assi…
alexggh Aug 22, 2023
9f1e9ea
Bound number of assets which can be withdrawn to pay for execution. (…
gavofyork Aug 22, 2023
d5fc80c
Enable approval-coalescing by default
alexggh Aug 22, 2023
301dab1
Increase num-workers
alexggh Aug 22, 2023
d861d81
Increase SMALL_POV_SIZE
alexggh Aug 22, 2023
a216036
Put logs on trace
alexggh Aug 22, 2023
759fe21
Fix xcm-builder mock (#7652)
ggwpez Aug 22, 2023
a908447
Merge remote-tracking branch 'origin/sandreim/vrf_modulo_comapct_assi…
alexggh Aug 23, 2023
e8ac355
Do not cleanup peer-knowledge on peer-connected/disconnected
alexggh Aug 23, 2023
7f0eb5f
Companion: restructure macro related exports (#7626)
juangirini Aug 23, 2023
7972d31
Revert some hacks
alexggh Aug 23, 2023
e988782
Merge remote-tracking branch 'origin/sandreim/vrf_modulo_comapct_assi…
alexggh Aug 23, 2023
feefbbb
Build image with network vstaging
alexggh Aug 23, 2023
71c08a4
Fixup sending approvals more than once
alexggh Aug 23, 2023
0349441
Fixup unify_with_peer
alexggh Aug 23, 2023
52034bd
Bump chevdor/srtool-actions from 0.7.0 to 0.8.0 (#7660)
dependabot[bot] Aug 24, 2023
ebb610f
Bump rustls-webpki from 0.101.2 to 0.101.4 (#7653)
dependabot[bot] Aug 24, 2023
f3da93d
extend abridged host config (#7659)
slumber Aug 24, 2023
598f4c6
cli: disallow BEEFY and warp sync together (#7661)
acatangiu Aug 25, 2023
52209dc
update readme: archived repo (#7654)
the-right-joyce Aug 25, 2023
a218e59
Merge remote-tracking branch 'origin/master' into approve_multiple_ca…
alexggh Aug 25, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ fn make_block_entry(
approved_bitfield: make_bitvec(candidates.len()),
candidates,
children: Vec::new(),
candidates_pending_signature: Default::default(),
}
}

Expand Down
15 changes: 13 additions & 2 deletions node/core/approval-voting/src/approval_db/v2/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ use polkadot_node_primitives::approval::{v1::DelayTranche, v2::AssignmentCertV2}
use polkadot_node_subsystem::{SubsystemError, SubsystemResult};
use polkadot_node_subsystem_util::database::{DBTransaction, Database};
use polkadot_primitives::{
BlockNumber, CandidateHash, CandidateReceipt, CoreIndex, GroupIndex, Hash, SessionIndex,
ValidatorIndex, ValidatorSignature,
BlockNumber, CandidateHash, CandidateIndex, CandidateReceipt, CoreIndex, GroupIndex, Hash,
SessionIndex, ValidatorIndex, ValidatorSignature,
};

use sp_consensus_slots::Slot;
Expand Down Expand Up @@ -238,6 +238,17 @@ pub struct BlockEntry {
// block. The block can be considered approved if the bitfield has all bits set to `true`.
pub approved_bitfield: Bitfield,
pub children: Vec<Hash>,
// A list of candidates that has been approved, but we didn't not sign and
// advertise the vote yet.
pub candidates_pending_signature: BTreeMap<CandidateIndex, CandidateSigningContext>,
}

#[derive(Encode, Decode, Debug, Clone, PartialEq)]

/// Context needed for creating an approval signature for a given candidate.
pub struct CandidateSigningContext {
pub candidate_hash: CandidateHash,
pub approved_time_since_unix_epoch: u128,
}

impl From<crate::Tick> for Tick {
Expand Down
1 change: 1 addition & 0 deletions node/core/approval-voting/src/approval_db/v2/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ fn make_block_entry(
approved_bitfield: make_bitvec(candidates.len()),
candidates,
children: Vec::new(),
candidates_pending_signature: Default::default(),
}
}

Expand Down
80 changes: 80 additions & 0 deletions node/core/approval-voting/src/approvals_timers.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
// Copyright (C) Parity Technologies (UK) Ltd.
// This file is part of Polkadot.

// Polkadot is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.

// Polkadot is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.

// You should have received a copy of the GNU General Public License
// along with Polkadot. If not, see <http://www.gnu.org/licenses/>.

//! A simple implementation of a timer per block hash .
//!
use std::{collections::HashSet, task::Poll, time::Duration};

use futures::{
future::BoxFuture,
stream::{FusedStream, FuturesUnordered},
Stream, StreamExt,
};
use futures_timer::Delay;
use polkadot_primitives::{Hash, ValidatorIndex};
// A list of delayed futures that gets triggered when the waiting time has expired and it is
// time to sign the candidate.
// We have a timer per relay-chain block.
#[derive(Default)]
pub struct SignApprovalsTimers {
timers: FuturesUnordered<BoxFuture<'static, (Hash, ValidatorIndex)>>,
blocks: HashSet<Hash>,
}

impl SignApprovalsTimers {
/// Starts a single timer per block hash
///
/// Guarantees that if a timer already exits for the give block hash,
/// no additional timer is started.
pub fn maybe_start_timer_for_block(
&mut self,
timer_duration_ms: u32,
block_hash: Hash,
validator_index: ValidatorIndex,
) {
if self.blocks.insert(block_hash) {
let delay = Delay::new(Duration::from_millis(timer_duration_ms as _));
self.timers.push(Box::pin(async move {
delay.await;
(block_hash, validator_index)
}));
}
}
}

impl Stream for SignApprovalsTimers {
type Item = (Hash, ValidatorIndex);

fn poll_next(
mut self: std::pin::Pin<&mut Self>,
cx: &mut std::task::Context<'_>,
) -> std::task::Poll<Option<Self::Item>> {
let poll_result = self.timers.poll_next_unpin(cx);
match poll_result {
Poll::Ready(Some(result)) => {
self.blocks.remove(&result.0);
Poll::Ready(Some(result))
},
_ => poll_result,
}
}
}

impl FusedStream for SignApprovalsTimers {
fn is_terminated(&self) -> bool {
self.timers.is_terminated()
}
}
2 changes: 2 additions & 0 deletions node/core/approval-voting/src/import.rs
Original file line number Diff line number Diff line change
Expand Up @@ -511,6 +511,7 @@ pub(crate) async fn handle_new_head<Context, B: Backend>(
.collect(),
approved_bitfield,
children: Vec::new(),
candidates_pending_signature: Default::default(),
};

gum::trace!(
Expand Down Expand Up @@ -1264,6 +1265,7 @@ pub(crate) mod tests {
candidates: Vec::new(),
approved_bitfield: Default::default(),
children: Vec::new(),
candidates_pending_signature: Default::default(),
}
.into(),
);
Expand Down
Loading