Reduce network bandwidth, improve parablock times: optimize approval-…

…distribution (#5164) * gossip-support: be explicit about dimensions * some guide updates * update network-bridge to distinguish x and y dimensions * get everything to compile * beginnings * some TODOs * polkadot runtime: use relevant_authorities * make gossip topologies per-session * better formatting * gossip support: use current session validators * expand in comment * adjust tests and fix index bug * add past/present/future connection test and clean up code * fmt * network bridge: updated types * update protocols to new gossip topology message * guide updates * add session to BlockApprovalMeta * add session to block info * refactor knowledge and remove most unify logic * start replacing gossip_peers with new SessionTopologies * add routing information to message state * add some utilities to SessionTopology * implement new gossip topology logic * re-implement unify_with_peer * distribute assignments according to topology * finish grid topology implementation * refactor network bridge slightly * issue connection requests on all past/present/future * fmt * address grumbles * tighten invariants in unify_with_peer * implement random propagation * refactor: extract required routing adjustment logic * some block-age logic * aggressively propagate messages when finality is slow * overhaul aggression system to have 3 levels * add aggression metrics * remove aggression L3 * reduce random circulation * remove PeerData * get approval tests compiling * use btree_map in known_by to make deterministic * Revert "use btree_map in known_by to make deterministic" This reverts commit 3a3a4cc. * test XY grid propagation * remove stray println * test unshared dimension propagation * add random gossip check * test unify_with_peer better * test sending after getting gossip topology * test L1 aggression on originator * test L1 aggression for non-originators * test non-originator aggression L2 * fnt * ~spellcheck * fix statement-distribution tests * fix flaky test * fix metrics typo * re-send periodically * test resending * typo Co-authored-by: Bernhard Schuster <bernhard@ahoi.io> * add more metrics about apd messages * add back unify_with_peer logs * make Resend an enum * be more explicit when resending * fmt * fix error * add a TODO for refactoring * remove debug metrics * add some guide stuff * fmt * update runtime API in test-runtim Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
paritytech · Apr 19, 2022 · 8bb84d2 · 8bb84d2
1 parent dc248c0
commit 8bb84d2
Show file tree

Hide file tree

Showing 25 changed files with 2,554 additions and 490 deletions.
diff --git a/Cargo.lock b/Cargo.lock
diff --git a/node/core/approval-voting/src/import.rs b/node/core/approval-voting/src/import.rs
@@ -574,6 +574,7 @@ pub(crate) async fn handle_new_head(
 			parent_hash: block_header.parent_hash,
 			candidates: included_candidates.iter().map(|(hash, _, _, _)| *hash).collect(),
 			slot,
+			session: session_index,
 		});
 
 		imported_candidates.push(BlockImportedCandidates {

diff --git a/node/core/approval-voting/src/lib.rs b/node/core/approval-voting/src/lib.rs
@@ -1003,6 +1003,7 @@ fn distribution_messages_for_activation(
 			parent_hash: block_entry.parent_hash(),
 			candidates: block_entry.candidates().iter().map(|(_, c_hash)| *c_hash).collect(),
 			slot: block_entry.slot(),
+			session: block_entry.session(),
 		});
 
 		for (i, (_, candidate_hash)) in block_entry.candidates().iter().enumerate() {

diff --git a/node/network/approval-distribution/Cargo.toml b/node/network/approval-distribution/Cargo.toml
@@ -10,11 +10,13 @@ polkadot-node-network-protocol = { path = "../protocol" }
 polkadot-node-subsystem = { path = "../../subsystem" }
 polkadot-node-subsystem-util = { path = "../../subsystem-util" }
 polkadot-primitives = { path = "../../../primitives" }
+rand = "0.8"
 
 futures = "0.3.21"
 gum = { package = "tracing-gum", path = "../../gum" }
 
 [dev-dependencies]
+sp-authority-discovery = { git = "https://github.com/paritytech/substrate", branch = "master" }
 sp-core = { git = "https://github.com/paritytech/substrate", branch = "master", features = ["std"] }
 
 polkadot-node-subsystem-util = { path = "../../subsystem-util" }
@@ -23,5 +25,6 @@ polkadot-node-subsystem-test-helpers = { path = "../../subsystem-test-helpers" }
 assert_matches = "1.4.0"
 schnorrkel = { version = "0.9.1", default-features = false }
 rand_core = "0.5.1" # should match schnorrkel
+rand_chacha = "0.3.1"
 env_logger = "0.9.0"
 log = "0.4.16"