Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

collation-generation + collator-protocol: collate on multiple assigned cores #3795

Merged
merged 31 commits into from
Mar 27, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
8f5d259
DistributeCollation includes CoreIndex
sandreim Mar 19, 2024
dc84796
SubmitCollationParams includes CoreIndex
sandreim Mar 19, 2024
665ef06
collator-protocol updates
sandreim Mar 19, 2024
78ee8b2
collation-generation pull support for chained collations
sandreim Mar 19, 2024
a61d64a
adjust validator buffer capacity for elastic scaling
sandreim Mar 20, 2024
1cbb090
fmt
sandreim Mar 20, 2024
493a7a2
Per core index tracking of multiple collations
sandreim Mar 20, 2024
8497413
fix tests
sandreim Mar 20, 2024
0dbc63a
enable collator-protocol elastic scaling extension for adder colaltor
sandreim Mar 22, 2024
3906d34
Use backing state to get candidates pending availability
sandreim Mar 22, 2024
eb7d842
Merge branch 'master' of github.com:paritytech/polkadot-sdk into sand…
sandreim Mar 22, 2024
d344a2d
use ok_or
sandreim Mar 22, 2024
16e1cb4
subsystem-util request_para_backing_state
sandreim Mar 22, 2024
91f9c40
Use the exposed claim queue
sandreim Mar 22, 2024
154f3dd
refactor
sandreim Mar 22, 2024
60ec79f
remove type annotation
sandreim Mar 22, 2024
a6c4afe
remove more merge damage
sandreim Mar 22, 2024
62c13dc
make tests compile
sandreim Mar 22, 2024
ddf057e
happy clippy is good clippy
sandreim Mar 22, 2024
13b149f
lookahead use core_index in SubmitCollationParams
sandreim Mar 25, 2024
64a53f4
taplo fix
sandreim Mar 25, 2024
732440b
enable elastic scaling in undying collator (also enables in zombienet…
sandreim Mar 25, 2024
f80897d
Collation generation test fixes and new tests
sandreim Mar 25, 2024
7d8e6b5
Merge branch 'master' of github.com:paritytech/polkadot-sdk into sand…
sandreim Mar 25, 2024
83af4e1
bring test-runtime up to date
sandreim Mar 25, 2024
9721b69
review feedback
sandreim Mar 25, 2024
4eb804e
await await
sandreim Mar 25, 2024
8b70bc2
review feedback
sandreim Mar 27, 2024
16607dd
prdoc
sandreim Mar 27, 2024
ba80669
Merge branch 'master' of github.com:paritytech/polkadot-sdk into sand…
sandreim Mar 27, 2024
69a63cc
nice new syntax
sandreim Mar 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion polkadot/node/collation-generation/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -305,7 +305,8 @@ async fn handle_new_activations<Context>(

let para_backing_state =
request_para_backing_state(relay_parent, config.para_id, ctx.sender())
.await.await??
.await
.await??
.ok_or(crate::error::Error::MissingParaBackingState)?;

// We are being very optimistic here, but one of the cores could pend availability some more
Expand Down
25 changes: 14 additions & 11 deletions polkadot/node/network/collator-protocol/src/collator_side/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -748,13 +748,20 @@ async fn advertise_collation<Context>(
}
}

let should_advertise = per_relay_parent
.validator_group
.get(&core_index)
.map_or(ShouldAdvertiseTo::NotAuthority, |group| {
group.should_advertise_to(candidate_hash, peer_ids, &peer)
});
let validator_group =
sandreim marked this conversation as resolved.
Show resolved Hide resolved
if let Some(validator_group) = per_relay_parent.validator_group.get_mut(&core_index) {
validator_group
} else {
gum::debug!(
target: LOG_TARGET,
?relay_parent,
?core_index,
"Skipping advertising to validator, validator group for core not found",
);
return
};

let should_advertise = validator_group.should_advertise_to(candidate_hash, peer_ids, &peer);
match should_advertise {
ShouldAdvertiseTo::Yes => {},
ShouldAdvertiseTo::NotAuthority | ShouldAdvertiseTo::AlreadyAdvertised => {
Expand Down Expand Up @@ -802,11 +809,7 @@ async fn advertise_collation<Context>(
))
.await;

per_relay_parent
.validator_group
.get_mut(&core_index)
.expect("should_advertise returned `ShouldAdvertiseTo::Yes` above; qed")
.advertised_to_peer(candidate_hash, &peer_ids, peer);
validator_group.advertised_to_peer(candidate_hash, &peer_ids, peer);

advertisement_timeouts.push(ResetInterestTimeout::new(
*candidate_hash,
Expand Down
14 changes: 14 additions & 0 deletions prdoc/pr_3795.prdoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
title: Enable collators to build on multiple cores

doc:
- audience: Node Dev
description: |
Introduces a `CoreIndex` parameter in `SubmitCollationParams`. This enables
the collators to make use of potentially multiple cores assigned at some relay
chain block. This extra parameter is used by the collator protocol and collation
generation subsystems to forward the collation to the approapriate backing group.

crates:
- name: polkadot-node-collation-generation
- name: polkadot-collator-protocol
bump: minor
Loading