Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[resharding] Nightshade V3 shard layout for protocol version 65 #10725

Merged
merged 7 commits into from
Mar 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions chain/chain/src/store.rs
Original file line number Diff line number Diff line change
Expand Up @@ -607,6 +607,7 @@ impl ChainStore {
) -> Result<(), Error> {
tracing::trace!(target: "resharding", ?protocol_version, shard_id, receipts_shard_id, "reassign_outgoing_receipts_for_resharding");
// If simple nightshade v2 is enabled and stable use that.
// Same reassignment of outgoing receipts works for simple nightshade v3
if checked_feature!("stable", SimpleNightshadeV2, protocol_version) {
Self::reassign_outgoing_receipts_for_resharding_v2(
receipts,
Expand Down
3 changes: 3 additions & 0 deletions core/primitives-core/src/version.rs
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,8 @@ pub enum ProtocolFeature {
/// Resharding V2. A new implementation for resharding and a new shard
/// layout for the production networks.
SimpleNightshadeV2,
/// Built on top of Resharding V2. Changes shard layout to V3 to split shard 2 into two parts.
SimpleNightshadeV3,
tayfunelmas marked this conversation as resolved.
Show resolved Hide resolved
/// In case not all validator seats are occupied our algorithm provide incorrect minimal seat
/// price - it reports as alpha * sum_stake instead of alpha * sum_stake / (1 - alpha), where
/// alpha is min stake ratio
Expand Down Expand Up @@ -184,6 +186,7 @@ impl ProtocolFeature {
ProtocolFeature::RestrictTla
| ProtocolFeature::TestnetFewerBlockProducers
| ProtocolFeature::SimpleNightshadeV2 => 64,
ProtocolFeature::SimpleNightshadeV3 => 65,
Copy link
Contributor Author

@shreyan-gupta shreyan-gupta Mar 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be noted, STABLE_PROTOCOL_VERSION is already set as 65 on master. We may need to change this while cherry picking to 1.38.0


// StatelessNet features
ProtocolFeature::StatelessValidationV0 => 80,
Expand Down
5 changes: 5 additions & 0 deletions core/primitives/src/epoch_manager.rs
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,11 @@ impl AllEpochConfig {
}

fn config_nightshade(config: &mut EpochConfig, protocol_version: ProtocolVersion) {
if checked_feature!("stable", SimpleNightshadeV3, protocol_version) {
Self::config_nightshade_impl(config, ShardLayout::get_simple_nightshade_layout_v3());
return;
}

if checked_feature!("stable", SimpleNightshadeV2, protocol_version) {
Self::config_nightshade_impl(config, ShardLayout::get_simple_nightshade_layout_v2());
return;
Expand Down
59 changes: 59 additions & 0 deletions core/primitives/src/shard_layout.rs
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,24 @@ impl ShardLayout {
)
}

/// Returns the simple nightshade layout, version 3, that will be used in production.
pub fn get_simple_nightshade_layout_v3() -> ShardLayout {
ShardLayout::v1(
vec![
"aurora",
"aurora-0",
"game.hot.tg",
"kkuuue2akv_1630967379.near",
"tge-lockup.sweat",
]
.into_iter()
.map(|s| s.parse().unwrap())
.collect(),
Some(vec![vec![0], vec![1], vec![2, 3], vec![4], vec![5]]),
3,
)
}

/// Given a parent shard id, return the shard uids for the shards in the current shard layout that
/// are split from this parent shard. If this shard layout has no parent shard layout, return None
pub fn get_children_shards_uids(&self, parent_shard_id: ShardId) -> Option<Vec<ShardUId>> {
Expand Down Expand Up @@ -568,6 +586,7 @@ mod tests {
let v0 = ShardLayout::v0(1, 0);
let v1 = ShardLayout::get_simple_nightshade_layout();
let v2 = ShardLayout::get_simple_nightshade_layout_v2();
let v3 = ShardLayout::get_simple_nightshade_layout_v3();

insta::assert_snapshot!(serde_json::to_string_pretty(&v0).unwrap(), @r###"
{
Expand Down Expand Up @@ -638,5 +657,45 @@ mod tests {
}
}
"###);
insta::assert_snapshot!(serde_json::to_string_pretty(&v3).unwrap(), @r###"
{
"V1": {
"boundary_accounts": [
"aurora",
"aurora-0",
"game.hot.tg",
"kkuuue2akv_1630967379.near",
"tge-lockup.sweat"
],
"shards_split_map": [
[
0
],
[
1
],
[
2,
3
],
[
4
],
[
5
]
],
"to_parent_shard_map": [
0,
1,
2,
2,
3,
4
],
"version": 3
}
}
"###);
}
}
3 changes: 3 additions & 0 deletions core/store/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -235,10 +235,13 @@ impl Default for StoreConfig {
(ShardUId { version: 1, shard_id: 3 }, bytesize::ByteSize::gb(3)),
// In simple nightshade v2 the heavy contract "token.sweat" is in shard 4
(ShardUId { version: 2, shard_id: 4 }, bytesize::ByteSize::gb(3)),
// In simple nightshade v3 the heavy contract "token.sweat" is in shard 5
(ShardUId { version: 3, shard_id: 5 }, bytesize::ByteSize::gb(3)),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirming, shard_id gets pushed by one here

// Shard 1 is dedicated to aurora and it had very few cache
// misses even with cache size of only 50MB
(ShardUId { version: 1, shard_id: 1 }, bytesize::ByteSize::mb(50)),
(ShardUId { version: 2, shard_id: 1 }, bytesize::ByteSize::mb(50)),
(ShardUId { version: 3, shard_id: 1 }, bytesize::ByteSize::mb(50)),
]),
shard_cache_deletions_queue_capacity: DEFAULT_SHARD_CACHE_DELETIONS_QUEUE_CAPACITY,
},
Expand Down
Loading