Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(COP): only in testnet: reduce the number of block producers #9563

Merged
merged 64 commits into from
Oct 11, 2023
Merged
Show file tree
Hide file tree
Changes from 48 commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
fa266eb
feat(COP): Increase the number of chunk-only producers in testnet
nikurt Sep 22, 2023
ce230e1
refactor: undo gas_limit renaming (#9558)
pugachAG Sep 22, 2023
3db13dc
perf: don't clone all incoming `PeerMessage`s (#9559)
jakmeier Sep 22, 2023
41a772e
Logging (2/2): Replace DelayDetector (#9543)
nikurt Sep 25, 2023
6b46020
RoutingTableView: add nonces, display tombstone edges separately (#9560)
saketh-are Sep 25, 2023
0a3c16d
chore: bump crossbeam-channel to 0.5.8 (#9592)
jakmeier Sep 26, 2023
96a8181
[cleanup] Remove redundant pattern of manually creating store, epoch …
Sep 26, 2023
97d5859
fix: remove nightly changes from parameters.snap (#9612)
jakmeier Sep 29, 2023
cb6a1f0
chore: bump rusqlite version (#9615)
jakmeier Sep 29, 2023
7dd8b32
[refactor] Break chain/client/src/test_utils.rs into module (#9611)
Sep 29, 2023
c18ebd9
ci: improve test resilience and run style checks within `cargo nextes…
nagisa Oct 2, 2023
2103b39
optimize curve25519 for tests (#9606)
Ekleog-NEAR Oct 2, 2023
cf4589a
refactor(state-sync): Rename module: `syncing` -> `state_sync` (#9594)
nikurt Oct 2, 2023
d2d28b4
chore: bump openssl version (#9614)
jakmeier Oct 2, 2023
9c25249
fix(state-viewer): fix balance overflow in state-dump command (#9546)
marcelo-gonzalez Oct 2, 2023
475bd2b
refactor(epoch-sync): Move EpochSync responses to a separate file (#9…
nikurt Oct 2, 2023
b39ca93
[resharding] Remove unit tests in favor of integration tests (#9597)
Oct 2, 2023
e2865be
[resharding] Use state sync snapshots to get flat storage iterator at…
Oct 2, 2023
a18c96f
feat(runtime): restrict creation of non-implicit TLA (#9589)
bowenwang1996 Oct 2, 2023
a611048
[resharding] Enable snapshot iterator for resharding (#9607)
Oct 2, 2023
2595c0a
chore(deps): bump urllib3 from 1.26.13 to 1.26.17 in /debug_scripts (…
dependabot[bot] Oct 3, 2023
00ac8ec
refactor(state-sync): Refer to other nodes by PeerId (#9591)
nikurt Oct 3, 2023
e15802d
feat: implement post-state-root chunk production (#9537)
pugachAG Oct 3, 2023
9192147
Add hash of the node to the MissingTrieValue (#9616)
staffik Oct 3, 2023
2a7d676
[memtrie] (1/n) Encoding and decoding of in-memory trie nodes. (#9541)
robin-near Oct 3, 2023
bc6acc0
[refactor] Move ShardTries snapshot related code to a separate file (…
Oct 4, 2023
5692811
add status field to response in RPC methods `tx`, `EXPERIMENTAL_tx_st…
telezhnaya Oct 4, 2023
50572d5
drop EXPERIMENTAL_check_tx RPC method (#9601)
telezhnaya Oct 4, 2023
0993de3
doc: fix typos in CONTRIBUTING.md (#9624)
i-fix-typos Oct 5, 2023
c0e5de7
feat: resharding - use full alphabet in tests (#9643)
wacban Oct 5, 2023
c265eb2
[resharding] Add logic to poll for snapshot before resharding (#9636)
Oct 6, 2023
da939ed
fix: always take gas price from previous block (#9638)
Longarithm Oct 6, 2023
889fd84
feat: Change block options for FlatState column (#9389)
Oct 6, 2023
7a15a73
ci: move a chunk of the Rust CI over to GHA (#9290)
nagisa Oct 7, 2023
db153fa
rust: upgrade to 1.73.0 (#9649)
nagisa Oct 7, 2023
c208642
ci: do not run the CI checks after a merge queue (#9652)
nagisa Oct 9, 2023
2b7de2f
jsonrpc: add tx serialisation tests (#9653)
telezhnaya Oct 9, 2023
0b55942
V2 Routing: Refresh distance vector nonces (#9651)
saketh-are Oct 9, 2023
54f0f7c
cleanup: removed mock-node (#9654)
wacban Oct 10, 2023
b9d2635
refactor: rename field to next_gas_price (#9641)
Longarithm Oct 10, 2023
0d1f9ed
chore: update borsh dependency (#9432)
dj8yfo Oct 10, 2023
d11fa4f
feat: stabilize NEP 492 (#9658)
bowenwang1996 Oct 10, 2023
89a8ae4
nit: always show full output in run_clippy.sh (#9659)
wacban Oct 11, 2023
28b6e89
Limit output for view-trie full mode (#9631)
staffik Oct 11, 2023
5506502
feat(COP): Increase the number of chunk-only producers in testnet
nikurt Sep 22, 2023
6f1d917
feat(COP): Increase the number of chunk-only producers in testnet
nikurt Oct 11, 2023
429391b
feat(COP): Increase the number of chunk-only producers in testnet
nikurt Oct 11, 2023
5bcd9b5
Merge branch 'master' into nikurt-more-cop
nikurt Oct 11, 2023
f696e63
refactor: Introduce constants for commonly used chain names (#9666)
nikurt Oct 11, 2023
f94e2dc
feat(COP): Increase the number of chunk-only producers in testnet
nikurt Sep 22, 2023
877f113
feat(runtime): restrict creation of non-implicit TLA (#9589)
bowenwang1996 Oct 2, 2023
09f7c6a
feat(COP): Increase the number of chunk-only producers in testnet
nikurt Oct 11, 2023
5682fd5
feat(COP): Increase the number of chunk-only producers in testnet
nikurt Oct 11, 2023
e403572
merge
nikurt Oct 11, 2023
dee2cc6
feat(COP): Increase the number of chunk-only producers in testnet
nikurt Sep 22, 2023
ae12179
[refactor] Break chain/client/src/test_utils.rs into module (#9611)
Sep 29, 2023
0f706d2
[resharding] Use state sync snapshots to get flat storage iterator at…
Oct 2, 2023
f9d7c8a
ci: move a chunk of the Rust CI over to GHA (#9290)
nagisa Oct 7, 2023
f419514
jsonrpc: add tx serialisation tests (#9653)
telezhnaya Oct 9, 2023
54d775d
chore: update borsh dependency (#9432)
dj8yfo Oct 10, 2023
f92c9a5
feat(COP): Increase the number of chunk-only producers in testnet
nikurt Oct 11, 2023
f37ce4f
feat(COP): Increase the number of chunk-only producers in testnet
nikurt Oct 11, 2023
c37e310
merge
nikurt Oct 11, 2023
dfd0420
Merge branch 'master' into nikurt-more-cop
nikurt Oct 11, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
### Protocol Changes
* The support for fixed shards in shard layout was removed. [#9219](https://github.com/near/nearcore/pull/9219)
* Restrict the creation of non-implicit top-level account that are longer than 32 bytes. Only the registrar account can create them. [#9589](https://github.com/near/nearcore/pull/9589)
* Adjust the number of block producers and chunk producers on testnet to facilitate testing of chunk-only producers [#9563](https://github.com/near/nearcore/pull/9563)


### Non-protocol Changes
Expand Down
2 changes: 1 addition & 1 deletion chain/epoch-manager/src/shard_tracker.rs
Original file line number Diff line number Diff line change
Expand Up @@ -237,7 +237,7 @@ mod tests {
};
EpochManager::new(
store,
AllEpochConfig::new(use_production_config, initial_epoch_config),
AllEpochConfig::new(use_production_config, initial_epoch_config, "test-chain"),
genesis_protocol_version,
reward_calculator,
vec![ValidatorStake::new(
Expand Down
2 changes: 1 addition & 1 deletion chain/epoch-manager/src/test_utils.rs
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ pub fn epoch_config_with_production_config(
shard_layout: ShardLayout::v0(num_shards, 0),
validator_max_kickout_stake_perc: 100,
};
AllEpochConfig::new(use_production_config, epoch_config)
AllEpochConfig::new(use_production_config, epoch_config, "test-chain")
}

pub fn epoch_config(
Expand Down
2 changes: 1 addition & 1 deletion chain/epoch-manager/src/tests/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2207,7 +2207,7 @@ fn test_protocol_version_switch_with_many_seats() {
validator_selection_config: Default::default(),
validator_max_kickout_stake_perc: 100,
};
let config = AllEpochConfig::new(false, epoch_config);
let config = AllEpochConfig::new(false, epoch_config, "test-chain");
let amount_staked = 1_000_000;
let validators = vec![
stake("test1".parse().unwrap(), amount_staked),
Expand Down
6 changes: 5 additions & 1 deletion core/chain-configs/src/genesis_config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -217,7 +217,11 @@ impl From<&GenesisConfig> for EpochConfig {
impl From<&GenesisConfig> for AllEpochConfig {
fn from(genesis_config: &GenesisConfig) -> Self {
let initial_epoch_config = EpochConfig::from(genesis_config);
let epoch_config = Self::new(genesis_config.use_production_config(), initial_epoch_config);
let epoch_config = Self::new(
genesis_config.use_production_config(),
initial_epoch_config,
&genesis_config.chain_id,
);
epoch_config
}
}
Expand Down
3 changes: 3 additions & 0 deletions core/primitives-core/src/version.rs
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,8 @@ pub enum ProtocolFeature {
/// Enables block production with post-state-root.
/// NEP: https://github.com/near/NEPs/pull/507
PostStateRoot,
/// Increases the number of chunk producers.
TestnetFewerBlockProducers,
}

impl ProtocolFeature {
Expand Down Expand Up @@ -181,6 +183,7 @@ impl ProtocolFeature {
#[cfg(feature = "protocol_feature_simple_nightshade_v2")]
ProtocolFeature::SimpleNightshadeV2 => 135,
ProtocolFeature::PostStateRoot => 136,
ProtocolFeature::TestnetFewerBlockProducers => 137,
}
}
}
Expand Down
32 changes: 28 additions & 4 deletions core/primitives/src/epoch_manager.rs
Original file line number Diff line number Diff line change
Expand Up @@ -83,11 +83,17 @@ pub struct AllEpochConfig {
use_production_config: bool,
/// EpochConfig from genesis
genesis_epoch_config: EpochConfig,
/// Chain Id. Some parameters are specific to certain chains.
chain_id: String,
}

impl AllEpochConfig {
pub fn new(use_production_config: bool, genesis_epoch_config: EpochConfig) -> Self {
Self { use_production_config, genesis_epoch_config }
pub fn new(
use_production_config: bool,
genesis_epoch_config: EpochConfig,
chain_id: &str,
Comment on lines +92 to +94
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not related but it would be cool to remove use_production_config as a follow up. I don't remember why but it really annoyed me.
Generally I love the idea of customizing EpochConfigs based on the chain. I don't know if the chain name is the best way to go about it but it is an improvement.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, but better to do it in a separate PR.

) -> Self {
Self { use_production_config, genesis_epoch_config, chain_id: chain_id.to_string() }
}

pub fn for_protocol_version(&self, protocol_version: ProtocolVersion) -> EpochConfig {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When testing on mocknet please remember to set use_production_config to true, otherwise your logic won't be triggered - see two lines below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Above you suggested to remove use_production_config. That should take care of mocknet configuration

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, as long as you do that first, you'll be fine. It will make my life easier as well when testing resharding.

Expand All @@ -98,7 +104,7 @@ impl AllEpochConfig {

Self::config_nightshade(&mut config, protocol_version);

Self::config_chunk_only_producers(&mut config, protocol_version);
Self::config_chunk_only_producers(&mut config, &self.chain_id, protocol_version);

Self::config_max_kickout_stake(&mut config, protocol_version);

Expand Down Expand Up @@ -126,7 +132,11 @@ impl AllEpochConfig {
config.avg_hidden_validator_seats_per_shard = vec![0; num_shards];
}

fn config_chunk_only_producers(config: &mut EpochConfig, protocol_version: u32) {
fn config_chunk_only_producers(
config: &mut EpochConfig,
chain_id: &str,
protocol_version: u32,
) {
if checked_feature!("stable", ChunkOnlyProducers, protocol_version) {
let num_shards = config.shard_layout.num_shards() as usize;
// On testnet, genesis config set num_block_producer_seats to 200
Expand All @@ -139,6 +149,20 @@ impl AllEpochConfig {
config.chunk_producer_kickout_threshold = 80;
config.validator_selection_config.num_chunk_only_producer_seats = 200;
}

// Adjust the number of block and chunk producers for all chains except
// mainnet, to make it easier to test the change.
if chain_id != "mainnet"
&& checked_feature!("stable", TestnetFewerBlockProducers, protocol_version)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will include mocknet.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, and that is intentional. I'd like to test this change in mocknet.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, okay but I am a bit weary about diverging mocknet and mainnet.

{
let num_shards = config.shard_layout.num_shards() as usize;
// Decrease the number of block producers from 100 to 20.
config.num_block_producer_seats = 20;
config.num_block_producer_seats_per_shard =
vec![config.num_block_producer_seats; num_shards];
// Decrease the number of chunk producers.
config.validator_selection_config.num_chunk_only_producer_seats = 100;
}
}

fn config_max_kickout_stake(config: &mut EpochConfig, protocol_version: u32) {
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/src/tests/client/resharding.rs
Original file line number Diff line number Diff line change
Expand Up @@ -742,7 +742,7 @@ fn setup_genesis(
genesis.config.protocol_upgrade_stake_threshold = Rational32::new(7, 10);

let default_epoch_config = EpochConfig::from(&genesis.config);
let all_epoch_config = AllEpochConfig::new(true, default_epoch_config);
let all_epoch_config = AllEpochConfig::new(true, default_epoch_config, "test-chain");
let epoch_config = all_epoch_config.for_protocol_version(genesis_protocol_version);

genesis.config.shard_layout = epoch_config.shard_layout;
Expand Down