Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Benchmark paras inherent enter #4044

Closed
wants to merge 19 commits into from
Closed

Conversation

emostov
Copy link
Contributor

@emostov emostov commented Oct 9, 2021

WIP

rel: #3850

probably would be affected by #4020

@github-actions github-actions bot added the A3-in_progress Pull request is in progress. No review needed at this stage. label Oct 9, 2021
@emostov emostov added B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit. labels Oct 9, 2021
@emostov
Copy link
Contributor Author

emostov commented Oct 18, 2021

/benchmark runtime kusama runtime_parachains::paras_inherent

1 similar comment
@emostov
Copy link
Contributor Author

emostov commented Oct 18, 2021

/benchmark runtime kusama runtime_parachains::paras_inherent

@emostov emostov marked this pull request as ready for review October 18, 2021 20:32
@emostov
Copy link
Contributor Author

emostov commented Oct 18, 2021

/benchmark runtime kusama runtime_parachains::paras_inherent

@github-actions github-actions bot added A0-please_review Pull request needs code review. and removed A3-in_progress Pull request is in progress. No review needed at this stage. labels Oct 18, 2021
@joao-paulo-parity
Copy link
Contributor

/benchmark runtime kusama runtime_parachains::paras_inherent

@parity-benchapp
Copy link

parity-benchapp bot commented Oct 19, 2021

Benchmark Runtime Kusama Pallet for branch "zeke-benchmark-paras-inherent" with command cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras_inherent.rs

Results
2021-10-19 02:26:24 [0] 💸 generated 1 npos voters, 1 from validators and 0 nominators    
2021-10-19 02:26:24 Took active validators from set with wrong size    
2021-10-19 02:26:24 Took active validators from set with wrong size.    
2021-10-19 02:26:24 Took active validators from set with wrong size    
2021-10-19 02:26:25 Took active validators from set with wrong size    
2021-10-19 02:26:25 Took active validators from set with wrong size.    
2021-10-19 02:26:25 Took active validators from set with wrong size    
Error: 
   0: �[91mInvalid input: Error executing and verifying runtime benchmark: Other("Wasm execution trapped: wasm trap: unreachable\nwasm backtrace:\n    0: 0x5e9062 - <unknown>!rust_panic\n    1: 0x5e9058 - <unknown>!std::panicking::rust_panic_with_hook::hfafe25f4665a6ad6\n    2: 0x5e90a6 - <unknown>!std::panicking::begin_panic_handler::{{closure}}::h144b14ba4012f089\n    3: 0x5e9086 - <unknown>!std::sys_common::backtrace::__rust_end_short_backtrace::h3623ed8dcc8a9359\n    4: 0x33f8 - <unknown>!rust_begin_unwind\n    5: 0x313c - <unknown>!core::panicking::panic_fmt::hec165b0930ecb32e\n    6: 0x5e1052 - <unknown>!core::panicking::panic_str::haf0c39acf3aa63ee.26349\n    7: 0x5e1058 - <unknown>!<schnorrkel::rand_hack::PanicRng as rand_core::RngCore>::fill_bytes::hbe82d36553a342bc\n    8: 0x5e71db - <unknown>!schnorrkel::context::SigningTranscript::witness_scalar::h88da02a35e057f3a\n    9: 0x5e75b0 - <unknown>!schnorrkel::sign::<impl schnorrkel::keys::Keypair>::sign::h114e34adb368df2d\n   10: 0x3995e6 - <unknown>!<sp_core::sr25519::Pair as sp_core::crypto::Pair>::sign::h991f3ad05eff5bdc\n   11: 0x3993fa - <unknown>!polkadot_primitives::v1::signed::UncheckedSigned<Payload,RealPayload>::benchmark_sign::hf962e5d7b2f51dd2\n   12: 0x27f70a - <unknown>!polkadot_runtime_parachains::paras_inherent::benchmarking::BenchBuilder<T>::build::h2bc95d4b849ce145\n   13: 0x22b231 - <unknown>!<kusama_runtime::Runtime as frame_benchmarking::utils::runtime_decl_for_Benchmark::Benchmark<sp_runtime::generic::block::Block<sp_runtime::generic::header::Header<u32,sp_runtime::traits::BlakeTwo256>,sp_runtime::generic::unchecked_extrinsic::UncheckedExtrinsic<sp_runtime::multiaddress::MultiAddress<<<sp_runtime::MultiSignature as sp_runtime::traits::Verify>::Signer as sp_runtime::traits::IdentifyAccount>::AccountId,()>,kusama_runtime::Call,sp_runtime::MultiSignature,(frame_system::extensions::check_spec_version::CheckSpecVersion<kusama_runtime::Runtime>,frame_system::extensions::check_tx_version::CheckTxVersion<kusama_runtime::Runtime>,frame_system::extensions::check_genesis::CheckGenesis<kusama_runtime::Runtime>,frame_system::extensions::check_mortality::CheckMortality<kusama_runtime::Runtime>,frame_system::extensions::check_nonce::CheckNonce<kusama_runtime::Runtime>,frame_system::extensions::check_weight::CheckWeight<kusama_runtime::Runtime>,pallet_transaction_payment::ChargeTransactionPayment<kusama_runtime::Runtime>)>>>>::dispatch_benchmark::h4a5abc79b95789e1\n   14: 0x4291ae - <unknown>!Benchmark_dispatch_benchmark\n")�[0m

Backtrace omitted.
Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

ERROR: Unable to commit file ./runtime/kusama/src/weights/runtime_parachains_paras_inherent.rs

* WIP transition to signing with Public

* Add test ext to testing mock

* Maybe working? But so slow its hard to tell

* add some primitive debug logs
@emostov
Copy link
Contributor Author

emostov commented Oct 20, 2021

/benchmark runtime kusama runtime_parachains::paras_inherent

@parity-benchapp
Copy link

parity-benchapp bot commented Oct 20, 2021

Benchmark Runtime Kusama Pallet for branch "zeke-benchmark-paras-inherent" with command cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras_inherent.rs

Results
Pallet: "runtime_parachains::paras_inherent", Extrinsic: "enter_dispute_dominant", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParaInherent Included (r:1 w:1)
Storage: System ParentHash (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: ParaScheduler AvailabilityCores (r:1 w:1)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: Paras Parachains (r:1 w:0)
Storage: ParaInclusion PendingAvailability (r:41 w:40)
Storage: ParaInclusion PendingAvailabilityCommitments (r:40 w:40)
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: Session Validators (r:1 w:0)
Storage: ParasShared ActiveValidatorIndices (r:1 w:0)
Storage: Staking ActiveEra (r:1 w:0)
Storage: Staking ErasRewardPoints (r:1 w:1)
Storage: Dmp DownwardMessageQueues (r:40 w:1)
Storage: Hrmp HrmpChannelDigests (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:40 w:0)
Storage: ParaScheduler SessionStartBlock (r:1 w:0)
Storage: ParaScheduler ParathreadQueue (r:1 w:1)
Storage: ParaScheduler Scheduled (r:1 w:1)
Storage: ParaScheduler ValidatorGroups (r:1 w:0)
Storage: System BlockWeight (r:1 w:0)
Storage: Paras PastCodeMeta (r:40 w:0)
Storage: Paras CurrentCodeHash (r:40 w:0)
Storage: Ump RelayDispatchQueueSize (r:40 w:0)
Storage: Hrmp HrmpWatermarks (r:39 w:1)
Storage: Paras Heads (r:39 w:1)
Storage: Ump NeedsDispatch (r:1 w:1)
Storage: Ump NextDispatchRoundStartWith (r:1 w:1)
Storage: ParaInclusion AvailabilityBitfields (r:0 w:200)
Storage: ParaInherent OnChainVotes (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    28060
    + d        0
              µs

Reads = 378 + (0 * d)
Writes = 292 + (0 * d)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    d   mean µs  sigma µs       %
    0     27840     24.21    0.0%
    1     27770     33.83    0.1%
    2     27360     22.55    0.0%
    3     27060     34.77    0.1%
    4     26710     47.81    0.1%
    5     26410     32.27    0.1%
    6     26110     21.27    0.0%
    7     25780     34.56    0.1%
    8     25520     37.33    0.1%
    9     25140     24.08    0.0%
   10     24850     20.76    0.0%
   11     24440     42.93    0.1%
   12     24210      41.8    0.1%
   13     23900     41.58    0.1%
   14     23520     26.85    0.1%
   15     23190     47.66    0.2%
   16     22860     35.96    0.1%
   17     22510     49.08    0.2%
   18     22190     15.79    0.0%
   19     21920     46.08    0.2%
   20     21540     37.54    0.1%
   21     21170     26.51    0.1%
   22     20860     40.19    0.1%
   23     20510     21.19    0.1%
   24     20150     33.72    0.1%
   25     19900     36.28    0.1%
   26     19640     28.61    0.1%
   27     19320     48.75    0.2%
   28     18950     28.66    0.1%
   29     18640     30.95    0.1%
   30     18300     27.33    0.1%
   31     18010     43.54    0.2%
   32     17650     32.34    0.1%
   33     17350     20.47    0.1%
   34     16960     19.95    0.1%
   35     16660     13.85    0.0%
   36     16330     16.19    0.0%
   37     16050     35.11    0.2%
   38     15670     45.85    0.2%
   39     15350     34.78    0.2%
   40     14900     25.61    0.1%

Quality and confidence:
param     error
d         0.265

Model:
Time ~=    28050
    + d        0
              µs

Reads = 378 + (0 * d)
Writes = 292 + (0 * d)

Pallet: "runtime_parachains::paras_inherent", Extrinsic: "enter_disputes_only", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParaInherent Included (r:1 w:1)
Storage: System ParentHash (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: ParaScheduler AvailabilityCores (r:1 w:1)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: Paras Parachains (r:1 w:0)
Storage: ParaInclusion PendingAvailability (r:41 w:0)
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParaScheduler SessionStartBlock (r:1 w:0)
Storage: ParaScheduler ParathreadQueue (r:1 w:1)
Storage: ParaScheduler Scheduled (r:1 w:1)
Storage: ParaScheduler ValidatorGroups (r:1 w:0)
Storage: System BlockWeight (r:1 w:0)
Storage: Ump NeedsDispatch (r:1 w:1)
Storage: Ump NextDispatchRoundStartWith (r:1 w:1)
Storage: ParaInclusion AvailabilityBitfields (r:0 w:200)
Storage: ParaInherent OnChainVotes (r:0 w:1)
Storage: ParaInclusion PendingAvailabilityCommitments (r:1 w:1)
Storage: Session Validators (r:1 w:0)
Storage: ParasShared ActiveValidatorIndices (r:1 w:0)
Storage: Staking ActiveEra (r:1 w:0)
Storage: Staking ErasRewardPoints (r:1 w:1)
Storage: Dmp DownwardMessageQueues (r:1 w:1)
Storage: Hrmp HrmpChannelDigests (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:1 w:0)
Storage: Hrmp HrmpWatermarks (r:0 w:1)
Storage: Paras Heads (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    10840
    + d    103.8
              µs

Reads = 62 + (1 * d)
Writes = 212 + (2 * d)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    d   mean µs  sigma µs       %
    0     10770     21.06    0.1%
    1     10950     17.68    0.1%
    2     11040     8.284    0.0%
    3     11140     29.77    0.2%
    4     11270     23.85    0.2%
    5     11320      18.2    0.1%
    6     11460     22.05    0.1%
    7     11590     9.625    0.0%
    8     11650      28.1    0.2%
    9     11790     32.33    0.2%
   10     11900     27.64    0.2%
   11     12000     25.54    0.2%
   12     12060     32.95    0.2%
   13     12230     34.45    0.2%
   14     12340     20.88    0.1%
   15     12390     30.79    0.2%
   16     12510     28.83    0.2%
   17     12630     27.72    0.2%
   18     12710     17.84    0.1%
   19     12800     16.02    0.1%
   20     12940     49.64    0.3%
   21     13020     34.28    0.2%
   22     13090     26.16    0.1%
   23     13190     26.07    0.1%
   24     13410     26.32    0.1%
   25     13500      34.3    0.2%
   26     13600     31.87    0.2%
   27     13670     34.87    0.2%
   28     13760     17.08    0.1%
   29     13880     18.53    0.1%
   30     13950     18.39    0.1%
   31     14070     21.42    0.1%
   32     14200     32.44    0.2%
   33     14280      41.8    0.2%
   34     14390     27.38    0.1%
   35     14490     30.15    0.2%
   36     14540     34.42    0.2%
   37     14670     34.26    0.2%
   38     14750     33.26    0.2%
   39     14840     27.49    0.1%
   40     14960     51.13    0.3%

Quality and confidence:
param     error
d         0.178

Model:
Time ~=    10840
    + d    103.8
              µs

Reads = 61 + (1 * d)
Writes = 212 + (2 * d)

Pallet: "runtime_parachains::paras_inherent", Extrinsic: "enter_backed_dominant", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParaInherent Included (r:1 w:1)
Storage: System ParentHash (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: ParaInclusion PendingAvailability (r:41 w:40)
Storage: ParaScheduler AvailabilityCores (r:1 w:1)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: Paras Parachains (r:1 w:0)
Storage: ParaInclusion PendingAvailabilityCommitments (r:40 w:40)
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: Session Validators (r:1 w:0)
Storage: ParasShared ActiveValidatorIndices (r:1 w:0)
Storage: Staking ActiveEra (r:1 w:0)
Storage: Staking ErasRewardPoints (r:1 w:1)
Storage: Dmp DownwardMessageQueues (r:1 w:1)
Storage: Hrmp HrmpChannelDigests (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:1 w:0)
Storage: ParaScheduler SessionStartBlock (r:1 w:0)
Storage: ParaScheduler ParathreadQueue (r:1 w:1)
Storage: ParaScheduler Scheduled (r:1 w:1)
Storage: ParaScheduler ValidatorGroups (r:1 w:0)
Storage: System BlockWeight (r:1 w:0)
Storage: Ump NeedsDispatch (r:1 w:1)
Storage: Ump NextDispatchRoundStartWith (r:1 w:1)
Storage: ParaInclusion AvailabilityBitfields (r:0 w:200)
Storage: ParaInherent OnChainVotes (r:0 w:1)
Storage: Hrmp HrmpWatermarks (r:0 w:1)
Storage: Paras Heads (r:0 w:1)
Storage: Paras PastCodeMeta (r:1 w:0)
Storage: Paras CurrentCodeHash (r:1 w:0)
Storage: Ump RelayDispatchQueueSize (r:1 w:0)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    15030
    + b    325.6
              µs

Reads = 98 + (7 * b)
Writes = 292 + (0 * b)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    b   mean µs  sigma µs       %
    0     14890     19.72    0.1%
    1     15330     46.03    0.3%
    2     15620     29.33    0.1%
    3     16020      36.7    0.2%
    4     16320     28.07    0.1%
    5     16670     32.79    0.1%
    6     17010     42.39    0.2%
    7     17340     49.42    0.2%
    8     17660     26.01    0.1%
    9     17960     20.13    0.1%
   10     18330     26.08    0.1%
   11     18680     31.49    0.1%
   12     19030     49.28    0.2%
   13     19330     30.75    0.1%
   14     19650      42.3    0.2%
   15     19940     26.43    0.1%
   16     20190      53.1    0.2%
   17     20560     47.99    0.2%
   18     20860     43.91    0.2%
   19     21170      39.3    0.1%
   20     21560     32.42    0.1%
   21     21830     28.33    0.1%
   22     22190     27.11    0.1%
   23     22500     36.35    0.1%
   24     22830     44.56    0.1%
   25     23140     34.14    0.1%
   26     23540     47.45    0.2%
   27     23870     31.95    0.1%
   28     24220     42.57    0.1%
   29     24470     37.63    0.1%
   30     24750      38.1    0.1%
   31     25140     45.85    0.1%
   32     25480      24.5    0.0%
   33     25800     44.17    0.1%
   34     26140     46.13    0.1%
   35     26440     50.36    0.1%
   36     26740     36.86    0.1%
   37     27070     51.72    0.1%
   38     27390     64.47    0.2%
   39     27750     70.41    0.2%
   40     27860     41.13    0.1%

Quality and confidence:
param     error
b         0.277

Model:
Time ~=    15040
    + b    325.3
              µs

Reads = 98 + (7 * b)
Writes = 292 + (0 * b)

Pallet: "runtime_parachains::paras_inherent", Extrinsic: "enter_backed_only", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParaInherent Included (r:1 w:1)
Storage: System ParentHash (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: ParaScheduler AvailabilityCores (r:1 w:1)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: Paras Parachains (r:1 w:0)
Storage: ParaInclusion PendingAvailability (r:41 w:0)
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParaScheduler SessionStartBlock (r:1 w:0)
Storage: ParaScheduler ParathreadQueue (r:1 w:1)
Storage: ParaScheduler Scheduled (r:1 w:1)
Storage: ParaScheduler ValidatorGroups (r:1 w:0)
Storage: System BlockWeight (r:1 w:0)
Storage: Ump NeedsDispatch (r:1 w:1)
Storage: Ump NextDispatchRoundStartWith (r:1 w:1)
Storage: ParaInclusion AvailabilityBitfields (r:0 w:200)
Storage: ParaInherent OnChainVotes (r:0 w:1)
Storage: ParaInclusion PendingAvailabilityCommitments (r:1 w:1)
Storage: Session Validators (r:1 w:0)
Storage: ParasShared ActiveValidatorIndices (r:1 w:0)
Storage: Staking ActiveEra (r:1 w:0)
Storage: Staking ErasRewardPoints (r:1 w:1)
Storage: Dmp DownwardMessageQueues (r:1 w:1)
Storage: Hrmp HrmpChannelDigests (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:1 w:0)
Storage: Paras PastCodeMeta (r:1 w:0)
Storage: Paras CurrentCodeHash (r:1 w:0)
Storage: Ump RelayDispatchQueueSize (r:1 w:0)
Storage: Hrmp HrmpWatermarks (r:0 w:1)
Storage: Paras Heads (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    10900
    + b    424.5
              µs

Reads = 58 + (8 * b)
Writes = 212 + (2 * b)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    b   mean µs  sigma µs       %
    0     10850     19.57    0.1%
    1     11310     26.82    0.2%
    2     11760     27.56    0.2%
    3     12160      36.5    0.3%
    4     12620      23.4    0.1%
    5     13020     35.46    0.2%
    6     13480     59.98    0.4%
    7     13850     7.525    0.0%
    8     14310     28.69    0.2%
    9     14750      28.8    0.1%
   10     15170     32.24    0.2%
   11     15610     18.64    0.1%
   12     16000     32.55    0.2%
   13     16390     29.87    0.1%
   14     16820     56.57    0.3%
   15     17250     16.61    0.0%
   16     17690     61.69    0.3%
   17     18140     60.32    0.3%
   18     18530     49.01    0.2%
   19     18930     15.74    0.0%
   20     19350     32.22    0.1%
   21     19800     41.76    0.2%
   22     20210     31.38    0.1%
   23     20690     31.34    0.1%
   24     21100     41.07    0.1%
   25     21500      16.9    0.0%
   26     21950     49.89    0.2%
   27     22360     32.28    0.1%
   28     22820     46.26    0.2%
   29     23210     19.87    0.0%
   30     23600     46.96    0.1%
   31     24060     30.15    0.1%
   32     24500      33.4    0.1%
   33     24940     48.46    0.1%
   34     25360     40.29    0.1%
   35     25770     30.92    0.1%
   36     26170     23.37    0.0%
   37     26600     41.43    0.1%
   38     27000        49    0.1%
   39     27450     38.43    0.1%
   40     27890     48.23    0.1%

Quality and confidence:
param     error
b         0.184

Model:
Time ~=    10900
    + b    424.6
              µs

Reads = 58 + (8 * b)
Writes = 212 + (2 * b)


Parity Bot and others added 5 commits October 21, 2021 01:51
…k --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras_inherent.rs
@emostov
Copy link
Contributor Author

emostov commented Oct 28, 2021

/benchmark runtime polkadot runtime_parachains::paras_inherent

@parity-benchapp
Copy link

parity-benchapp bot commented Oct 28, 2021

Error running benchmark: zeke-benchmark-paras-inherent

stdoutFrom https://github.com/paritytech/polkadot * branch master -> FETCH_HEAD

@emostov
Copy link
Contributor Author

emostov commented Nov 1, 2021

superseded by #4202

@emostov emostov closed this Nov 1, 2021
@KiChjang KiChjang deleted the zeke-benchmark-paras-inherent branch November 1, 2021 21:10
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants