Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

paras: include_pvf_check_statement rt bench #4938

Merged
merged 8 commits into from
Mar 24, 2022
Merged

paras: include_pvf_check_statement rt bench #4938

merged 8 commits into from
Mar 24, 2022

Conversation

pepyakin
Copy link
Contributor

@pepyakin pepyakin commented Feb 17, 2022 •
edited
Loading

Closes #4933

This PR adds a benchmark for the include_pvf_check_statement
dispatchable. This is a necessary step to make it work without
modifications. That enables us to proceed with testing on Versi.

This introduces 5 new benchmarks. Those measure performance of the
include_pvf_check_statement under 2 different conditions:

  1. regular vote submission. That's the common case.
  2. submission of the last vote. That happens only once and leads to a
    heavy finalization stage.

There are 2 different types of finalization (one for onboarding, one for
upgrading) and there are two outcomes: accepted and rejected. Those 4
are similar but I decided to cover them all and assign the maximum of
all 4. This is to avoid a situation when one of those paths becomes more
heavier than others and opens up an attack venue.

The regular vote submission weight is drastically different from the
submission last vote weight. That's why in case during runtime
finalization was not executed the weight consumed value will be lowered
down to the regular vote submission.

The finalization weight is proportional to the number of "causes", i.e.
the events that caused the PVF pre-checking vote in the first place, and
here we assume that the maximum number of causes is 100.

Theoretically, there is nothing that prevents an adversary to
register/upgrade to more than 100 parachains. In that case, the consumed
weight will be lower than the actual time consumed by the finalization
process. That can enable a DoS vector.

However, practically, it is not very possible. Right now it is very
expensive to call schedule_para_initialize because it requires a very
large lock up of funds. Moreover, finalizing a vote with 100 causes
leads to around 31ms time spent. Finalizing more will require more time.
However, finalizing with 200 causes will cause ≈62ms delay. This is not
that bad since even though we had a full block and the adversary tried
to finalize 200 causes it won't be able to even exceed the operational
extrinsic boundary of 250ms and even if so it won't make big difference.

That said, this should be addressed later on, esp. when we enable
parathreads, which will make creating causes easier. One of potential
solutions will be shifting the logic of finalization into
on_initialize/on_finalize. Another is to create a maximum number of
causes and then reject upgrades or onboardings if that was reached.

TODO:

  • Rerun benchmarks

@pepyakin
Copy link
Contributor Author

pepyakin commented Feb 17, 2022 •
edited
Loading

Current dependencies on/for this PR:

This comment was auto-generated by Graphite.

@pepyakin pepyakin mentioned this pull request Feb 17, 2022
Merged
@github-actions github-actions bot added the A0-please_review Pull request needs code review. label Feb 17, 2022
@pepyakin pepyakin added B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit. labels Feb 17, 2022
@pepyakin pepyakin force-pushed the pep-pvf-include-pvf-check-benchmark branch from 9520bed to 86256cf Compare February 17, 2022 16:23
@pepyakin
Copy link
Contributor Author

/benchmark runtime polkadot runtime_parachains::paras

@parity-benchapp
Copy link

parity-benchapp bot commented Feb 21, 2022 •
edited
Loading

Benchmark Runtime Polkadot Pallet for branch "pep-pvf-include-pvf-check-benchmark" with command cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras.rs

Toolchain: stable-x86_64-unknown-linux-gnu (default)
rustc 1.57.0 (f1edd0429 2021-11-29)

Results 
Pallet: "runtime_parachains::paras", Extrinsic: "force_set_current_code", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras CurrentCodeHash (r:1 w:1)
Storage: Paras CodeByHashRefs (r:1 w:1)
Storage: Paras PastCodeMeta (r:1 w:1)
Storage: Paras PastCodePruning (r:1 w:1)
Storage: Paras PastCodeHash (r:0 w:1)
Storage: Paras CodeByHash (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 4 + (0 * c)
Writes = 6 + (0 * c)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    c   mean µs  sigma µs       %
    1     28.96     0.126    0.4%
62915     159.6     0.466    0.2%
125829     282.5     0.526    0.1%
188743     396.7     0.497    0.1%
251657     517.6     7.914    1.5%
314571     632.9     0.741    0.1%
377485     752.7     9.088    1.2%
440399     872.4     7.527    0.8%
503313     991.3     11.11    1.1%
566227      1103     3.511    0.3%
629141      1226     8.359    0.6%
692055      1354     13.11    0.9%
754969      1468     12.07    0.8%
817883      1603      11.9    0.7%
880797      1719     13.88    0.8%
943711      1838     14.34    0.7%
1006625      1951     10.24    0.5%
1069539      2082     16.04    0.7%
1132453      2191     6.767    0.3%
1195367      2332     24.72    1.0%
1258281      2463     10.88    0.4%
1321195      2596     14.43    0.5%
1384109      2727     11.84    0.4%
1447023      2843     6.226    0.2%
1509937      2982     16.74    0.5%
1572851      3099     12.03    0.3%
1635765      3226     12.64    0.3%
1698679      3350     15.42    0.4%
1761593      3502     17.54    0.5%
1824507      3629     9.984    0.2%
1887421      3750     15.14    0.4%
1950335      3886     18.08    0.4%
2013249      4010     19.74    0.4%
2076163      4150     13.83    0.3%
2139077      4275     16.44    0.3%
2201991      4413     16.89    0.3%
2264905      4522     28.23    0.6%
2327819      4677     22.25    0.4%
2390733      4798     13.56    0.2%
2453647      4926     15.11    0.3%
2516561      5058     19.86    0.3%
2579475      5193     20.44    0.3%
2642389      5335     36.55    0.6%
2705303      5448     29.66    0.5%
2768217      5611     35.75    0.6%
2831131      5718     16.21    0.2%
2894045      5839     23.35    0.3%
2956959      5973     12.75    0.2%
3019873      6107     17.17    0.2%
3082787      6248     20.12    0.3%
3145701      6378     18.23    0.2%

Quality and confidence:
param     error
c             0

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 4 + (0 * c)
Writes = 6 + (0 * c)

Pallet: "runtime_parachains::paras", Extrinsic: "force_set_current_head", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras Heads (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    10.95
    + s        0
              µs

Reads = 0 + (0 * s)
Writes = 1 + (0 * s)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    s   mean µs  sigma µs       %
    1     7.552     0.056    0.7%
20972     19.09     0.067    0.3%
41943     45.44     0.274    0.6%
62914     61.71      0.33    0.5%
83885     76.34     0.649    0.8%
104856     93.25      0.36    0.3%
125827     108.7     0.418    0.3%
146798       120     0.434    0.3%
167769     136.3     0.854    0.6%
188740     149.7     0.649    0.4%
209711     167.8      1.33    0.7%
230682     178.5     0.525    0.2%
251653     193.6     0.544    0.2%
272624     208.2     0.269    0.1%
293595     223.4     0.472    0.2%
314566     238.6     0.426    0.1%
335537     253.7     0.902    0.3%
356508     267.7     0.363    0.1%
377479     284.1     1.433    0.5%
398450     299.5     0.944    0.3%
419421     312.6     1.067    0.3%
440392     327.8     0.807    0.2%
461363     346.3     4.009    1.1%
482334     357.9     0.753    0.2%
503305     376.9     10.13    2.6%
524276     388.3     0.588    0.1%
545247     404.3      2.32    0.5%
566218     418.8     2.264    0.5%
587189       433     1.374    0.3%
608160     451.8       9.6    2.1%
629131     462.9     0.957    0.2%
650102     479.1     3.158    0.6%
671073     494.6     1.788    0.3%
692044     508.9      2.07    0.4%
713015       529     8.752    1.6%
733986     540.9     1.511    0.2%
754957     554.4     0.684    0.1%
775928     570.1     1.697    0.2%
796899     612.2     7.433    1.2%
817870     619.3     2.273    0.3%
838841     646.2     1.675    0.2%
859812     640.5     10.16    1.5%
880783     652.6     4.434    0.6%
901754     665.9     2.863    0.4%
922725     686.7     11.66    1.6%
943696     702.9     8.209    1.1%
964667     716.4     2.892    0.4%
985638     738.9      8.67    1.1%
1006609     759.2     12.74    1.6%
1027580     778.5     12.88    1.6%
1048551     781.8     2.216    0.2%

Quality and confidence:
param     error
s             0

Model:
Time ~=    8.514
    + s    0.001
              µs

Reads = 0 + (0 * s)
Writes = 1 + (0 * s)

Pallet: "runtime_parachains::paras", Extrinsic: "force_schedule_code_upgrade", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: Paras FutureCodeHash (r:1 w:1)
Storage: Paras CurrentCodeHash (r:1 w:0)
Storage: Paras UpgradeCooldowns (r:1 w:1)
Storage: Paras PvfActiveVoteMap (r:1 w:0)
Storage: Paras CodeByHash (r:1 w:1)
Storage: Paras UpcomingUpgrades (r:1 w:1)
Storage: System Digest (r:1 w:1)
Storage: Paras CodeByHashRefs (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:0 w:1)
Storage: Paras UpgradeRestrictionSignal (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 9 + (0 * c)
Writes = 8 + (0 * c)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    c   mean µs  sigma µs       %
    1     53.89     0.385    0.7%
62915     185.1     0.717    0.3%
125829       307     0.316    0.1%
188743     421.2     0.468    0.1%
251657     546.3     7.966    1.4%
314571     656.6     1.083    0.1%
377485     783.8     13.11    1.6%
440399     902.4     13.43    1.4%
503313      1028     15.33    1.4%
566227      1153     14.84    1.2%
629141      1268     15.78    1.2%
692055      1386     15.33    1.1%
754969      1503     14.75    0.9%
817883      1614     11.72    0.7%
880797      1743     13.14    0.7%
943711      1858     13.07    0.7%
1006625      1987     13.06    0.6%
1069539      2110     9.478    0.4%
1132453      2247     15.28    0.6%
1195367      2398     16.26    0.6%
1258281      2509     7.563    0.3%
1321195      2633     15.19    0.5%
1384109      2782     11.69    0.4%
1447023      2880     15.48    0.5%
1509937      3020     17.64    0.5%
1572851      3169     11.99    0.3%
1635765      3300     15.75    0.4%
1698679      3435     10.67    0.3%
1761593      3567      11.3    0.3%
1824507      3705     5.309    0.1%
1887421      3845     23.95    0.6%
1950335      3985     13.12    0.3%
2013249      4095     7.041    0.1%
2076163      4235     7.779    0.1%
2139077      4370     12.11    0.2%
2201991      4504     17.34    0.3%
2264905      4643     11.85    0.2%
2327819      4767     8.571    0.1%
2390733      4907     15.34    0.3%
2453647      5048     14.35    0.2%
2516561      5181     13.72    0.2%
2579475      5316      13.3    0.2%
2642389      5971     24.38    0.4%
2705303      6147     30.91    0.5%
2768217      6270     19.76    0.3%
2831131      6414     12.12    0.1%
2894045      6567     23.37    0.3%
2956959      6726     10.73    0.1%
3019873      6861     37.87    0.5%
3082787      6377     12.39    0.1%
3145701      6515     13.79    0.2%

Quality and confidence:
param     error
c             0

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 9 + (0 * c)
Writes = 8 + (0 * c)

Pallet: "runtime_parachains::paras", Extrinsic: "force_note_new_head", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras FutureCodeUpgrades (r:1 w:0)
Storage: Paras Heads (r:0 w:1)
Storage: Paras UpgradeGoAheadSignal (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    16.14
    + s        0
              µs

Reads = 1 + (0 * s)
Writes = 2 + (0 * s)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    s   mean µs  sigma µs       %
    1      10.8     0.097    0.8%
20972     22.69      0.16    0.7%
41943     49.59     0.408    0.8%
62914     65.49     0.282    0.4%
83885     81.45     0.243    0.2%
104856     96.51     0.909    0.9%
125827     110.8     0.426    0.3%
146798     123.8     0.448    0.3%
167769     140.5     1.274    0.9%
188740     153.9     0.222    0.1%
209711     167.7     0.414    0.2%
230682     183.3     0.797    0.4%
251653     197.5     0.603    0.3%
272624     217.7     0.737    0.3%
293595     228.5     0.956    0.4%
314566     243.1     0.515    0.2%
335537     265.3     4.736    1.7%
356508     278.2     0.882    0.3%
377479     288.4     0.902    0.3%
398450     319.4     7.664    2.3%
419421     326.5     1.061    0.3%
440392       342     1.318    0.3%
461363     356.6     1.693    0.4%
482334     370.7     0.503    0.1%
503305     380.7     5.227    1.3%
524276     392.3     1.028    0.2%
545247     407.6     1.984    0.4%
566218     421.8     0.833    0.1%
587189     458.5     8.672    1.8%
608160     463.6     2.101    0.4%
629131     477.3      5.24    1.0%
650102       493     11.32    2.2%
671073     497.3     1.915    0.3%
692044     531.4     2.094    0.3%
713015       544     1.629    0.2%
733986     558.8      2.48    0.4%
754957     562.6     6.396    1.1%
775928     585.9     8.477    1.4%
796899     599.9     11.01    1.8%
817870     608.1     2.855    0.4%
838841     630.3     5.747    0.9%
859812     646.1     12.25    1.8%
880783       660     6.305    0.9%
901754     696.7      2.77    0.3%
922725     702.4     8.967    1.2%
943696     705.1     3.139    0.4%
964667     729.9     14.18    1.9%
985638     740.8     8.165    1.1%
1006609     764.4     10.58    1.3%
1027580     783.7     10.66    1.3%
1048551     795.9     10.14    1.2%

Quality and confidence:
param     error
s             0

Model:
Time ~=    13.96
    + s    0.001
              µs

Reads = 1 + (0 * s)
Writes = 2 + (0 * s)

Pallet: "runtime_parachains::paras", Extrinsic: "force_queue_action", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras ActionsQueue (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    15.93
              µs

Reads = 2
Writes = 1

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    15.93
              µs

Reads = 2
Writes = 1

Pallet: "runtime_parachains::paras", Extrinsic: "add_trusted_validation_code", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras PvfActiveVoteMap (r:1 w:0)
Storage: Paras CodeByHash (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 2 + (0 * c)
Writes = 1 + (0 * c)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    c   mean µs  sigma µs       %
    1     4.146      0.03    0.7%
62915     136.7     0.428    0.3%
125829     255.7     0.275    0.1%
188743     368.9     0.507    0.1%
251657     487.4     0.531    0.1%
314571     615.3     9.685    1.5%
377485     732.5     13.07    1.7%
440399     864.5     3.736    0.4%
503313     969.9     11.95    1.2%
566227      1097     12.09    1.1%
629141      1211     15.34    1.2%
692055      1318     8.764    0.6%
754969      1441     13.94    0.9%
817883      1573     12.21    0.7%
880797      1678      11.2    0.6%
943711      1809     18.17    1.0%
1006625      1934     17.72    0.9%
1069539      2061     14.12    0.6%
1132453      2185     12.53    0.5%
1195367      2309     15.36    0.6%
1258281      2439     14.18    0.5%
1321195      2564     12.05    0.4%
1384109      2709     17.05    0.6%
1447023      2836     11.07    0.3%
1509937      2942     14.36    0.4%
1572851      3096     17.54    0.5%
1635765      3227      15.5    0.4%
1698679      3369     12.97    0.3%
1761593      3486     10.45    0.2%
1824507      3629     18.88    0.5%
1887421      3758     7.203    0.1%
1950335      3909     15.28    0.3%
2013249      4030     18.56    0.4%
2076163      4158      19.7    0.4%
2139077      4300     9.008    0.2%
2201991      4451     19.94    0.4%
2264905      4554     14.55    0.3%
2327819      4703     18.56    0.3%
2390733      4841     22.49    0.4%
2453647      4979     14.75    0.2%
2516561      5102     18.57    0.3%
2579475      5216     17.41    0.3%
2642389      5356     23.55    0.4%
2705303      5496     16.68    0.3%
2768217      5612     11.62    0.2%
2831131      5750     13.66    0.2%
2894045      5918     25.93    0.4%
2956959      6048     15.71    0.2%
3019873      6149     16.39    0.2%
3082787      6275     22.79    0.3%
3145701      6443     20.93    0.3%

Quality and confidence:
param     error
c             0

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 2 + (0 * c)
Writes = 1 + (0 * c)

Pallet: "runtime_parachains::paras", Extrinsic: "poke_unused_validation_code", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras CodeByHashRefs (r:1 w:0)
Storage: Paras CodeByHash (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    2.406
              µs

Reads = 1
Writes = 1

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    2.406
              µs

Reads = 1
Writes = 1

Pallet: "runtime_parachains::paras", Extrinsic: "include_pvf_check_statement", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras PvfActiveVoteMap (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=      120
              µs

Reads = 4
Writes = 1

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=      120
              µs

Reads = 4
Writes = 1

Pallet: "runtime_parachains::paras", Extrinsic: "include_pvf_check_statement_finalize_upgrade_accept", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras PvfActiveVoteMap (r:1 w:1)
Storage: Paras PvfActiveVoteList (r:1 w:1)
Storage: Paras UpcomingUpgrades (r:1 w:1)
Storage: System Digest (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:0 w:100)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    623.5
              µs

Reads = 7
Writes = 104

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    623.5
              µs

Reads = 7
Writes = 104

Pallet: "runtime_parachains::paras", Extrinsic: "include_pvf_check_statement_finalize_upgrade_reject", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras PvfActiveVoteMap (r:1 w:1)
Storage: Paras PvfActiveVoteList (r:1 w:1)
Storage: Paras CodeByHashRefs (r:1 w:1)
Storage: Paras CodeByHash (r:0 w:1)
Storage: Paras UpgradeGoAheadSignal (r:0 w:100)
Storage: Paras FutureCodeHash (r:0 w:100)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=      552
              µs

Reads = 6
Writes = 204

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=      552
              µs

Reads = 6
Writes = 204

Pallet: "runtime_parachains::paras", Extrinsic: "include_pvf_check_statement_finalize_onboarding_accept", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras PvfActiveVoteMap (r:1 w:1)
Storage: Paras PvfActiveVoteList (r:1 w:1)
Storage: Paras ActionsQueue (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    498.5
              µs

Reads = 6
Writes = 3

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    498.5
              µs

Reads = 6
Writes = 3

Pallet: "runtime_parachains::paras", Extrinsic: "include_pvf_check_statement_finalize_onboarding_reject", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras PvfActiveVoteMap (r:1 w:1)
Storage: Paras PvfActiveVoteList (r:1 w:1)
Storage: Paras CodeByHashRefs (r:1 w:1)
Storage: Paras ParaLifecycles (r:0 w:100)
Storage: Paras CodeByHash (r:0 w:1)
Storage: Paras CurrentCodeHash (r:0 w:100)
Storage: Paras UpcomingParasGenesis (r:0 w:100)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    611.3
              µs

Reads = 6
Writes = 304

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    611.3
              µs

Reads = 6
Writes = 304

@pepyakin
Copy link
Contributor Author

/benchmark runtime kusama runtime_parachains::paras

@pepyakin
Copy link
Contributor Author

/benchmark runtime westend runtime_parachains::paras

@pepyakin
Copy link
Contributor Author

/benchmark runtime custom --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs

@pepyakin pepyakin linked an issue Feb 21, 2022 that may be closed by this pull request
weights for include_pvf_check_statement #4933
Closed
@parity-benchapp
Copy link

parity-benchapp bot commented Feb 21, 2022 •
edited
Loading

Benchmark Runtime Kusama Pallet for branch "pep-pvf-include-pvf-check-benchmark" with command cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs

Toolchain: stable-x86_64-unknown-linux-gnu (default)
rustc 1.57.0 (f1edd0429 2021-11-29)

Results 
Pallet: "runtime_parachains::paras", Extrinsic: "force_set_current_code", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras CurrentCodeHash (r:1 w:1)
Storage: Paras CodeByHashRefs (r:1 w:1)
Storage: Paras PastCodeMeta (r:1 w:1)
Storage: Paras PastCodePruning (r:1 w:1)
Storage: Paras PastCodeHash (r:0 w:1)
Storage: Paras CodeByHash (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 4 + (0 * c)
Writes = 6 + (0 * c)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    c   mean µs  sigma µs       %
    1      29.3     0.074    0.2%
62915     160.6      0.32    0.1%
125829     288.7     0.933    0.3%
188743     395.5      0.61    0.1%
251657     513.3     1.146    0.2%
314571     636.1     11.29    1.7%
377485     747.5     1.719    0.2%
440399     871.8     13.25    1.5%
503313     984.2     8.784    0.8%
566227      1103     6.481    0.5%
629141      1228     14.68    1.1%
692055      1340     10.47    0.7%
754969      1472     16.76    1.1%
817883      1588     15.04    0.9%
880797      1721     7.817    0.4%
943711      1835     13.27    0.7%
1006625      1937     10.85    0.5%
1069539      2080     12.71    0.6%
1132453      2212     15.59    0.7%
1195367      2335     18.21    0.7%
1258281      2449      14.8    0.6%
1321195      2566     15.46    0.6%
1384109      2690       9.6    0.3%
1447023      2825     19.16    0.6%
1509937      2963     10.55    0.3%
1572851      3096     33.46    1.0%
1635765      3220     12.28    0.3%
1698679      3339     17.09    0.5%
1761593      3471     18.79    0.5%
1824507      3600     17.55    0.4%
1887421      3727     19.01    0.5%
1950335      3861      10.4    0.2%
2013249      3979     5.364    0.1%
2076163      4101     21.98    0.5%
2139077      4231     15.13    0.3%
2201991      4391     15.98    0.3%
2264905      4501     11.31    0.2%
2327819      4617     10.26    0.2%
2390733      4750     22.75    0.4%
2453647      4898     11.18    0.2%
2516561      5472     52.64    0.9%
2579475      5669     6.863    0.1%
2642389      5319     27.33    0.5%
2705303      5443     29.07    0.5%
2768217      5563     18.44    0.3%
2831131      5695     31.06    0.5%
2894045      5812     17.75    0.3%
2956959      5956     24.95    0.4%
3019873      6087     24.37    0.4%
3082787      6210     14.41    0.2%
3145701      6348     19.76    0.3%

Quality and confidence:
param     error
c             0

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 4 + (0 * c)
Writes = 6 + (0 * c)

Pallet: "runtime_parachains::paras", Extrinsic: "force_set_current_head", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras Heads (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    12.74
    + s        0
              µs

Reads = 0 + (0 * s)
Writes = 1 + (0 * s)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    s   mean µs  sigma µs       %
    1     7.852     0.163    2.0%
20972     19.33     0.104    0.5%
41943     45.06     0.333    0.7%
62914     60.58     0.295    0.4%
83885     76.68     0.108    0.1%
104856        93     0.898    0.9%
125827     107.8      0.34    0.3%
146798     119.2     1.217    1.0%
167769     135.5     0.572    0.4%
188740     146.8     0.458    0.3%
209711     161.9     1.091    0.6%
230682     176.8     0.846    0.4%
251653     192.4     3.096    1.6%
272624     206.4     1.499    0.7%
293595     219.7     0.494    0.2%
314566     235.6     0.768    0.3%
335537     250.7     2.213    0.8%
356508     265.3     2.453    0.9%
377479     278.6     0.807    0.2%
398450     292.8     0.714    0.2%
419421     309.8     4.475    1.4%
440392     335.8     11.74    3.4%
461363     336.8      2.48    0.7%
482334     349.8     0.761    0.2%
503305     365.8     1.528    0.4%
524276     381.6     2.122    0.5%
545247     396.9     7.363    1.8%
566218     410.1      2.25    0.5%
587189     432.6     4.375    1.0%
608160     438.9     0.831    0.1%
629131     456.4      2.92    0.6%
650102     470.9      2.14    0.4%
671073     485.9     3.616    0.7%
692044     499.1     2.183    0.4%
713015     516.3     3.624    0.7%
733986     529.6     1.345    0.2%
754957     543.6     1.662    0.3%
775928     559.2     5.141    0.9%
796899     580.2     9.003    1.5%
817870     591.2       4.4    0.7%
838841     606.4      3.25    0.5%
859812     630.5      11.4    1.8%
880783     647.6     10.13    1.5%
901754     652.9     3.244    0.4%
922725     696.1     15.31    2.2%
943696     683.9     1.628    0.2%
964667       706     3.925    0.5%
985638       718     3.084    0.4%
1006609     737.5     4.114    0.5%
1027580     758.3     7.541    0.9%
1048551     777.1     9.799    1.2%

Quality and confidence:
param     error
s             0

Model:
Time ~=    10.15
    + s    0.001
              µs

Reads = 0 + (0 * s)
Writes = 1 + (0 * s)

Pallet: "runtime_parachains::paras", Extrinsic: "force_schedule_code_upgrade", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: Paras FutureCodeHash (r:1 w:1)
Storage: Paras CurrentCodeHash (r:1 w:0)
Storage: Paras UpgradeCooldowns (r:1 w:1)
Storage: Paras PvfActiveVoteMap (r:1 w:0)
Storage: Paras CodeByHash (r:1 w:1)
Storage: Paras UpcomingUpgrades (r:1 w:1)
Storage: System Digest (r:1 w:1)
Storage: Paras CodeByHashRefs (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:0 w:1)
Storage: Paras UpgradeRestrictionSignal (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 9 + (0 * c)
Writes = 8 + (0 * c)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    c   mean µs  sigma µs       %
    1     54.21     0.263    0.4%
62915     184.9     0.398    0.2%
125829     307.4     1.396    0.4%
188743     421.1     3.251    0.7%
251657     539.8     4.803    0.8%
314571     659.2     7.061    1.0%
377485     784.2     13.82    1.7%
440399     906.6     15.58    1.7%
503313      1015     12.89    1.2%
566227      1134     14.53    1.2%
629141      1263     13.24    1.0%
692055      1360     4.534    0.3%
754969      1498     11.93    0.7%
817883      1624     12.81    0.7%
880797      1746     13.62    0.7%
943711      1859     17.69    0.9%
1006625      1997     11.13    0.5%
1069539      2107     17.66    0.8%
1132453      2250     14.32    0.6%
1195367      2363     17.55    0.7%
1258281      2480     15.67    0.6%
1321195      2672     11.07    0.4%
1384109      2758     11.67    0.4%
1447023      2885      9.75    0.3%
1509937      3032     15.84    0.5%
1572851      3165     18.88    0.5%
1635765      3281      8.23    0.2%
1698679      3406     11.83    0.3%
1761593      3545      13.5    0.3%
1824507      3675     25.07    0.6%
1887421      3813        14    0.3%
1950335      3952     10.54    0.2%
2013249      4078     10.78    0.2%
2076163      4217     13.92    0.3%
2139077      4346     12.17    0.2%
2201991      4481     15.12    0.3%
2264905      4620     19.13    0.4%
2327819      4753     13.57    0.2%
2390733      4913     36.47    0.7%
2453647      5032     25.11    0.4%
2516561      5165     16.37    0.3%
2579475      5312     6.803    0.1%
2642389      5413     11.56    0.2%
2705303      5645     9.885    0.1%
2768217      5714     18.57    0.3%
2831131      5825      21.1    0.3%
2894045      5947     18.57    0.3%
2956959      6127     19.09    0.3%
3019873      6216     16.61    0.2%
3082787      6356     22.47    0.3%
3145701      6475      24.4    0.3%

Quality and confidence:
param     error
c             0

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 9 + (0 * c)
Writes = 8 + (0 * c)

Pallet: "runtime_parachains::paras", Extrinsic: "force_note_new_head", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras FutureCodeUpgrades (r:1 w:0)
Storage: Paras Heads (r:0 w:1)
Storage: Paras UpgradeGoAheadSignal (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    17.27
    + s        0
              µs

Reads = 1 + (0 * s)
Writes = 2 + (0 * s)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    s   mean µs  sigma µs       %
    1     10.88     0.135    1.2%
20972     22.09     0.066    0.2%
41943     49.33     0.221    0.4%
62914     65.33     0.248    0.3%
83885     80.88     0.297    0.3%
104856     95.21      0.82    0.8%
125827     110.8     1.708    1.5%
146798     122.2     0.259    0.2%
167769     138.1     0.948    0.6%
188740       152     0.155    0.1%
209711     166.2     0.415    0.2%
230682       184     3.213    1.7%
251653     195.7     0.455    0.2%
272624     213.7     5.125    2.3%
293595     226.2     2.468    1.0%
314566     239.6     0.731    0.3%
335537     255.5     1.154    0.4%
356508     268.6     0.924    0.3%
377479       289     4.304    1.4%
398450     325.8     7.432    2.2%
419421     321.6      1.66    0.5%
440392     341.2     2.683    0.7%
461363     349.9     0.605    0.1%
482334     367.8     2.129    0.5%
503305     379.9     1.114    0.2%
524276     387.6     1.023    0.2%
545247     402.4     1.616    0.4%
566218       417     1.043    0.2%
587189     434.1     7.706    1.7%
608160     451.1     9.681    2.1%
629131     463.4     2.708    0.5%
650102     479.7     3.643    0.7%
671073     492.3     3.521    0.7%
692044     521.8     2.086    0.3%
713015     535.1     1.511    0.2%
733986     549.5     1.343    0.2%
754957     552.9     2.786    0.5%
775928       568     3.181    0.5%
796899     589.2     10.95    1.8%
817870     599.9      4.08    0.6%
838841     638.3      5.11    0.8%
859812     630.2     5.135    0.8%
880783       646     2.832    0.4%
901754     678.3      13.7    2.0%
922725       676      1.91    0.2%
943696     702.6      11.8    1.6%
964667     714.3     7.839    1.0%
985638       725     1.262    0.1%
1006609     744.8     4.371    0.5%
1027580       776     12.94    1.6%
1048551     776.1     6.053    0.7%

Quality and confidence:
param     error
s             0

Model:
Time ~=    15.43
    + s    0.001
              µs

Reads = 1 + (0 * s)
Writes = 2 + (0 * s)

Pallet: "runtime_parachains::paras", Extrinsic: "force_queue_action", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras ActionsQueue (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    16.16
              µs

Reads = 2
Writes = 1

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    16.16
              µs

Reads = 2
Writes = 1

Pallet: "runtime_parachains::paras", Extrinsic: "add_trusted_validation_code", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras PvfActiveVoteMap (r:1 w:0)
Storage: Paras CodeByHash (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 2 + (0 * c)
Writes = 1 + (0 * c)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    c   mean µs  sigma µs       %
    1     4.326     0.074    1.7%
62915       133     0.347    0.2%
125829     256.5     2.011    0.7%
188743     367.6     1.051    0.2%
251657       485     1.379    0.2%
314571     601.6      1.42    0.2%
377485     719.7     2.319    0.3%
440399     841.2     8.803    1.0%
503313     961.5      13.2    1.3%
566227      1090     16.26    1.4%
629141      1199     15.21    1.2%
692055      1323     17.26    1.3%
754969      1437     12.93    0.8%
817883      1562     15.45    0.9%
880797      1677     15.42    0.9%
943711      1795     14.92    0.8%
1006625      1922     16.14    0.8%
1069539      2047     16.55    0.8%
1132453      2165      12.6    0.5%
1195367      2288     13.03    0.5%
1258281      2421     10.29    0.4%
1321195      2558     14.66    0.5%
1384109      2690     14.74    0.5%
1447023      2818      12.9    0.4%
1509937      2964     15.93    0.5%
1572851      3080     18.94    0.6%
1635765      3203     13.08    0.4%
1698679      3336     18.87    0.5%
1761593      3483     9.312    0.2%
1824507      3614     13.21    0.3%
1887421      3752     15.01    0.4%
1950335      3867     17.57    0.4%
2013249      4009      16.1    0.4%
2076163      4140     18.66    0.4%
2139077      4277     16.19    0.3%
2201991      4417      21.4    0.4%
2264905      4537     16.25    0.3%
2327819      4662     27.92    0.5%
2390733      4802     16.78    0.3%
2453647      4932     25.93    0.5%
2516561      5050     20.98    0.4%
2579475      5199     16.85    0.3%
2642389      5323     19.65    0.3%
2705303      5462     21.43    0.3%
2768217      5618     31.41    0.5%
2831131      5726     12.28    0.2%
2894045      5850     12.58    0.2%
2956959      5988     16.15    0.2%
3019873      6108     28.97    0.4%
3082787      6248      14.7    0.2%
3145701      6406     27.88    0.4%

Quality and confidence:
param     error
c             0

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 2 + (0 * c)
Writes = 1 + (0 * c)

Pallet: "runtime_parachains::paras", Extrinsic: "poke_unused_validation_code", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras CodeByHashRefs (r:1 w:0)
Storage: Paras CodeByHash (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    2.464
              µs

Reads = 1
Writes = 1

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    2.464
              µs

Reads = 1
Writes = 1

Pallet: "runtime_parachains::paras", Extrinsic: "include_pvf_check_statement", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras PvfActiveVoteMap (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    117.2
              µs

Reads = 4
Writes = 1

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    117.2
              µs

Reads = 4
Writes = 1

Pallet: "runtime_parachains::paras", Extrinsic: "include_pvf_check_statement_finalize_upgrade_accept", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras PvfActiveVoteMap (r:1 w:1)
Storage: Paras PvfActiveVoteList (r:1 w:1)
Storage: Paras UpcomingUpgrades (r:1 w:1)
Storage: System Digest (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:0 w:100)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    624.8
              µs

Reads = 7
Writes = 104

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    624.8
              µs

Reads = 7
Writes = 104

Pallet: "runtime_parachains::paras", Extrinsic: "include_pvf_check_statement_finalize_upgrade_reject", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras PvfActiveVoteMap (r:1 w:1)
Storage: Paras PvfActiveVoteList (r:1 w:1)
Storage: Paras CodeByHashRefs (r:1 w:1)
Storage: Paras CodeByHash (r:0 w:1)
Storage: Paras UpgradeGoAheadSignal (r:0 w:100)
Storage: Paras FutureCodeHash (r:0 w:100)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    551.3
              µs

Reads = 6
Writes = 204

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    551.3
              µs

Reads = 6
Writes = 204

Pallet: "runtime_parachains::paras", Extrinsic: "include_pvf_check_statement_finalize_onboarding_accept", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras PvfActiveVoteMap (r:1 w:1)
Storage: Paras PvfActiveVoteList (r:1 w:1)
Storage: Paras ActionsQueue (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    498.9
              µs

Reads = 6
Writes = 3

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    498.9
              µs

Reads = 6
Writes = 3

Pallet: "runtime_parachains::paras", Extrinsic: "include_pvf_check_statement_finalize_onboarding_reject", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras PvfActiveVoteMap (r:1 w:1)
Storage: Paras PvfActiveVoteList (r:1 w:1)
Storage: Paras CodeByHashRefs (r:1 w:1)
Storage: Paras ParaLifecycles (r:0 w:100)
Storage: Paras CodeByHash (r:0 w:1)
Storage: Paras CurrentCodeHash (r:0 w:100)
Storage: Paras UpcomingParasGenesis (r:0 w:100)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    609.4
              µs

Reads = 6
Writes = 304

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    609.4
              µs

Reads = 6
Writes = 304

@pepyakin pepyakin force-pushed the pep-paras-initialize-now-refactors branch from 8a22999 to f80186a Compare February 21, 2022 15:14
@parity-benchapp
Copy link

parity-benchapp bot commented Feb 21, 2022 •
edited
Loading

Benchmark Runtime Westend Pallet for branch "pep-pvf-include-pvf-check-benchmark" with command cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

Toolchain: stable-x86_64-unknown-linux-gnu (default)
rustc 1.57.0 (f1edd0429 2021-11-29)

Results 
Pallet: "runtime_parachains::paras", Extrinsic: "force_set_current_code", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras CurrentCodeHash (r:1 w:1)
Storage: Paras CodeByHashRefs (r:1 w:1)
Storage: Paras PastCodeMeta (r:1 w:1)
Storage: Paras PastCodePruning (r:1 w:1)
Storage: Paras PastCodeHash (r:0 w:1)
Storage: Paras CodeByHash (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 4 + (0 * c)
Writes = 6 + (0 * c)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    c   mean µs  sigma µs       %
    1     29.29     0.104    0.3%
62915     161.8     0.733    0.4%
125829     283.9     0.494    0.1%
188743       417     16.14    3.8%
251657     516.1     0.798    0.1%
314571     636.7     9.325    1.4%
377485     758.2     10.73    1.4%
440399     872.5     9.252    1.0%
503313     992.7     10.97    1.1%
566227      1113     10.67    0.9%
629141      1239     12.67    1.0%
692055      1354      13.4    0.9%
754969      1480     15.14    1.0%
817883      1592     11.59    0.7%
880797      1713     13.63    0.7%
943711      1846     13.93    0.7%
1006625      1962     15.74    0.8%
1069539      2096     10.64    0.5%
1132453      2227     15.53    0.6%
1195367      2353     5.793    0.2%
1258281      2471     8.004    0.3%
1321195      2610     21.56    0.8%
1384109      2726     18.41    0.6%
1447023      2853     16.84    0.5%
1509937      2982     12.56    0.4%
1572851      3129     14.35    0.4%
1635765      3240     15.88    0.4%
1698679      3364     13.09    0.3%
1761593      3502     18.36    0.5%
1824507      3653     20.12    0.5%
1887421      3775     19.81    0.5%
1950335      3894     23.57    0.6%
2013249      4017     10.21    0.2%
2076163      4162     19.05    0.4%
2139077      4263     22.01    0.5%
2201991      4416     16.66    0.3%
2264905      4519     11.47    0.2%
2327819      4679     14.27    0.3%
2390733      4795     12.26    0.2%
2453647      4948     17.53    0.3%
2516561      5068     19.11    0.3%
2579475      5200     13.71    0.2%
2642389      5335     21.99    0.4%
2705303      5458     15.61    0.2%
2768217      5568     25.26    0.4%
2831131      5722     19.73    0.3%
2894045      5946     8.595    0.1%
2956959      5987     25.25    0.4%
3019873      6121     21.49    0.3%
3082787      6245     28.94    0.4%
3145701      6375     12.67    0.1%

Quality and confidence:
param     error
c             0

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 4 + (0 * c)
Writes = 6 + (0 * c)

Pallet: "runtime_parachains::paras", Extrinsic: "force_set_current_head", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras Heads (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    12.34
    + s        0
              µs

Reads = 0 + (0 * s)
Writes = 1 + (0 * s)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    s   mean µs  sigma µs       %
    1     7.701     0.036    0.4%
20972     18.94     0.079    0.4%
41943     45.72     0.232    0.5%
62914     61.57     0.272    0.4%
83885     77.61     0.237    0.3%
104856     92.73      0.89    0.9%
125827     109.4     0.164    0.1%
146798     119.5     0.239    0.1%
167769     135.2     0.276    0.2%
188740     150.1     0.654    0.4%
209711     164.4     0.632    0.3%
230682       180     0.304    0.1%
251653     198.5     0.807    0.4%
272624     210.3     0.416    0.1%
293595     224.8     0.537    0.2%
314566     240.4     0.802    0.3%
335537     255.6     1.381    0.5%
356508     269.3     0.897    0.3%
377479     284.2     0.493    0.1%
398450     299.4     0.538    0.1%
419421     313.3     1.222    0.3%
440392     328.9     0.914    0.2%
461363     344.3     0.722    0.2%
482334     359.3     0.603    0.1%
503305     383.7      12.8    3.3%
524276       389     1.467    0.3%
545247     408.3     9.679    2.3%
566218     417.6      0.84    0.2%
587189     433.8      1.36    0.3%
608160     450.1     1.507    0.3%
629131     463.9     1.609    0.3%
650102     478.2     0.919    0.1%
671073       495     3.031    0.6%
692044     511.6     1.524    0.2%
713015     527.3     1.682    0.3%
733986     544.2     3.669    0.6%
754957     558.8     1.735    0.3%
775928     575.7      5.04    0.8%
796899     590.1     2.067    0.3%
817870     606.5     5.289    0.8%
838841     619.1     1.029    0.1%
859812       630     2.137    0.3%
880783     651.7     2.335    0.3%
901754       665     2.274    0.3%
922725     698.8     16.85    2.4%
943696     703.6     3.328    0.4%
964667     721.3     10.38    1.4%
985638     737.3     2.038    0.2%
1006609     748.5     1.844    0.2%
1027580     767.8     3.738    0.4%
1048551     791.7     12.45    1.5%

Quality and confidence:
param     error
s             0

Model:
Time ~=    10.16
    + s    0.001
              µs

Reads = 0 + (0 * s)
Writes = 1 + (0 * s)

Pallet: "runtime_parachains::paras", Extrinsic: "force_schedule_code_upgrade", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras FutureCodeHash (r:1 w:1)
Storage: Paras CurrentCodeHash (r:1 w:0)
Storage: Paras UpgradeCooldowns (r:1 w:1)
Storage: Paras PvfActiveVoteMap (r:1 w:0)
Storage: Paras CodeByHash (r:1 w:1)
Storage: Paras UpcomingUpgrades (r:1 w:1)
Storage: System Digest (r:1 w:1)
Storage: Paras CodeByHashRefs (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:0 w:1)
Storage: Paras UpgradeRestrictionSignal (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 8 + (0 * c)
Writes = 8 + (0 * c)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    c   mean µs  sigma µs       %
    1     54.88     0.417    0.7%
62915     185.8     0.613    0.3%
125829     315.2     0.683    0.2%
188743     423.2     0.541    0.1%
251657     541.4     1.375    0.2%
314571     658.8     1.442    0.2%
377485     776.2     1.107    0.1%
440399     899.8     10.38    1.1%
503313      1024     13.32    1.3%
566227      1139     10.69    0.9%
629141      1258     10.54    0.8%
692055      1377     9.653    0.7%
754969      1508     13.29    0.8%
817883      1630     17.34    1.0%
880797      1754     15.02    0.8%
943711      1897     17.66    0.9%
1006625      1998     11.67    0.5%
1069539      2111      6.85    0.3%
1132453      2257     13.49    0.5%
1195367      2386     10.37    0.4%
1258281      2515     8.179    0.3%
1321195      2640     12.77    0.4%
1384109      2775     14.18    0.5%
1447023      2920     18.94    0.6%
1509937      3044     9.607    0.3%
1572851      3180     16.14    0.5%
1635765      3305     13.23    0.4%
1698679      3450     22.25    0.6%
1761593      3573     7.775    0.2%
1824507      3700     22.29    0.6%
1887421      3854     13.64    0.3%
1950335      3981     17.37    0.4%
2013249      4100     16.78    0.4%
2076163      4231     16.29    0.3%
2139077      4372     9.854    0.2%
2201991      4494     15.28    0.3%
2264905      4660     18.05    0.3%
2327819      4788     14.59    0.3%
2390733      4913     14.81    0.3%
2453647      5060     21.52    0.4%
2516561      5195      15.3    0.2%
2579475      5320     14.52    0.2%
2642389      5455        16    0.2%
2705303      5572     11.53    0.2%
2768217      5813     8.161    0.1%
2831131      5847      31.1    0.5%
2894045      5989     28.58    0.4%
2956959      6109     10.39    0.1%
3019873      6265     23.78    0.3%
3082787      6399     14.49    0.2%
3145701      6530     7.085    0.1%

Quality and confidence:
param     error
c             0

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 8 + (0 * c)
Writes = 8 + (0 * c)

Pallet: "runtime_parachains::paras", Extrinsic: "force_note_new_head", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras FutureCodeUpgrades (r:1 w:0)
Storage: Paras Heads (r:0 w:1)
Storage: Paras UpgradeGoAheadSignal (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    18.16
    + s        0
              µs

Reads = 1 + (0 * s)
Writes = 2 + (0 * s)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    s   mean µs  sigma µs       %
    1     10.86     0.059    0.5%
20972     22.94     0.098    0.4%
41943     49.92     0.335    0.6%
62914     66.17     0.302    0.4%
83885     82.26     0.488    0.5%
104856     97.87     1.095    1.1%
125827     113.2     1.123    0.9%
146798     124.3     0.316    0.2%
167769     139.5     0.389    0.2%
188740     154.2     0.353    0.2%
209711     168.8     0.386    0.2%
230682     184.2     0.386    0.2%
251653     200.1     0.945    0.4%
272624     214.2     0.703    0.3%
293595     235.7     2.021    0.8%
314566     251.9     1.376    0.5%
335537     266.7     0.942    0.3%
356508     281.8     1.113    0.3%
377479     298.5     1.723    0.5%
398450     313.1     0.989    0.3%
419421     328.1     1.006    0.3%
440392       341     0.574    0.1%
461363     356.8     0.816    0.2%
482334     363.4     0.633    0.1%
503305     378.7     0.856    0.2%
524276     393.8     1.752    0.4%
545247     418.5     7.404    1.7%
566218     425.8     1.366    0.3%
587189     438.5     0.874    0.1%
608160     452.6     0.596    0.1%
629131       486     2.268    0.4%
650102     498.2     1.539    0.3%
671073     522.2     1.572    0.3%
692044     517.6     9.916    1.9%
713015     530.2     2.283    0.4%
733986     559.2     12.66    2.2%
754957     562.7     7.747    1.3%
775928     590.1     13.19    2.2%
796899       594      10.1    1.7%
817870     606.6     1.388    0.2%
838841     629.3     11.35    1.8%
859812     660.3     0.886    0.1%
880783     684.2     13.35    1.9%
901754     689.6     18.01    2.6%
922725     694.2     10.95    1.5%
943696     716.5     11.04    1.5%
964667     729.3     14.23    1.9%
985638     742.9     11.01    1.4%
1006609     754.1     9.725    1.2%
1027580     775.1     12.24    1.5%
1048551     794.1     12.76    1.6%

Quality and confidence:
param     error
s             0

Model:
Time ~=    15.41
    + s    0.001
              µs

Reads = 1 + (0 * s)
Writes = 2 + (0 * s)

Pallet: "runtime_parachains::paras", Extrinsic: "force_queue_action", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras ActionsQueue (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    15.99
              µs

Reads = 2
Writes = 1

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    15.99
              µs

Reads = 2
Writes = 1

Pallet: "runtime_parachains::paras", Extrinsic: "add_trusted_validation_code", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras PvfActiveVoteMap (r:1 w:0)
Storage: Paras CodeByHash (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 2 + (0 * c)
Writes = 1 + (0 * c)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    c   mean µs  sigma µs       %
    1     4.208      0.05    1.1%
62915     137.6     0.348    0.2%
125829     257.7     1.309    0.5%
188743     370.7     0.447    0.1%
251657     490.9      3.91    0.7%
314571     611.7     6.531    1.0%
377485     728.2     9.126    1.2%
440399     857.6      16.7    1.9%
503313       982     15.29    1.5%
566227      1097     15.42    1.4%
629141      1203     9.003    0.7%
692055      1338     17.88    1.3%
754969      1452      17.1    1.1%
817883      1573     14.04    0.8%
880797      1709     6.227    0.3%
943711      1813     14.38    0.7%
1006625      1950     15.91    0.8%
1069539      2069     14.47    0.6%
1132453      2178     13.98    0.6%
1195367      2308     10.87    0.4%
1258281      2448     17.02    0.6%
1321195      2590     10.15    0.3%
1384109      2711      20.7    0.7%
1447023      2850     14.09    0.4%
1509937      2994     16.06    0.5%
1572851      3128     15.03    0.4%
1635765      3257     11.86    0.3%
1698679      3372     14.99    0.4%
1761593      3496      15.6    0.4%
1824507      3631     16.48    0.4%
1887421      3795      17.6    0.4%
1950335      3916      11.1    0.2%
2013249      4046     24.26    0.5%
2076163      4175     15.74    0.3%
2139077      4297      14.2    0.3%
2201991      4441     25.42    0.5%
2264905      4549     20.96    0.4%
2327819      4698     22.83    0.4%
2390733      4828     9.402    0.1%
2453647      4953     15.36    0.3%
2516561      5102     26.42    0.5%
2579475      5235     16.12    0.3%
2642389      5367     21.19    0.3%
2705303      5506     18.02    0.3%
2768217      5652     11.39    0.2%
2831131      5777     25.08    0.4%
2894045      5861     9.775    0.1%
2956959      6069     19.78    0.3%
3019873      6193      16.3    0.2%
3082787      6312     25.81    0.4%
3145701      6421     27.79    0.4%

Quality and confidence:
param     error
c             0

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 2 + (0 * c)
Writes = 1 + (0 * c)

Pallet: "runtime_parachains::paras", Extrinsic: "poke_unused_validation_code", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras CodeByHashRefs (r:1 w:0)
Storage: Paras CodeByHash (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    2.445
              µs

Reads = 1
Writes = 1

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    2.445
              µs

Reads = 1
Writes = 1

Pallet: "runtime_parachains::paras", Extrinsic: "include_pvf_check_statement", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras PvfActiveVoteMap (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    116.5
              µs

Reads = 3
Writes = 1

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    116.5
              µs

Reads = 3
Writes = 1

Pallet: "runtime_parachains::paras", Extrinsic: "include_pvf_check_statement_finalize_upgrade_accept", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras PvfActiveVoteMap (r:1 w:1)
Storage: Paras PvfActiveVoteList (r:1 w:1)
Storage: Paras UpcomingUpgrades (r:1 w:1)
Storage: System Digest (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:0 w:100)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=      622
              µs

Reads = 6
Writes = 104

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=      622
              µs

Reads = 6
Writes = 104

Pallet: "runtime_parachains::paras", Extrinsic: "include_pvf_check_statement_finalize_upgrade_reject", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras PvfActiveVoteMap (r:1 w:1)
Storage: Paras PvfActiveVoteList (r:1 w:1)
Storage: Paras CodeByHashRefs (r:1 w:1)
Storage: Paras CodeByHash (r:0 w:1)
Storage: Paras UpgradeGoAheadSignal (r:0 w:100)
Storage: Paras FutureCodeHash (r:0 w:100)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    550.5
              µs

Reads = 5
Writes = 204

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    550.5
              µs

Reads = 5
Writes = 204

Pallet: "runtime_parachains::paras", Extrinsic: "include_pvf_check_statement_finalize_onboarding_accept", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras PvfActiveVoteMap (r:1 w:1)
Storage: Paras PvfActiveVoteList (r:1 w:1)
Storage: Paras ActionsQueue (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    499.5
              µs

Reads = 5
Writes = 3

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    499.5
              µs

Reads = 5
Writes = 3

Pallet: "runtime_parachains::paras", Extrinsic: "include_pvf_check_statement_finalize_onboarding_reject", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras PvfActiveVoteMap (r:1 w:1)
Storage: Paras PvfActiveVoteList (r:1 w:1)
Storage: Paras CodeByHashRefs (r:1 w:1)
Storage: Paras ParaLifecycles (r:0 w:100)
Storage: Paras CodeByHash (r:0 w:1)
Storage: Paras CurrentCodeHash (r:0 w:100)
Storage: Paras UpcomingParasGenesis (r:0 w:100)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    602.3
              µs

Reads = 5
Writes = 304

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    602.3
              µs

Reads = 5
Writes = 304

@parity-benchapp
Copy link

parity-benchapp bot commented Feb 21, 2022 •
edited
Loading

Benchmark Runtime Custom for branch "pep-pvf-include-pvf-check-benchmark" with command cargo run --quiet --profile=production --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs

Toolchain: stable-x86_64-unknown-linux-gnu (default)
rustc 1.57.0 (f1edd0429 2021-11-29)

Results 
Pallet: "runtime_parachains::paras", Extrinsic: "force_set_current_code", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras CurrentCodeHash (r:1 w:1)
Storage: Paras CodeByHashRefs (r:1 w:1)
Storage: Paras PastCodeMeta (r:1 w:1)
Storage: Paras PastCodePruning (r:1 w:1)
Storage: Paras PastCodeHash (r:0 w:1)
Storage: Paras CodeByHash (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 4 + (0 * c)
Writes = 6 + (0 * c)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    c   mean µs  sigma µs       %
    1      29.3     0.085    0.2%
62915     164.9     0.567    0.3%
125829     286.8     2.383    0.8%
188743     405.4     14.76    3.6%
251657     516.3     1.056    0.2%
314571     640.6     10.81    1.6%
377485     753.7     3.457    0.4%
440399       880     14.63    1.6%
503313     993.6      8.62    0.8%
566227      1111     8.222    0.7%
629141      1240      13.1    1.0%
692055      1354     13.31    0.9%
754969      1511     4.058    0.2%
817883      1614     10.72    0.6%
880797      1752     14.94    0.8%
943711      1835     14.05    0.7%
1006625      1963     13.32    0.6%
1069539      2100     15.63    0.7%
1132453      2225     9.877    0.4%
1195367      2345     12.03    0.5%
1258281      2457     16.06    0.6%
1321195      2611     15.82    0.6%
1384109      2721     16.84    0.6%
1447023      2853     22.57    0.7%
1509937      2984     10.72    0.3%
1572851      3118     16.72    0.5%
1635765      3229     15.36    0.4%
1698679      3362     15.49    0.4%
1761593      3502     18.77    0.5%
1824507      3615     15.45    0.4%
1887421      3753     18.46    0.4%
1950335      3897     19.15    0.4%
2013249      3999     9.658    0.2%
2076163      4159     8.565    0.2%
2139077      4289     11.03    0.2%
2201991      4401     10.19    0.2%
2264905      4528     8.558    0.1%
2327819      4654     17.14    0.3%
2390733      4813     21.83    0.4%
2453647      4926     16.27    0.3%
2516561      5065     11.97    0.2%
2579475      5212     20.66    0.3%
2642389      5344     23.32    0.4%
2705303      5478      25.5    0.4%
2768217      5599     27.21    0.4%
2831131      5715     19.66    0.3%
2894045      5860     23.28    0.3%
2956959      5996     19.34    0.3%
3019873      6128     17.31    0.2%
3082787      6255     19.51    0.3%
3145701      6401     23.68    0.3%

Quality and confidence:
param     error
c             0

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 4 + (0 * c)
Writes = 6 + (0 * c)

Pallet: "runtime_parachains::paras", Extrinsic: "force_set_current_head", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras Heads (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    12.74
    + s        0
              µs

Reads = 0 + (0 * s)
Writes = 1 + (0 * s)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    s   mean µs  sigma µs       %
    1     7.615     0.064    0.8%
20972     18.85     0.095    0.5%
41943     45.62      0.22    0.4%
62914     61.53     0.222    0.3%
83885        78     0.279    0.3%
104856     93.67     0.215    0.2%
125827     109.1     0.648    0.5%
146798     122.6     0.362    0.2%
167769     134.9     0.156    0.1%
188740     152.3     1.579    1.0%
209711     164.8     0.212    0.1%
230682     179.6     0.443    0.2%
251653     195.3     0.431    0.2%
272624       209     0.533    0.2%
293595     224.6     0.634    0.2%
314566     239.9     0.705    0.2%
335537     255.1     0.217    0.0%
356508     269.8     0.219    0.0%
377479       285     0.442    0.1%
398450     298.8     0.506    0.1%
419421     314.7      0.34    0.1%
440392     329.5     0.242    0.0%
461363     344.2     0.937    0.2%
482334     359.3     0.435    0.1%
503305     377.2     5.814    1.5%
524276     389.7     0.796    0.2%
545247       403     1.021    0.2%
566218     419.1     5.022    1.1%
587189     432.1     0.803    0.1%
608160     447.4      0.85    0.1%
629131     461.9     0.988    0.2%
650102     477.9     0.719    0.1%
671073     495.1     4.467    0.9%
692044     508.8     1.625    0.3%
713015     522.5     0.894    0.1%
733986     538.7     0.909    0.1%
754957     555.7     1.707    0.3%
775928     570.5     1.656    0.2%
796899     587.2      1.42    0.2%
817870     604.1     3.964    0.6%
838841     619.1     1.451    0.2%
859812     638.1     12.41    1.9%
880783     659.9     12.82    1.9%
901754     667.5     2.012    0.3%
922725     680.6      0.78    0.1%
943696     700.4     6.879    0.9%
964667     714.4     4.141    0.5%
985638     736.7     12.91    1.7%
1006609       750     5.304    0.7%
1027580     768.3     10.12    1.3%
1048551     781.8      8.84    1.1%

Quality and confidence:
param     error
s             0

Model:
Time ~=    10.85
    + s    0.001
              µs

Reads = 0 + (0 * s)
Writes = 1 + (0 * s)

Pallet: "runtime_parachains::paras", Extrinsic: "force_schedule_code_upgrade", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: Paras FutureCodeHash (r:1 w:1)
Storage: Paras CurrentCodeHash (r:1 w:0)
Storage: Paras UpgradeCooldowns (r:1 w:1)
Storage: Paras PvfActiveVoteMap (r:1 w:0)
Storage: Paras CodeByHash (r:1 w:1)
Storage: Paras UpcomingUpgrades (r:1 w:1)
Storage: System Digest (r:1 w:1)
Storage: Paras CodeByHashRefs (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:0 w:1)
Storage: Paras UpgradeRestrictionSignal (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 9 + (0 * c)
Writes = 8 + (0 * c)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    c   mean µs  sigma µs       %
    1     53.98     0.309    0.5%
62915       190      0.22    0.1%
125829     309.1      0.67    0.2%
188743     422.4     0.772    0.1%
251657     543.1     3.194    0.5%
314571     660.1     1.257    0.1%
377485     785.8     10.82    1.3%
440399     911.3     12.73    1.3%
503313      1023     11.16    1.0%
566227      1135      5.87    0.5%
629141      1267      14.7    1.1%
692055      1388     13.89    1.0%
754969      1498     14.27    0.9%
817883      1623     13.08    0.8%
880797      1747     12.83    0.7%
943711      1869     14.18    0.7%
1006625      1994      13.1    0.6%
1069539      2150     11.87    0.5%
1132453      2268     7.565    0.3%
1195367      2371     13.22    0.5%
1258281      2508     14.72    0.5%
1321195      2660     13.54    0.5%
1384109      2764     12.51    0.4%
1447023      2893     14.71    0.5%
1509937      3038     8.303    0.2%
1572851      3166     15.54    0.4%
1635765      3309     24.01    0.7%
1698679      3437     13.91    0.4%
1761593      3572     13.65    0.3%
1824507      3716     12.63    0.3%
1887421      3829     22.02    0.5%
1950335      3958     9.996    0.2%
2013249      4103     12.86    0.3%
2076163      4241     10.18    0.2%
2139077      4368      14.3    0.3%
2201991      4525     21.78    0.4%
2264905      4651      17.3    0.3%
2327819      4782     11.28    0.2%
2390733      4996     13.25    0.2%
2453647      5031     11.59    0.2%
2516561      5169     9.553    0.1%
2579475      5315      16.6    0.3%
2642389      5443     12.58    0.2%
2705303      5570     9.603    0.1%
2768217      5721     15.48    0.2%
2831131      5838     13.92    0.2%
2894045      6011     32.93    0.5%
2956959      6135      21.7    0.3%
3019873      6249     16.67    0.2%
3082787      6370     11.99    0.1%
3145701      6543     22.51    0.3%

Quality and confidence:
param     error
c             0

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 9 + (0 * c)
Writes = 8 + (0 * c)

Pallet: "runtime_parachains::paras", Extrinsic: "force_note_new_head", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras FutureCodeUpgrades (r:1 w:0)
Storage: Paras Heads (r:0 w:1)
Storage: Paras UpgradeGoAheadSignal (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    17.29
    + s        0
              µs

Reads = 1 + (0 * s)
Writes = 2 + (0 * s)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    s   mean µs  sigma µs       %
    1     10.97     0.179    1.6%
20972      22.9     0.262    1.1%
41943     49.45     0.173    0.3%
62914     65.48     0.301    0.4%
83885     81.55     0.242    0.2%
104856     97.75     0.229    0.2%
125827     113.2     0.534    0.4%
146798     126.6     0.769    0.6%
167769     141.9     0.277    0.1%
188740     153.6     0.265    0.1%
209711     172.2      0.45    0.2%
230682     184.1     0.843    0.4%
251653     198.6     0.595    0.2%
272624     215.4     2.078    0.9%
293595     234.4      2.17    0.9%
314566     248.7     0.292    0.1%
335537     264.8     0.759    0.2%
356508     279.8     0.458    0.1%
377479     295.1     0.906    0.3%
398450     310.8     0.872    0.2%
419421     317.7     0.365    0.1%
440392     333.2     1.062    0.3%
461363     349.5       1.8    0.5%
482334     362.9     0.671    0.1%
503305     377.1     0.764    0.2%
524276     402.4     0.737    0.1%
545247     416.8     1.225    0.2%
566218     435.5     2.309    0.5%
587189     450.2     1.481    0.3%
608160     467.4     1.775    0.3%
629131     481.3     1.042    0.2%
650102     500.4     3.245    0.6%
671073     510.8      0.96    0.1%
692044     513.8     1.415    0.2%
713015     540.9     10.31    1.9%
733986     549.8     10.39    1.8%
754957     596.3     18.19    3.0%
775928     585.1     14.01    2.3%
796899     599.7     14.31    2.3%
817870     615.4     9.887    1.6%
838841       628     9.476    1.5%
859812     653.6     9.573    1.4%
880783     664.7     10.32    1.5%
901754     679.3     6.196    0.9%
922725     699.7     13.24    1.8%
943696     713.3     13.75    1.9%
964667     725.6     9.308    1.2%
985638     747.3     11.78    1.5%
1006609     760.7     8.782    1.1%
1027580     797.5     6.118    0.7%
1048551       807     13.12    1.6%

Quality and confidence:
param     error
s             0

Model:
Time ~=    13.76
    + s    0.001
              µs

Reads = 1 + (0 * s)
Writes = 2 + (0 * s)

Pallet: "runtime_parachains::paras", Extrinsic: "force_queue_action", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras ActionsQueue (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    15.94
              µs

Reads = 2
Writes = 1

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    15.94
              µs

Reads = 2
Writes = 1

Pallet: "runtime_parachains::paras", Extrinsic: "add_trusted_validation_code", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras PvfActiveVoteMap (r:1 w:0)
Storage: Paras CodeByHash (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 2 + (0 * c)
Writes = 1 + (0 * c)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    c   mean µs  sigma µs       %
    1     4.398     0.051    1.1%
62915     136.8     0.483    0.3%
125829     257.6     2.542    0.9%
188743     370.7     1.698    0.4%
251657     488.4     0.646    0.1%
314571       620     8.345    1.3%
377485     734.3     15.19    2.0%
440399     856.1     14.47    1.6%
503313     972.7     15.44    1.5%
566227      1093     15.13    1.3%
629141      1206     12.64    1.0%
692055      1326     13.38    1.0%
754969      1445     13.37    0.9%
817883      1577     12.88    0.8%
880797      1699     12.95    0.7%
943711      1816     16.26    0.8%
1006625      1954     12.01    0.6%
1069539      2066     15.23    0.7%
1132453      2195      17.7    0.8%
1195367      2299     10.65    0.4%
1258281      2443     18.04    0.7%
1321195      2576     20.52    0.7%
1384109      2693     16.85    0.6%
1447023      2836     16.92    0.5%
1509937      2988     16.76    0.5%
1572851      3107     6.628    0.2%
1635765      3239     10.69    0.3%
1698679      3366     20.25    0.6%
1761593      3526     23.82    0.6%
1824507      3640      16.7    0.4%
1887421      3767     14.76    0.3%
1950335      3904     11.22    0.2%
2013249      4034     16.24    0.4%
2076163      4173     21.46    0.5%
2139077      4298     11.08    0.2%
2201991      4432     16.52    0.3%
2264905      4556     15.34    0.3%
2327819      4696     16.87    0.3%
2390733      4841     14.83    0.3%
2453647      4954     15.12    0.3%
2516561      5101     19.86    0.3%
2579475      5233     12.32    0.2%
2642389      5372     36.56    0.6%
2705303      5492     9.705    0.1%
2768217      5611     15.64    0.2%
2831131      5738     16.48    0.2%
2894045      5897     19.55    0.3%
2956959      6017     19.68    0.3%
3019873      6157     22.04    0.3%
3082787      6300     18.22    0.2%
3145701      6430     28.69    0.4%

Quality and confidence:
param     error
c             0

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 2 + (0 * c)
Writes = 1 + (0 * c)

Pallet: "runtime_parachains::paras", Extrinsic: "poke_unused_validation_code", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras CodeByHashRefs (r:1 w:0)
Storage: Paras CodeByHash (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    2.509
              µs

Reads = 1
Writes = 1

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    2.509
              µs

Reads = 1
Writes = 1

Pallet: "runtime_parachains::paras", Extrinsic: "include_pvf_check_statement", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras PvfActiveVoteMap (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    117.6
              µs

Reads = 4
Writes = 1

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    117.6
              µs

Reads = 4
Writes = 1

Pallet: "runtime_parachains::paras", Extrinsic: "include_pvf_check_statement_finalize_upgrade_accept", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras PvfActiveVoteMap (r:1 w:1)
Storage: Paras PvfActiveVoteList (r:1 w:1)
Storage: Paras UpcomingUpgrades (r:1 w:1)
Storage: System Digest (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:0 w:100)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    618.7
              µs

Reads = 7
Writes = 104

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    618.7
              µs

Reads = 7
Writes = 104

Pallet: "runtime_parachains::paras", Extrinsic: "include_pvf_check_statement_finalize_upgrade_reject", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras PvfActiveVoteMap (r:1 w:1)
Storage: Paras PvfActiveVoteList (r:1 w:1)
Storage: Paras CodeByHashRefs (r:1 w:1)
Storage: Paras CodeByHash (r:0 w:1)
Storage: Paras UpgradeGoAheadSignal (r:0 w:100)
Storage: Paras FutureCodeHash (r:0 w:100)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    553.3
              µs

Reads = 6
Writes = 204

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    553.3
              µs

Reads = 6
Writes = 204

Pallet: "runtime_parachains::paras", Extrinsic: "include_pvf_check_statement_finalize_onboarding_accept", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras PvfActiveVoteMap (r:1 w:1)
Storage: Paras PvfActiveVoteList (r:1 w:1)
Storage: Paras ActionsQueue (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    507.5
              µs

Reads = 6
Writes = 3

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    507.5
              µs

Reads = 6
Writes = 3

Pallet: "runtime_parachains::paras", Extrinsic: "include_pvf_check_statement_finalize_onboarding_reject", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Configuration ActiveConfig (r:1 w:0)
Storage: ParasShared ActiveValidatorKeys (r:1 w:0)
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras PvfActiveVoteMap (r:1 w:1)
Storage: Paras PvfActiveVoteList (r:1 w:1)
Storage: Paras CodeByHashRefs (r:1 w:1)
Storage: Paras ParaLifecycles (r:0 w:100)
Storage: Paras CodeByHash (r:0 w:1)
Storage: Paras CurrentCodeHash (r:0 w:100)
Storage: Paras UpcomingParasGenesis (r:0 w:100)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    609.8
              µs

Reads = 6
Writes = 304

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    609.8
              µs

Reads = 6
Writes = 304

@pepyakin pepyakin force-pushed the pep-paras-initialize-now-refactors branch from f80186a to 92c0ce5 Compare February 21, 2022 17:31
@pepyakin pepyakin force-pushed the pep-pvf-include-pvf-check-benchmark branch from 853f8a0 to 35bb16c Compare February 21, 2022 17:31
@pepyakin pepyakin force-pushed the pep-paras-initialize-now-refactors branch from 92c0ce5 to 2a2b2f1 Compare February 21, 2022 18:05
@pepyakin pepyakin force-pushed the pep-pvf-include-pvf-check-benchmark branch 2 times, most recently from 3cbd9aa to bc536c8 Compare February 21, 2022 18:44
Base automatically changed from pep-paras-initialize-now-refactors to master February 22, 2022 13:52
pepyakin and others added 5 commits February 22, 2022 13:52
@pepyakin
paras: include_pvf_check_statement rt bench
2902f62 
Resolves #4933

This PR adds a benchmark for the `include_pvf_check_statement`
dispatchable. This is a necessary step to make it work without
modifications. That enables us to proceed with testing on Versi.

This introduces 5 new benchmarks. Those measure performance of the
`include_pvf_check_statement` under 2 different conditions:

1. regular vote submission. That's the common case.
2. submission of the last vote. That happens only once and leads to a
   heavy finalization stage.

There are 2 different types of finalization (one for onboarding, one for
upgrading) and there are two outcomes: accepted and rejected. Those 4
are similar but I decided to cover them all and assign the maximum of
all 4. This is to avoid a situation when one of those paths becomes more
heavier than others and opens up an attack venue.

The regular vote submission weight is drastically different from the
submission last vote weight. That's why in case during runtime
finalization was not executed the weight consumed value will be lowered
down to the regular vote submission.

The finalization weight is proportional to the number of "causes", i.e.
the events that caused the PVF pre-checking vote in the first place, and
here we assume that the maximum number of causes is 100.

Theoretically, there is nothing that prevents an adversary to
register/upgrade to more than 100 parachains. In that case, the consumed
weight will be lower than the actual time consumed by the finalization
process. That can enable a DoS vector.

However, practically, it is not very possible. Right now it is very
expensive to call `schedule_para_initialize` because it requires a very
large lock up of funds. Moreover, finalizing a vote with 100 causes
leads to around 31ms time spent. Finalizing more will require more time.
However, finalizing with 200 causes will cause ≈62ms delay. This is not
that bad since even though we had a full block and the adversary tried
to finalize 200 causes it won't be able to even exceed the operational
extrinsic boundary of 250ms and even if so it won't make big difference.

That said, this should be addressed later on, esp. when we enable
parathreads, which will make creating causes easier. One of potential
solutions will be shifting the logic of finalization into
`on_initialize`/`on_finalize`. Another is to create a maximum number of
causes and then reject upgrades or onboardings if that was reached.
@pepyakin
cargo run --quiet --profile=production --features=runtime-benchmarks …
608be47 
…-- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras.rs
@pepyakin
cargo run --quiet --profile=production --features=runtime-benchmarks …
c7bf8c4 
…-- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras.rs
@pepyakin
cargo run --quiet --profile=production --features=runtime-benchmarks …
b15ec61 
…-- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs
@pepyakin
cargo run --quiet --profile=production --features runtime-benchmarks …
be6b1da 
…-- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/rococo/src/weights/runtime_parachains_paras.rs
@pepyakin pepyakin force-pushed the pep-pvf-include-pvf-check-benchmark branch from bc536c8 to be6b1da Compare February 22, 2022 13:53
Lldenaurois
Lldenaurois approved these changes Mar 16, 2022
View reviewed changes
Copy link
Contributor

@Lldenaurois Lldenaurois left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eskimor
Copy link
Member

eskimor commented Mar 16, 2022

Damn it. Can't merge because of conflicts. Could you quickly resolve them @Lldenaurois ? Thanks for the review!

@eskimor
Copy link
Member

eskimor commented Mar 23, 2022

/benchmark runtime westend runtime_parachains::paras

@parity-benchapp
Copy link

parity-benchapp bot commented Mar 23, 2022 •
edited
Loading

Error running benchmark: pep-pvf-include-pvf-check-benchmark

stdoutFrom https://github.com/paritytech/polkadot * branch master -> FETCH_HEAD 6d5578d..1ec8f9e master -> origin/master

@eskimor eskimor merged commit ebaa924 into master Mar 24, 2022
@eskimor eskimor deleted the pep-pvf-include-pvf-check-benchmark branch March 24, 2022 16:52
ordian added a commit that referenced this pull request Mar 24, 2022
@ordian
Merge branch 'master' into ao-remove-old-migrations
b323434 
* master:
  Try to fix out of view statements (#5177)
  Companion for Substrate#11107 (#5197)
  paras: `include_pvf_check_statement` rt bench (#4938)
  [ci] Run short benchmarks only in PR pipelines (#5200)
  Companion for paritytech/substrate#10242 (#4862)
  [ci] Add short benchmarks to the pipeline (#5188)
  upgrade coarsetime to 0.1.22 to fix a potential panic (#5193)
  Update docs and enable DOT-over-XCM (#4809)
  enable disputes on all chains (#5182)
  companion for validator self-vote in bags (#5088)
  Extract MAX_FINALITY_LAG constant from relay_chain_selection (#5159)
@slumber slumber mentioned this pull request Mar 21, 2023
Merged
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@Lldenaurois Lldenaurois Lldenaurois approved these changes

Assignees
No one assigned
Labels
A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

weights for include_pvf_check_statement
3 participants
@pepyakin @eskimor @Lldenaurois