Make election benchmarks more memory-aware #9286

kianenigma · 2021-07-06T09:52:04Z

Related to #9285

For now I added two new benchmarks. They can only be executed in isolation, and if they work, then it is good news.

polkadot companion: paritytech/polkadot#3443

kianenigma · 2021-07-06T11:56:47Z

/benchmark pallet pallet_election_provider_multi_phase

parity-benchapp · 2021-07-06T11:56:49Z

Starting benchmark for branch: kiz-mem-aware-benchmarks (vs master)

Comment will be updated.

shawntabrizi · 2021-07-07T01:09:15Z

/benchmark pallet pallet_election_provider_multi_phase

parity-benchapp · 2021-07-07T01:09:17Z

Starting benchmark for branch: kiz-mem-aware-benchmarks (vs master)

Comment will be updated.

shawntabrizi · 2021-07-07T01:09:58Z

/benchmark runtime pallet pallet_election_provider_multi_phase

parity-benchapp · 2021-07-07T01:09:59Z

Starting benchmark for branch: kiz-mem-aware-benchmarks (vs master)

Comment will be updated.

joao-paulo-parity · 2021-07-07T04:43:31Z

/benchmark runtime pallet pallet_election_provider_multi_phase

parity-benchapp · 2021-07-07T04:43:33Z

Benchmark Runtime Pallet for branch "kiz-mem-aware-benchmarks" with command cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs

Results

Pallet: "pallet_election_provider_multi_phase", Extrinsic: "on_initialize_nothing", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    33.17
              µs

Reads = 8
Writes = 0
Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    33.17
              µs

Reads = 8
Writes = 0
Pallet: "pallet_election_provider_multi_phase", Extrinsic: "on_initialize_open_signed", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    113.6
              µs

Reads = 10
Writes = 4
Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    113.6
              µs

Reads = 10
Writes = 4
Pallet: "pallet_election_provider_multi_phase", Extrinsic: "on_initialize_open_unsigned_with_snapshot", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    113.6
              µs

Reads = 10
Writes = 4
Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    113.6
              µs

Reads = 10
Writes = 4
Pallet: "pallet_election_provider_multi_phase", Extrinsic: "finalize_signed_phase_accept_solution", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    60.18
              µs

Reads = 1
Writes = 2
Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    60.18
              µs

Reads = 1
Writes = 2
Pallet: "pallet_election_provider_multi_phase", Extrinsic: "finalize_signed_phase_reject_solution", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    40.15
              µs

Reads = 1
Writes = 1
Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    40.15
              µs

Reads = 1
Writes = 1
Pallet: "pallet_election_provider_multi_phase", Extrinsic: "on_initialize_open_unsigned_without_snapshot", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    23.83
              µs

Reads = 1
Writes = 1
Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    23.83
              µs

Reads = 1
Writes = 1
Pallet: "pallet_election_provider_multi_phase", Extrinsic: "elect_queued", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    59.24
    + v    0.012
    + t        0
    + a    1.955
    + d    0.483
              µs

Reads = 6 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Writes = 8 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    v     t     a     d   mean µs  sigma µs       %
 5000  2000  4000   800      8328     34.71    0.4%
 5100  2000  4000   800      8338     23.71    0.2%
 5200  2000  4000   800      8288     36.65    0.4%
 5300  2000  4000   800      8323     55.87    0.6%
 5400  2000  4000   800      8359     26.86    0.3%
 5500  2000  4000   800      8284     59.33    0.7%
 5600  2000  4000   800      8280     24.42    0.2%
 5700  2000  4000   800      8257     32.81    0.3%
 5800  2000  4000   800      8317     83.19    1.0%
 5900  2000  4000   800      8304     93.49    1.1%
 6000  2000  4000   800      8274     17.09    0.2%
 6100  2000  4000   800      8233     40.14    0.4%
 6200  2000  4000   800      8281     118.9    1.4%
 6300  2000  4000   800      8249     39.65    0.4%
 6400  2000  4000   800      8253     31.67    0.3%
 6500  2000  4000   800      8254     37.67    0.4%
 6600  2000  4000   800      8285      45.6    0.5%
 6700  2000  4000   800      8265     22.39    0.2%
 6800  2000  4000   800      8256     33.78    0.4%
 6900  2000  4000   800      8306     41.64    0.5%
 7000  2000  4000   800      8285     21.24    0.2%
 7100  2000  4000   800      8272     52.62    0.6%
 7200  2000  4000   800      8352      62.1    0.7%
 7300  2000  4000   800      8277     24.45    0.2%
 7400  2000  4000   800      8315     33.84    0.4%
 7500  2000  4000   800      8293     30.35    0.3%
 7600  2000  4000   800      8311     38.53    0.4%
 7700  2000  4000   800      8254     35.37    0.4%
 7800  2000  4000   800      8373     184.1    2.1%
 7900  2000  4000   800      8294     35.23    0.4%
 8000  2000  4000   800      8298     58.69    0.7%
 8100  2000  4000   800      8237      44.5    0.5%
 8200  2000  4000   800      8349     153.6    1.8%
 8300  2000  4000   800      8279     47.94    0.5%
 8400  2000  4000   800      8302     37.21    0.4%
 8500  2000  4000   800      8274     29.52    0.3%
 8600  2000  4000   800      8336     35.37    0.4%
 8700  2000  4000   800      8314     65.65    0.7%
 8800  2000  4000   800      8380     61.74    0.7%
 8900  2000  4000   800      8320     63.33    0.7%
 9000  2000  4000   800      8409     79.08    0.9%
 9100  2000  4000   800      8365     66.43    0.7%
 9200  2000  4000   800      8465     97.84    1.1%
 9300  2000  4000   800      8332     44.23    0.5%
 9400  2000  4000   800      8359     48.07    0.5%
 9500  2000  4000   800      8327      48.6    0.5%
 9600  2000  4000   800      8350      29.3    0.3%
 9700  2000  4000   800      8337      33.3    0.3%
 9800  2000  4000   800      8354     59.52    0.7%
 9900  2000  4000   800      8382     100.4    1.1%
10000  1000  4000   800      8323     34.78    0.4%
10000  1020  4000   800      8389     54.83    0.6%
10000  1040  4000   800      8388     90.99    1.0%
10000  1060  4000   800      8403     96.61    1.1%
10000  1080  4000   800      8389     75.23    0.8%
10000  1100  4000   800      8468     125.6    1.4%
10000  1120  4000   800      8306     40.28    0.4%
10000  1140  4000   800      8350     63.84    0.7%
10000  1160  4000   800      8392     92.45    1.1%
10000  1180  4000   800      8324     36.35    0.4%
10000  1200  4000   800      8357     35.85    0.4%
10000  1220  4000   800      8386     58.23    0.6%
10000  1240  4000   800      8344      43.6    0.5%
10000  1260  4000   800      8343     65.55    0.7%
10000  1280  4000   800      8317     24.28    0.2%
10000  1300  4000   800      8363     24.41    0.2%
10000  1320  4000   800      8373     74.85    0.8%
10000  1340  4000   800      8376     45.85    0.5%
10000  1360  4000   800      8354     37.45    0.4%
10000  1380  4000   800      8360     22.61    0.2%
10000  1400  4000   800      8382     53.51    0.6%
10000  1420  4000   800      8366     53.39    0.6%
10000  1440  4000   800      8387     38.61    0.4%
10000  1460  4000   800      8348     22.38    0.2%
10000  1480  4000   800      8303      27.9    0.3%
10000  1500  4000   800      8327     24.09    0.2%
10000  1520  4000   800      8318      39.3    0.4%
10000  1540  4000   800      8393     53.77    0.6%
10000  1560  4000   800      8282      25.1    0.3%
10000  1580  4000   800      8362     80.85    0.9%
10000  1600  4000   800      8443     189.3    2.2%
10000  1620  4000   800      8396     121.9    1.4%
10000  1640  4000   800      8334     41.99    0.5%
10000  1660  4000   800      8372     30.35    0.3%
10000  1680  4000   800      8393     40.76    0.4%
10000  1700  4000   800      8375      82.8    0.9%
10000  1720  4000   800      8375     70.02    0.8%
10000  1740  4000   800      8355      27.2    0.3%
10000  1760  4000   800      8360     46.33    0.5%
10000  1780  4000   800      8381     61.65    0.7%
10000  1800  4000   800      8345     55.91    0.6%
10000  1820  4000   800      8360     75.43    0.9%
10000  1840  4000   800      8328     34.09    0.4%
10000  1860  4000   800      8365     51.14    0.6%
10000  1880  4000   800      8293     29.65    0.3%
10000  1900  4000   800      8239     10.25    0.1%
10000  1920  4000   800      8332     60.91    0.7%
10000  1940  4000   800      8342     26.92    0.3%
10000  1960  4000   800      8325     66.36    0.7%
10000  1980  4000   800      8328     51.69    0.6%
10000  2000  1000   800      2477     16.74    0.6%
10000  2000  1060   800      2606     20.49    0.7%
10000  2000  1120   800      2690        13    0.4%
10000  2000  1180   800      2842     39.38    1.3%
10000  2000  1240   800      2902     14.19    0.4%
10000  2000  1300   800      3031     20.58    0.6%
10000  2000  1360   800      3178     54.99    1.7%
10000  2000  1420   800      3241     23.97    0.7%
10000  2000  1480   800      3399     36.17    1.0%
10000  2000  1540   800      3520     13.76    0.3%
10000  2000  1600   800      3609     10.85    0.3%
10000  2000  1660   800      3739     64.31    1.7%
10000  2000  1720   800      3828     13.66    0.3%
10000  2000  1780   800      3938     28.23    0.7%
10000  2000  1840   800      4070     31.28    0.7%
10000  2000  1900   800      4201     17.28    0.4%
10000  2000  1960   800      4335     23.13    0.5%
10000  2000  2020   800      4443     33.13    0.7%
10000  2000  2080   800      4548      23.1    0.5%
10000  2000  2140   800      4722     23.63    0.5%
10000  2000  2200   800      4818     34.23    0.7%
10000  2000  2260   800      4925     26.12    0.5%
10000  2000  2320   800      5042     24.19    0.4%
10000  2000  2380   800      5115     22.43    0.4%
10000  2000  2440   800      5319     70.84    1.3%
10000  2000  2500   800      5457       103    1.8%
10000  2000  2560   800      5521     37.94    0.6%
10000  2000  2620   800      5564     28.05    0.5%
10000  2000  2680   800      5762     43.21    0.7%
10000  2000  2740   800      5942     90.01    1.5%
10000  2000  2800   800      6056     47.05    0.7%
10000  2000  2860   800      6092     62.31    1.0%
10000  2000  2920   800      6186     30.75    0.4%
10000  2000  2980   800      6317     26.04    0.4%
10000  2000  3040   800      6426     43.14    0.6%
10000  2000  3100   800      6534     25.65    0.3%
10000  2000  3160   800      6627     29.16    0.4%
10000  2000  3220   800      6746     33.81    0.5%
10000  2000  3280   800      7040     120.1    1.7%
10000  2000  3340   800      7005     28.87    0.4%
10000  2000  3400   800      7111     48.76    0.6%
10000  2000  3460   800      7242     40.82    0.5%
10000  2000  3520   800      7417     75.95    1.0%
10000  2000  3580   800      7548     65.78    0.8%
10000  2000  3640   800      7620     39.12    0.5%
10000  2000  3700   800      7850     21.54    0.2%
10000  2000  3760   800      7922     56.42    0.7%
10000  2000  3820   800      7951     38.47    0.4%
10000  2000  3880   800      7993     53.85    0.6%
10000  2000  3940   800      8284     131.8    1.5%
10000  2000  4000   400      8157     18.48    0.2%
10000  2000  4000   408      8210     81.72    0.9%
10000  2000  4000   416      8230     58.38    0.7%
10000  2000  4000   424      8178     33.16    0.4%
10000  2000  4000   432      8216     40.08    0.4%
10000  2000  4000   440      8178     50.04    0.6%
10000  2000  4000   448      8149     40.42    0.4%
10000  2000  4000   456      8183     26.61    0.3%
10000  2000  4000   464      8135     41.33    0.5%
10000  2000  4000   472      8148     47.43    0.5%
10000  2000  4000   480      8075      31.3    0.3%
10000  2000  4000   488      8091     18.44    0.2%
10000  2000  4000   496      8230     160.8    1.9%
10000  2000  4000   504      8109     12.68    0.1%
10000  2000  4000   512      8093     26.67    0.3%
10000  2000  4000   520      8137     58.32    0.7%
10000  2000  4000   528      8085     13.48    0.1%
10000  2000  4000   536      8163     61.24    0.7%
10000  2000  4000   544      8100     16.51    0.2%
10000  2000  4000   552      8171      74.6    0.9%
10000  2000  4000   560      8118     28.64    0.3%
10000  2000  4000   568      8213     99.62    1.2%
10000  2000  4000   576      8127     32.36    0.3%
10000  2000  4000   584      8126     14.62    0.1%
10000  2000  4000   592      8131     30.79    0.3%
10000  2000  4000   600      8161      34.9    0.4%
10000  2000  4000   608      8136     32.86    0.4%
10000  2000  4000   616      8160     24.95    0.3%
10000  2000  4000   624      8119      20.4    0.2%
10000  2000  4000   632      8153     18.46    0.2%
10000  2000  4000   640      8150     42.54    0.5%
10000  2000  4000   648      8184      17.7    0.2%
10000  2000  4000   656      8198     71.62    0.8%
10000  2000  4000   664      8196     20.79    0.2%
10000  2000  4000   672      8258      39.2    0.4%
10000  2000  4000   680      8258      48.8    0.5%
10000  2000  4000   688      8288      42.5    0.5%
10000  2000  4000   696      8257      37.7    0.4%
10000  2000  4000   704      8241     40.76    0.4%
10000  2000  4000   712      8319     74.73    0.8%
10000  2000  4000   720      8397       150    1.7%
10000  2000  4000   728      8277     24.04    0.2%
10000  2000  4000   736      8236     48.09    0.5%
10000  2000  4000   744      8265     35.02    0.4%
10000  2000  4000   752      8274     39.96    0.4%
10000  2000  4000   760      8287     59.74    0.7%
10000  2000  4000   768      8267     47.16    0.5%
10000  2000  4000   776      8333     100.6    1.2%
10000  2000  4000   784      8287     48.45    0.5%
10000  2000  4000   792      8326     30.68    0.3%
10000  2000  4000   800      8327     42.13    0.5%

Quality and confidence:
param     error
v         0.001
t         0.007
a         0.002
d         0.018

Model:
Time ~=    51.57
    + v    0.009
    + t        0
    + a    1.957
    + d    0.588
              µs

Reads = 6 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Writes = 8 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Pallet: "pallet_election_provider_multi_phase", Extrinsic: "submit", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    77.48
    + c    0.279
              µs

Reads = 4 + (0 * c)
Writes = 3 + (0 * c)
Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    c   mean µs  sigma µs       %
    1     77.79      0.21    0.2%
    2     77.89     0.097    0.1%
    3     78.69     0.164    0.2%
    4     77.99     0.163    0.2%
    5     79.24     0.105    0.1%
    6     78.85     0.121    0.1%
    7     80.04     0.097    0.1%
    8     79.22     0.227    0.2%
    9     80.12     0.157    0.1%

Quality and confidence:
param     error
c         0.017

Model:
Time ~=    77.46
    + c    0.281
              µs

Reads = 4 + (0 * c)
Writes = 3 + (0 * c)
Pallet: "pallet_election_provider_multi_phase", Extrinsic: "submit_unsigned", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + v      3.6
    + t    0.483
    + a    11.38
    + d    4.175
              µs

Reads = 7 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Writes = 1 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    v     t     a     d   mean µs  sigma µs       %
 5000  2000  4000   800     62800     122.2    0.1%
 5100  2000  4000   800     63110     78.36    0.1%
 5200  2000  4000   800     63560     142.7    0.2%
 5300  2000  4000   800     63960     117.3    0.1%
 5400  2000  4000   800     64180     108.2    0.1%
 5500  2000  4000   800     64650     39.67    0.0%
 5600  2000  4000   800     64900        72    0.1%
 5700  2000  4000   800     65230     113.2    0.1%
 5800  2000  4000   800     65580     98.34    0.1%
 5900  2000  4000   800     65900     62.59    0.0%
 6000  2000  4000   800     66190     73.43    0.1%
 6100  2000  4000   800     66550     103.5    0.1%
 6200  2000  4000   800     66920     74.47    0.1%
 6300  2000  4000   800     67250     93.73    0.1%
 6400  2000  4000   800     67830       213    0.3%
 6500  2000  4000   800     67970     89.45    0.1%
 6600  2000  4000   800     68400     58.64    0.0%
 6700  2000  4000   800     68780     96.04    0.1%
 6800  2000  4000   800     69010     90.39    0.1%
 6900  2000  4000   800     69330     53.59    0.0%
 7000  2000  4000   800     69840     87.94    0.1%
 7100  2000  4000   800     70420     148.2    0.2%
 7200  2000  4000   800     70850     84.87    0.1%
 7300  2000  4000   800     70950     98.52    0.1%
 7400  2000  4000   800     71220     97.53    0.1%
 7500  2000  4000   800     71590     75.83    0.1%
 7600  2000  4000   800     71870     44.11    0.0%
 7700  2000  4000   800     72280     85.11    0.1%
 7800  2000  4000   800     72750     74.89    0.1%
 7900  2000  4000   800     73100     45.23    0.0%
 8000  2000  4000   800     73500     99.42    0.1%
 8100  2000  4000   800     73720     61.51    0.0%
 8200  2000  4000   800     74160     97.52    0.1%
 8300  2000  4000   800     74500     91.52    0.1%
 8400  2000  4000   800     74900     121.8    0.1%
 8500  2000  4000   800     75250     92.67    0.1%
 8600  2000  4000   800     75590     79.96    0.1%
 8700  2000  4000   800     76030     61.89    0.0%
 8800  2000  4000   800     76330     61.85    0.0%
 8900  2000  4000   800     76820     65.65    0.0%
 9000  2000  4000   800     77060     67.62    0.0%
 9100  2000  4000   800     77470     85.04    0.1%
 9200  2000  4000   800     77710     69.94    0.0%
 9300  2000  4000   800     78050      58.8    0.0%
 9400  2000  4000   800     78410     97.86    0.1%
 9500  2000  4000   800     78890      90.2    0.1%
 9600  2000  4000   800     79110     24.47    0.0%
 9700  2000  4000   800     79710     159.1    0.1%
 9800  2000  4000   800     79920     98.06    0.1%
 9900  2000  4000   800     80220     83.57    0.1%
10000  1000  4000   800     80530     48.84    0.0%
10000  1020  4000   800     80570     100.2    0.1%
10000  1040  4000   800     80320     47.22    0.0%
10000  1060  4000   800     80560     94.34    0.1%
10000  1080  4000   800     80410     44.95    0.0%
10000  1100  4000   800     80390     71.56    0.0%
10000  1120  4000   800     80550     96.74    0.1%
10000  1140  4000   800     80510     57.18    0.0%
10000  1160  4000   800     80390      58.6    0.0%
10000  1180  4000   800     80470     85.67    0.1%
10000  1200  4000   800     80440     85.54    0.1%
10000  1220  4000   800     80480     61.05    0.0%
10000  1240  4000   800     80500     107.6    0.1%
10000  1260  4000   800     80510     159.9    0.1%
10000  1280  4000   800     80540     111.2    0.1%
10000  1300  4000   800     80470     77.89    0.0%
10000  1320  4000   800     80530     109.7    0.1%
10000  1340  4000   800     80530     109.4    0.1%
10000  1360  4000   800     80830     65.78    0.0%
10000  1380  4000   800     80700      98.8    0.1%
10000  1400  4000   800     80650       124    0.1%
10000  1420  4000   800     80690     183.2    0.2%
10000  1440  4000   800     80670     81.83    0.1%
10000  1460  4000   800     80640     55.74    0.0%
10000  1480  4000   800     80570     64.47    0.0%
10000  1500  4000   800     80600     77.26    0.0%
10000  1520  4000   800     80520     55.37    0.0%
10000  1540  4000   800     80710     89.85    0.1%
10000  1560  4000   800     80600     121.3    0.1%
10000  1580  4000   800     80590     78.28    0.0%
10000  1600  4000   800     80640     96.85    0.1%
10000  1620  4000   800     80910     127.5    0.1%
10000  1640  4000   800     80700        89    0.1%
10000  1660  4000   800     80760     68.84    0.0%
10000  1680  4000   800     80820     169.5    0.2%
10000  1700  4000   800     80830      70.4    0.0%
10000  1720  4000   800     80770      85.8    0.1%
10000  1740  4000   800     80770        61    0.0%
10000  1760  4000   800     80960     48.84    0.0%
10000  1780  4000   800     80780     88.04    0.1%
10000  1800  4000   800     80770     82.49    0.1%
10000  1820  4000   800     80770     55.19    0.0%
10000  1840  4000   800     80680     66.55    0.0%
10000  1860  4000   800     80700     115.3    0.1%
10000  1880  4000   800     80810     88.58    0.1%
10000  1900  4000   800     80740     76.02    0.0%
10000  1920  4000   800     81110     199.6    0.2%
10000  1940  4000   800     80700     104.3    0.1%
10000  1960  4000   800     80720     53.49    0.0%
10000  1980  4000   800     81320     71.06    0.0%
10000  2000  1000   800     46920     173.4    0.3%
10000  2000  1060   800     47610     180.8    0.3%
10000  2000  1120   800     48600       169    0.3%
10000  2000  1180   800     48910     205.1    0.4%
10000  2000  1240   800     49450     195.3    0.3%
10000  2000  1300   800     50190     224.8    0.4%
10000  2000  1360   800     51030     192.7    0.3%
10000  2000  1420   800     52020     155.2    0.2%
10000  2000  1480   800     52770       169    0.3%
10000  2000  1540   800     53460     53.13    0.0%
10000  2000  1600   800     54050     105.4    0.1%
10000  2000  1660   800     54910     112.2    0.2%
10000  2000  1720   800     55420     146.7    0.2%
10000  2000  1780   800     56290     134.4    0.2%
10000  2000  1840   800     56840     117.4    0.2%
10000  2000  1900   800     57630     188.3    0.3%
10000  2000  1960   800     58340     171.3    0.2%
10000  2000  2020   800     58640     122.5    0.2%
10000  2000  2080   800     59490       169    0.2%
10000  2000  2140   800     60050     108.4    0.1%
10000  2000  2200   800     60640     164.9    0.2%
10000  2000  2260   800     61330     189.1    0.3%
10000  2000  2320   800     61890     87.44    0.1%
10000  2000  2380   800     62390     123.5    0.1%
10000  2000  2440   800     63000     193.7    0.3%
10000  2000  2500   800     63550     96.96    0.1%
10000  2000  2560   800     64700     185.2    0.2%
10000  2000  2620   800     64970     120.3    0.1%
10000  2000  2680   800     65700     155.8    0.2%
10000  2000  2740   800     66290     223.1    0.3%
10000  2000  2800   800     66980     153.7    0.2%
10000  2000  2860   800     68540     163.1    0.2%
10000  2000  2920   800     69240     131.8    0.1%
10000  2000  2980   800     69830     125.9    0.1%
10000  2000  3040   800     70580     164.8    0.2%
10000  2000  3100   800     71540     217.8    0.3%
10000  2000  3160   800     72030     300.5    0.4%
10000  2000  3220   800     72610     197.9    0.2%
10000  2000  3280   800     73170     125.4    0.1%
10000  2000  3340   800     74480     243.9    0.3%
10000  2000  3400   800     74690     204.7    0.2%
10000  2000  3460   800     75070     103.3    0.1%
10000  2000  3520   800     75830     164.3    0.2%
10000  2000  3580   800     76630     137.1    0.1%
10000  2000  3640   800     77380     130.8    0.1%
10000  2000  3700   800     77900     166.6    0.2%
10000  2000  3760   800     78560     133.7    0.1%
10000  2000  3820   800     79320     173.3    0.2%
10000  2000  3880   800     79860     57.33    0.0%
10000  2000  3940   800     80400     137.4    0.1%
10000  2000  4000   400     79000     144.8    0.1%
10000  2000  4000   408     79480     152.3    0.1%
10000  2000  4000   416     79270     224.5    0.2%
10000  2000  4000   424     79480     113.9    0.1%
10000  2000  4000   432     79720     173.7    0.2%
10000  2000  4000   440     80070     344.9    0.4%
10000  2000  4000   448     79550     161.2    0.2%
10000  2000  4000   456     79780     199.2    0.2%
10000  2000  4000   464     79540       169    0.2%
10000  2000  4000   472     79630       147    0.1%
10000  2000  4000   480     79710     118.8    0.1%
10000  2000  4000   488     79610       178    0.2%
10000  2000  4000   496     79850     323.2    0.4%
10000  2000  4000   504     79500     188.7    0.2%
10000  2000  4000   512     79770     172.1    0.2%
10000  2000  4000   520     80110       263    0.3%
10000  2000  4000   528     79380     124.1    0.1%
10000  2000  4000   536     79550     157.1    0.1%
10000  2000  4000   544     79410     124.3    0.1%
10000  2000  4000   552     79420     184.5    0.2%
10000  2000  4000   560     79460     193.8    0.2%
10000  2000  4000   568     79460     234.5    0.2%
10000  2000  4000   576     79590     154.1    0.1%
10000  2000  4000   584     79640     360.9    0.4%
10000  2000  4000   592     79420     107.8    0.1%
10000  2000  4000   600     79480     114.3    0.1%
10000  2000  4000   608     79610     178.6    0.2%
10000  2000  4000   616     79730     156.3    0.1%
10000  2000  4000   624     79740     132.2    0.1%
10000  2000  4000   632     79860     227.5    0.2%
10000  2000  4000   640     80030       217    0.2%
10000  2000  4000   648     80510     226.8    0.2%
10000  2000  4000   656     80080     272.5    0.3%
10000  2000  4000   664     80010     186.8    0.2%
10000  2000  4000   672     80080     166.6    0.2%
10000  2000  4000   680     80380     97.05    0.1%
10000  2000  4000   688     80270     119.4    0.1%
10000  2000  4000   696     80760     342.9    0.4%
10000  2000  4000   704     80210     115.6    0.1%
10000  2000  4000   712     80450     90.08    0.1%
10000  2000  4000   720     80700     148.9    0.1%
10000  2000  4000   728     80400     122.3    0.1%
10000  2000  4000   736     80560     197.7    0.2%
10000  2000  4000   744     80900     145.2    0.1%
10000  2000  4000   752     80940     178.8    0.2%
10000  2000  4000   760     80470     82.53    0.1%
10000  2000  4000   768     80780     102.3    0.1%
10000  2000  4000   776     80910     140.4    0.1%
10000  2000  4000   784     80950     102.5    0.1%
10000  2000  4000   792     81090     147.8    0.1%
10000  2000  4000   800     81000     161.9    0.1%

Quality and confidence:
param     error
v         0.005
t         0.029
a         0.009
d         0.073

Model:
Time ~=        0
    + v    3.667
    + t    0.497
    + a    11.22
    + d    4.432
              µs

Reads = 7 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Writes = 1 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Pallet: "pallet_election_provider_multi_phase", Extrinsic: "feasibility_check", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + v    3.592
    + t    0.351
    + a    9.789
    + d    3.679
              µs

Reads = 4 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Writes = 0 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    v     t     a     d   mean µs  sigma µs       %
 5000  2000  4000   800     56020     67.05    0.1%
 5100  2000  4000   800     56400     46.24    0.0%
 5200  2000  4000   800     56720     71.73    0.1%
 5300  2000  4000   800     57120     94.03    0.1%
 5400  2000  4000   800     57470     52.32    0.0%
 5500  2000  4000   800     57720     88.27    0.1%
 5600  2000  4000   800     58220     110.9    0.1%
 5700  2000  4000   800     58550     117.8    0.2%
 5800  2000  4000   800     58940     72.66    0.1%
 5900  2000  4000   800     59220     101.1    0.1%
 6000  2000  4000   800     59590     47.34    0.0%
 6100  2000  4000   800     59930     51.96    0.0%
 6200  2000  4000   800     60310     146.1    0.2%
 6300  2000  4000   800     60630     42.58    0.0%
 6400  2000  4000   800     61190     63.77    0.1%
 6500  2000  4000   800     61540     50.92    0.0%
 6600  2000  4000   800     61670     55.93    0.0%
 6700  2000  4000   800     62100     65.74    0.1%
 6800  2000  4000   800     62490     72.46    0.1%
 6900  2000  4000   800     62860     43.02    0.0%
 7000  2000  4000   800     63350     307.2    0.4%
 7100  2000  4000   800     63540     89.54    0.1%
 7200  2000  4000   800     63990     35.66    0.0%
 7300  2000  4000   800     64400     61.64    0.0%
 7400  2000  4000   800     64680     41.05    0.0%
 7500  2000  4000   800     64940     66.68    0.1%
 7600  2000  4000   800     65330     110.6    0.1%
 7700  2000  4000   800     65680     76.65    0.1%
 7800  2000  4000   800     65960     49.58    0.0%
 7900  2000  4000   800     66280     56.94    0.0%
 8000  2000  4000   800     66950     124.6    0.1%
 8100  2000  4000   800     67230     42.88    0.0%
 8200  2000  4000   800     67410     53.67    0.0%
 8300  2000  4000   800     67760     63.48    0.0%
 8400  2000  4000   800     68140     57.42    0.0%
 8500  2000  4000   800     68460     63.37    0.0%
 8600  2000  4000   800     68850     53.24    0.0%
 8700  2000  4000   800     69210     51.98    0.0%
 8800  2000  4000   800     69740     57.76    0.0%
 8900  2000  4000   800     70110     68.52    0.0%
 9000  2000  4000   800     70810     248.2    0.3%
 9100  2000  4000   800     70760     105.1    0.1%
 9200  2000  4000   800     71420     178.1    0.2%
 9300  2000  4000   800     71410     77.43    0.1%
 9400  2000  4000   800     71650     37.34    0.0%
 9500  2000  4000   800     72260     82.22    0.1%
 9600  2000  4000   800     72530     71.74    0.0%
 9700  2000  4000   800     72890     60.32    0.0%
 9800  2000  4000   800     73190     28.48    0.0%
 9900  2000  4000   800     73430     31.84    0.0%
10000  1000  4000   800     73720     38.45    0.0%
10000  1020  4000   800     73790     92.36    0.1%
10000  1040  4000   800     73720     68.27    0.0%
10000  1060  4000   800     73880     68.52    0.0%
10000  1080  4000   800     73810     134.5    0.1%
10000  1100  4000   800     73810     67.67    0.0%
10000  1120  4000   800     73880     100.2    0.1%
10000  1140  4000   800     73820     105.5    0.1%
10000  1160  4000   800     73730        53    0.0%
10000  1180  4000   800     73840     84.06    0.1%
10000  1200  4000   800     73990     185.6    0.2%
10000  1220  4000   800     73770     47.29    0.0%
10000  1240  4000   800     73720     52.88    0.0%
10000  1260  4000   800     73730     86.68    0.1%
10000  1280  4000   800     73850     65.25    0.0%
10000  1300  4000   800     73740     128.5    0.1%
10000  1320  4000   800     73850     132.4    0.1%
10000  1340  4000   800     73860     78.91    0.1%
10000  1360  4000   800     74030     54.23    0.0%
10000  1380  4000   800     73810     36.73    0.0%
10000  1400  4000   800     73810     58.53    0.0%
10000  1420  4000   800     73900      53.7    0.0%
10000  1440  4000   800     73840      85.1    0.1%
10000  1460  4000   800     73910       106    0.1%
10000  1480  4000   800     73750        59    0.0%
10000  1500  4000   800     73830     40.84    0.0%
10000  1520  4000   800     73910     125.3    0.1%
10000  1540  4000   800     74010     99.06    0.1%
10000  1560  4000   800     73810     59.91    0.0%
10000  1580  4000   800     73860     74.21    0.1%
10000  1600  4000   800     73950       123    0.1%
10000  1620  4000   800     74050     57.69    0.0%
10000  1640  4000   800     73830     71.12    0.0%
10000  1660  4000   800     74170     156.8    0.2%
10000  1680  4000   800     74100     43.49    0.0%
10000  1700  4000   800     74140     63.43    0.0%
10000  1720  4000   800     74000     76.13    0.1%
10000  1740  4000   800     74160     200.7    0.2%
10000  1760  4000   800     74120     52.75    0.0%
10000  1780  4000   800     74020     107.9    0.1%
10000  1800  4000   800     74100     115.9    0.1%
10000  1820  4000   800     74100     164.5    0.2%
10000  1840  4000   800     74000     56.83    0.0%
10000  1860  4000   800     73970     108.7    0.1%
10000  1880  4000   800     73960     130.5    0.1%
10000  1900  4000   800     74000     66.96    0.0%
10000  1920  4000   800     74080     95.91    0.1%
10000  1940  4000   800     73990     49.03    0.0%
10000  1960  4000   800     73990     103.3    0.1%
10000  1980  4000   800     74260     129.9    0.1%
10000  2000  1000   800     44840     156.7    0.3%
10000  2000  1060   800     45280      87.5    0.1%
10000  2000  1120   800     46330     147.3    0.3%
10000  2000  1180   800     46730     163.8    0.3%
10000  2000  1240   800     47170     101.7    0.2%
10000  2000  1300   800     47760     218.6    0.4%
10000  2000  1360   800     48240       121    0.2%
10000  2000  1420   800     48920       100    0.2%
10000  2000  1480   800     49760     127.1    0.2%
10000  2000  1540   800     50260     153.1    0.3%
10000  2000  1600   800     50790     58.82    0.1%
10000  2000  1660   800     51230     62.89    0.1%
10000  2000  1720   800     52120     191.4    0.3%
10000  2000  1780   800     52750     159.7    0.3%
10000  2000  1840   800     53220     181.2    0.3%
10000  2000  1900   800     53760     85.05    0.1%
10000  2000  1960   800     54450     111.4    0.2%
10000  2000  2020   800     54950     72.86    0.1%
10000  2000  2080   800     55690       188    0.3%
10000  2000  2140   800     56160     132.8    0.2%
10000  2000  2200   800     56820     163.5    0.2%
10000  2000  2260   800     57330     201.5    0.3%
10000  2000  2320   800     57880     148.5    0.2%
10000  2000  2380   800     58410     146.2    0.2%
10000  2000  2440   800     58650     85.88    0.1%
10000  2000  2500   800     59390     100.2    0.1%
10000  2000  2560   800     60610       267    0.4%
10000  2000  2620   800     60810       231    0.3%
10000  2000  2680   800     61190     132.6    0.2%
10000  2000  2740   800     61730     213.2    0.3%
10000  2000  2800   800     62330     211.9    0.3%
10000  2000  2860   800     63100     169.7    0.2%
10000  2000  2920   800     63520     136.9    0.2%
10000  2000  2980   800     64070     139.8    0.2%
10000  2000  3040   800     64660     122.6    0.1%
10000  2000  3100   800     65460     162.2    0.2%
10000  2000  3160   800     66020     163.3    0.2%
10000  2000  3220   800     66620     92.18    0.1%
10000  2000  3280   800     67340     116.6    0.1%
10000  2000  3340   800     67960     75.01    0.1%
10000  2000  3400   800     68460     172.3    0.2%
10000  2000  3460   800     68920     150.9    0.2%
10000  2000  3520   800     69650     185.1    0.2%
10000  2000  3580   800     70150     68.51    0.0%
10000  2000  3640   800     70680      72.3    0.1%
10000  2000  3700   800     71530     137.9    0.1%
10000  2000  3760   800     72000       183    0.2%
10000  2000  3820   800     72750     196.7    0.2%
10000  2000  3880   800     73180     170.8    0.2%
10000  2000  3940   800     73770     175.8    0.2%
10000  2000  4000   400     72610     141.2    0.1%
10000  2000  4000   408     72600     144.1    0.1%
10000  2000  4000   416     72940     123.1    0.1%
10000  2000  4000   424     72990       179    0.2%
10000  2000  4000   432     72900     221.9    0.3%
10000  2000  4000   440     73230     262.4    0.3%
10000  2000  4000   448     73060     186.9    0.2%
10000  2000  4000   456     72770     135.4    0.1%
10000  2000  4000   464     72880     183.7    0.2%
10000  2000  4000   472     72750     175.5    0.2%
10000  2000  4000   480     72820     96.51    0.1%
10000  2000  4000   488     72870     159.6    0.2%
10000  2000  4000   496     72940       204    0.2%
10000  2000  4000   504     72800     240.3    0.3%
10000  2000  4000   512     72900     147.1    0.2%
10000  2000  4000   520     72730     115.6    0.1%
10000  2000  4000   528     72800     113.6    0.1%
10000  2000  4000   536     72720     152.3    0.2%
10000  2000  4000   544     72660       147    0.2%
10000  2000  4000   552     72710     179.9    0.2%
10000  2000  4000   560     72660     141.1    0.1%
10000  2000  4000   568     72580     202.8    0.2%
10000  2000  4000   576     73020     331.9    0.4%
10000  2000  4000   584     72760     201.6    0.2%
10000  2000  4000   592     72800     114.1    0.1%
10000  2000  4000   600     72760     119.7    0.1%
10000  2000  4000   608     72850     240.6    0.3%
10000  2000  4000   616     72890        87    0.1%
10000  2000  4000   624     72750     154.5    0.2%
10000  2000  4000   632     73090       206    0.2%
10000  2000  4000   640     73090     129.4    0.1%
10000  2000  4000   648     73230     177.9    0.2%
10000  2000  4000   656     73100     269.2    0.3%
10000  2000  4000   664     73190     130.6    0.1%
10000  2000  4000   672     73260     197.7    0.2%
10000  2000  4000   680     73270     99.34    0.1%
10000  2000  4000   688     73460     158.2    0.2%
10000  2000  4000   696     73680     185.6    0.2%
10000  2000  4000   704     73420     182.4    0.2%
10000  2000  4000   712     73520     107.9    0.1%
10000  2000  4000   720     73650     125.2    0.1%
10000  2000  4000   728     73490     149.4    0.2%
10000  2000  4000   736     73680       124    0.1%
10000  2000  4000   744     73780     143.8    0.1%
10000  2000  4000   752     74120     219.2    0.2%
10000  2000  4000   760     73820     158.1    0.2%
10000  2000  4000   768     74020     145.3    0.1%
10000  2000  4000   776     74060     178.8    0.2%
10000  2000  4000   784     74010     132.8    0.1%
10000  2000  4000   792     74170     103.4    0.1%
10000  2000  4000   800     74210       169    0.2%

Quality and confidence:
param     error
v         0.004
t         0.023
a         0.007
d         0.058

Model:
Time ~=        0
    + v    3.613
    + t    0.286
    + a    9.677
    + d    4.178
              µs

Reads = 4 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Writes = 0 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Pallet: "pallet_election_provider_multi_phase", Extrinsic: "on_initialize_nothing", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    33.56
              µs

Reads = 8
Writes = 0
Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    33.56
              µs

Reads = 8
Writes = 0
Pallet: "pallet_election_provider_multi_phase", Extrinsic: "on_initialize_open_signed", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    114.5
              µs

Reads = 10
Writes = 4
Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    114.5
              µs

Reads = 10
Writes = 4
Pallet: "pallet_election_provider_multi_phase", Extrinsic: "on_initialize_open_unsigned_with_snapshot", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=      114
              µs

Reads = 10
Writes = 4
Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=      114
              µs

Reads = 10
Writes = 4
Pallet: "pallet_election_provider_multi_phase", Extrinsic: "finalize_signed_phase_accept_solution", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    59.76
              µs

Reads = 1
Writes = 2
Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    59.76
              µs

Reads = 1
Writes = 2
Pallet: "pallet_election_provider_multi_phase", Extrinsic: "finalize_signed_phase_reject_solution", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    39.89
              µs

Reads = 1
Writes = 1
Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    39.89
              µs

Reads = 1
Writes = 1
Pallet: "pallet_election_provider_multi_phase", Extrinsic: "on_initialize_open_unsigned_without_snapshot", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    23.59
              µs

Reads = 1
Writes = 1
Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    23.59
              µs

Reads = 1
Writes = 1
Pallet: "pallet_election_provider_multi_phase", Extrinsic: "elect_queued", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + v    0.016
    + t    0.004
    + a    1.958
    + d    0.506
              µs

Reads = 6 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Writes = 8 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    v     t     a     d   mean µs  sigma µs       %
 5000  2000  4000   800      8271     61.15    0.7%
 5100  2000  4000   800      8235     19.66    0.2%
 5200  2000  4000   800      8245     24.48    0.2%
 5300  2000  4000   800      8238     28.61    0.3%
 5400  2000  4000   800      8182     28.89    0.3%
 5500  2000  4000   800      8243     15.35    0.1%
 5600  2000  4000   800      8272     33.52    0.4%
 5700  2000  4000   800      8255     27.77    0.3%
 5800  2000  4000   800      8256     19.28    0.2%
 5900  2000  4000   800      8248     11.44    0.1%
 6000  2000  4000   800      8282     53.17    0.6%
 6100  2000  4000   800      8225     18.99    0.2%
 6200  2000  4000   800      8281     20.52    0.2%
 6300  2000  4000   800      8266     29.79    0.3%
 6400  2000  4000   800      8284     34.16    0.4%
 6500  2000  4000   800      8323     127.3    1.5%
 6600  2000  4000   800      8199     39.71    0.4%
 6700  2000  4000   800      8254     12.87    0.1%
 6800  2000  4000   800      8242     21.19    0.2%
 6900  2000  4000   800      8271     15.64    0.1%
 7000  2000  4000   800      8280     41.16    0.4%
 7100  2000  4000   800      8271     20.03    0.2%
 7200  2000  4000   800      8220     28.88    0.3%
 7300  2000  4000   800      8240     37.71    0.4%
 7400  2000  4000   800      8334     52.43    0.6%
 7500  2000  4000   800      8270     26.45    0.3%
 7600  2000  4000   800      8216     29.59    0.3%
 7700  2000  4000   800      8216     49.48    0.6%
 7800  2000  4000   800      8237     32.35    0.3%
 7900  2000  4000   800      8277     44.28    0.5%
 8000  2000  4000   800      8281     28.58    0.3%
 8100  2000  4000   800      8288     53.58    0.6%
 8200  2000  4000   800      8269     28.15    0.3%
 8300  2000  4000   800      8253      40.6    0.4%
 8400  2000  4000   800      8334     37.81    0.4%
 8500  2000  4000   800      8286     41.02    0.4%
 8600  2000  4000   800      8299     35.49    0.4%
 8700  2000  4000   800      8290     28.65    0.3%
 8800  2000  4000   800      8346     73.12    0.8%
 8900  2000  4000   800      8304     31.19    0.3%
 9000  2000  4000   800      8278     34.41    0.4%
 9100  2000  4000   800      8293     28.33    0.3%
 9200  2000  4000   800      8264     34.69    0.4%
 9300  2000  4000   800      8296     77.62    0.9%
 9400  2000  4000   800      8270     45.07    0.5%
 9500  2000  4000   800      8307      26.6    0.3%
 9600  2000  4000   800      8275     50.84    0.6%
 9700  2000  4000   800      8290     40.98    0.4%
 9800  2000  4000   800      8288     21.52    0.2%
 9900  2000  4000   800      8345     34.21    0.4%
10000  1000  4000   800      8263     51.92    0.6%
10000  1020  4000   800      8317      20.4    0.2%
10000  1040  4000   800      8374      36.9    0.4%
10000  1060  4000   800      8308      48.2    0.5%
10000  1080  4000   800      8343     38.79    0.4%
10000  1100  4000   800      8454     156.7    1.8%
10000  1120  4000   800      8354     26.02    0.3%
10000  1140  4000   800      8314     27.83    0.3%
10000  1160  4000   800      8331     52.91    0.6%
10000  1180  4000   800      8305     31.55    0.3%
10000  1200  4000   800      8291     49.28    0.5%
10000  1220  4000   800      8303      46.8    0.5%
10000  1240  4000   800      8315     29.74    0.3%
10000  1260  4000   800      8330     26.61    0.3%
10000  1280  4000   800      8305     14.62    0.1%
10000  1300  4000   800      8306     26.95    0.3%
10000  1320  4000   800      8315     70.96    0.8%
10000  1340  4000   800      8312     34.09    0.4%
10000  1360  4000   800      8326     41.43    0.4%
10000  1380  4000   800      8331     45.44    0.5%
10000  1400  4000   800      8315     38.31    0.4%
10000  1420  4000   800      8310     19.24    0.2%
10000  1440  4000   800      8281     24.33    0.2%
10000  1460  4000   800      8324     48.48    0.5%
10000  1480  4000   800      8336     42.14    0.5%
10000  1500  4000   800      8298      42.1    0.5%
10000  1520  4000   800      8336     30.51    0.3%
10000  1540  4000   800      8352     38.63    0.4%
10000  1560  4000   800      8320     30.45    0.3%
10000  1580  4000   800      8354     69.57    0.8%
10000  1600  4000   800      8342     37.74    0.4%
10000  1620  4000   800      8348     64.02    0.7%
10000  1640  4000   800      8367     26.68    0.3%
10000  1660  4000   800      8330     25.14    0.3%
10000  1680  4000   800      8295     51.48    0.6%
10000  1700  4000   800      8260        21    0.2%
10000  1720  4000   800      8269     14.06    0.1%
10000  1740  4000   800      8278     30.51    0.3%
10000  1760  4000   800      8307     46.97    0.5%
10000  1780  4000   800      8264     51.32    0.6%
10000  1800  4000   800      8289     42.37    0.5%
10000  1820  4000   800      8344     44.32    0.5%
10000  1840  4000   800      8288     59.06    0.7%
10000  1860  4000   800      8350      48.8    0.5%
10000  1880  4000   800      8333      69.6    0.8%
10000  1900  4000   800      8300     81.78    0.9%
10000  1920  4000   800      8292     49.13    0.5%
10000  1940  4000   800      8375     30.52    0.3%
10000  1960  4000   800      8267     22.77    0.2%
10000  1980  4000   800      8412     44.84    0.5%
10000  2000  1000   800      2477     15.56    0.6%
10000  2000  1060   800      2601     11.06    0.4%
10000  2000  1120   800      2720     15.51    0.5%
10000  2000  1180   800      2804     10.49    0.3%
10000  2000  1240   800      2924     10.59    0.3%
10000  2000  1300   800      3040     9.175    0.3%
10000  2000  1360   800      3155     12.28    0.3%
10000  2000  1420   800      3264     22.19    0.6%
10000  2000  1480   800      3367     17.29    0.5%
10000  2000  1540   800      3464      10.3    0.2%
10000  2000  1600   800      3588     17.45    0.4%
10000  2000  1660   800      3702     16.45    0.4%
10000  2000  1720   800      3817     26.63    0.6%
10000  2000  1780   800      3941     11.57    0.2%
10000  2000  1840   800      4060     15.49    0.3%
10000  2000  1900   800      4199     18.48    0.4%
10000  2000  1960   800      4349     13.97    0.3%
10000  2000  2020   800      4452     28.57    0.6%
10000  2000  2080   800      4582     36.12    0.7%
10000  2000  2140   800      4716     75.15    1.5%
10000  2000  2200   800      4825     39.52    0.8%
10000  2000  2260   800      4901     35.38    0.7%
10000  2000  2320   800      5081      30.9    0.6%
10000  2000  2380   800      5101     35.25    0.6%
10000  2000  2440   800      5251     33.23    0.6%
10000  2000  2500   800      5420     22.33    0.4%
10000  2000  2560   800      5571     39.87    0.7%
10000  2000  2620   800      5611      28.8    0.5%
10000  2000  2680   800      5761     35.91    0.6%
10000  2000  2740   800      5910     88.26    1.4%
10000  2000  2800   800      6009     48.65    0.8%
10000  2000  2860   800      6091     86.24    1.4%
10000  2000  2920   800      6136     24.25    0.3%
10000  2000  2980   800      6304     44.14    0.7%
10000  2000  3040   800      6388      59.5    0.9%
10000  2000  3100   800      6524     25.48    0.3%
10000  2000  3160   800      6635     33.64    0.5%
10000  2000  3220   800      6740     41.74    0.6%
10000  2000  3280   800      6864     21.09    0.3%
10000  2000  3340   800      7025     63.77    0.9%
10000  2000  3400   800      7141     24.56    0.3%
10000  2000  3460   800      7257     48.54    0.6%
10000  2000  3520   800      7307     40.19    0.5%
10000  2000  3580   800      7561     103.5    1.3%
10000  2000  3640   800      7574      25.8    0.3%
10000  2000  3700   800      7801     23.35    0.2%
10000  2000  3760   800      7853     48.12    0.6%
10000  2000  3820   800      8001     62.91    0.7%
10000  2000  3880   800      8081     21.03    0.2%
10000  2000  3940   800      8235      28.2    0.3%
10000  2000  4000   400      8125     24.29    0.2%
10000  2000  4000   408      8196     18.14    0.2%
10000  2000  4000   416      8149     19.17    0.2%
10000  2000  4000   424      8172     51.52    0.6%
10000  2000  4000   432      8125     22.41    0.2%
10000  2000  4000   440      8164     44.43    0.5%
10000  2000  4000   448      8188     29.03    0.3%
10000  2000  4000   456      8186     39.47    0.4%
10000  2000  4000   464      8139     21.76    0.2%
10000  2000  4000   472      8206     14.11    0.1%
10000  2000  4000   480      8145     11.03    0.1%
10000  2000  4000   488      8274     127.1    1.5%
10000  2000  4000   496      8137     18.41    0.2%
10000  2000  4000   504      8189     18.71    0.2%
10000  2000  4000   512      8167     21.85    0.2%
10000  2000  4000   520      8220     33.07    0.4%
10000  2000  4000   528      8193     41.13    0.5%
10000  2000  4000   536      8258     24.03    0.2%
10000  2000  4000   544      8270     142.5    1.7%
10000  2000  4000   552      8236     66.11    0.8%
10000  2000  4000   560      8252     81.14    0.9%
10000  2000  4000   568      8256     25.31    0.3%
10000  2000  4000   576      8267     50.63    0.6%
10000  2000  4000   584      8272     22.83    0.2%
10000  2000  4000   592      8235     39.17    0.4%
10000  2000  4000   600      8266     38.84    0.4%
10000  2000  4000   608      8268     38.94    0.4%
10000  2000  4000   616      8316     33.22    0.3%
10000  2000  4000   624      8204      18.3    0.2%
10000  2000  4000   632      8293     18.36    0.2%
10000  2000  4000   640      8285     36.25    0.4%
10000  2000  4000   648      8257     28.43    0.3%
10000  2000  4000   656      8295     73.82    0.8%
10000  2000  4000   664      8342     27.31    0.3%
10000  2000  4000   672      8261     29.67    0.3%
10000  2000  4000   680      8264     62.44    0.7%
10000  2000  4000   688      8287     36.43    0.4%
10000  2000  4000   696      8274     53.17    0.6%
10000  2000  4000   704      8294     23.88    0.2%
10000  2000  4000   712      8339     20.34    0.2%
10000  2000  4000   720      8286     39.01    0.4%
10000  2000  4000   728      8335     22.05    0.2%
10000  2000  4000   736      8302     28.34    0.3%
10000  2000  4000   744      8321     43.26    0.5%
10000  2000  4000   752      8287     19.64    0.2%
10000  2000  4000   760      8291        55    0.6%
10000  2000  4000   768      8322     44.34    0.5%
10000  2000  4000   776      8353     81.87    0.9%
10000  2000  4000   784      8378      84.6    1.0%
10000  2000  4000   792      8309     46.99    0.5%
10000  2000  4000   800      8356     27.97    0.3%

Quality and confidence:
param     error
v         0.001
t         0.005
a         0.001
d         0.014

Model:
Time ~=        0
    + v    0.019
    + t        0
    + a    1.959
    + d    0.392
              µs

Reads = 6 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Writes = 8 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Pallet: "pallet_election_provider_multi_phase", Extrinsic: "submit", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    77.64
    + c    0.212
              µs

Reads = 4 + (0 * c)
Writes = 3 + (0 * c)
Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    c   mean µs  sigma µs       %
    1     78.23     0.085    0.1%
    2     77.78     0.118    0.1%
    3     78.45     0.121    0.1%
    4     77.64     0.125    0.1%
    5     79.03     0.193    0.2%
    6     78.63      0.07    0.0%
    7     79.67      0.15    0.1%
    8     78.99     0.124    0.1%
    9     79.66     0.088    0.1%

Quality and confidence:
param     error
c         0.018

Model:
Time ~=    77.61
    + c    0.213
              µs

Reads = 4 + (0 * c)
Writes = 3 + (0 * c)
Pallet: "pallet_election_provider_multi_phase", Extrinsic: "submit_unsigned", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + v    3.608
    + t    0.187
    + a    11.42
    + d    4.542
              µs

Reads = 7 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Writes = 1 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    v     t     a     d   mean µs  sigma µs       %
 5000  2000  4000   800     62860     59.93    0.0%
 5100  2000  4000   800     63300     83.79    0.1%
 5200  2000  4000   800     63580     76.31    0.1%
 5300  2000  4000   800     63940     99.29    0.1%
 5400  2000  4000   800     64240     115.6    0.1%
 5500  2000  4000   800     64590     46.42    0.0%
 5600  2000  4000   800     65040     110.5    0.1%
 5700  2000  4000   800     65430     96.15    0.1%
 5800  2000  4000   800     65780     51.33    0.0%
 5900  2000  4000   800     66110     53.46    0.0%
 6000  2000  4000   800     66520     74.73    0.1%
 6100  2000  4000   800     66940     109.5    0.1%
 6200  2000  4000   800     67220     66.19    0.0%
 6300  2000  4000   800     67720     98.75    0.1%
 6400  2000  4000   800     68040     47.33    0.0%
 6500  2000  4000   800     68360     43.61    0.0%
 6600  2000  4000   800     68550     69.37    0.1%
 6700  2000  4000   800     68950     75.58    0.1%
 6800  2000  4000   800     69300     105.4    0.1%
 6900  2000  4000   800     69650     53.66    0.0%
 7000  2000  4000   800     69970      42.6    0.0%
 7100  2000  4000   800     70290     68.24    0.0%
 7200  2000  4000   800     70710     19.86    0.0%
 7300  2000  4000   800     71180     44.15    0.0%
 7400  2000  4000   800     71500     84.05    0.1%
 7500  2000  4000   800     71870     169.9    0.2%
 7600  2000  4000   800     72160     83.59    0.1%
 7700  2000  4000   800     72450     73.83    0.1%
 7800  2000  4000   800     73050     249.8    0.3%
 7900  2000  4000   800     73180     88.73    0.1%
 8000  2000  4000   800     73650     55.79    0.0%
 8100  2000  4000   800     74030     73.62    0.0%
 8200  2000  4000   800     74210     65.49    0.0%
 8300  2000  4000   800     74620     61.53    0.0%
 8400  2000  4000   800     75360     111.8    0.1%
 8500  2000  4000   800     75500     119.2    0.1%
 8600  2000  4000   800     75820     19.23    0.0%
 8700  2000  4000   800     76160     72.92    0.0%
 8800  2000  4000   800     76550     70.11    0.0%
 8900  2000  4000   800     76840      27.1    0.0%
 9000  2000  4000   800     77170       116    0.1%
 9100  2000  4000   800     77570     80.45    0.1%
 9200  2000  4000   800     78110     136.9    0.1%
 9300  2000  4000   800     78450     222.7    0.2%
 9400  2000  4000   800     78640     68.16    0.0%
 9500  2000  4000   800     78960      68.5    0.0%
 9600  2000  4000   800     79970     413.8    0.5%
 9700  2000  4000   800     80250     171.5    0.2%
 9800  2000  4000   800     80040     61.76    0.0%
 9900  2000  4000   800     80510     83.91    0.1%
10000  1000  4000   800     81130     333.5    0.4%
10000  1020  4000   800     81190     460.5    0.5%
10000  1040  4000   800     81020     220.8    0.2%
10000  1060  4000   800     81360     415.5    0.5%
10000  1080  4000   800     81040       246    0.3%
10000  1100  4000   800     80890     353.5    0.4%
10000  1120  4000   800     80870     144.9    0.1%
10000  1140  4000   800     80870     209.4    0.2%
10000  1160  4000   800     80900     280.9    0.3%
10000  1180  4000   800     81600     171.6    0.2%
10000  1200  4000   800     81130     329.1    0.4%
10000  1220  4000   800     81240     328.8    0.4%
10000  1240  4000   800     81010     351.5    0.4%
10000  1260  4000   800     81010     428.8    0.5%
10000  1280  4000   800     80940     246.6    0.3%
10000  1300  4000   800     81240     500.8    0.6%
10000  1320  4000   800     81280     234.6    0.2%
10000  1340  4000   800     81260     372.6    0.4%
10000  1360  4000   800     81510     363.5    0.4%
10000  1380  4000   800     81270     259.6    0.3%
10000  1400  4000   800     81350     362.6    0.4%
10000  1420  4000   800     81470     490.5    0.6%
10000  1440  4000   800     80950     215.7    0.2%
10000  1460  4000   800     81010     208.6    0.2%
10000  1480  4000   800     81570     260.6    0.3%
10000  1500  4000   800     81080     380.3    0.4%
10000  1520  4000   800     81490     248.2    0.3%
10000  1540  4000   800     81410       401    0.4%
10000  1560  4000   800     81030     337.8    0.4%
10000  1580  4000   800     81120     355.6    0.4%
10000  1600  4000   800     80820     55.74    0.0%
10000  1620  4000   800     81020     304.4    0.3%
10000  1640  4000   800     80960     196.2    0.2%
10000  1660  4000   800     81270     464.1    0.5%
10000  1680  4000   800     81320     243.2    0.2%
10000  1700  4000   800     81250       401    0.4%
10000  1720  4000   800     81210     369.1    0.4%
10000  1740  4000   800     81170       290    0.3%
10000  1760  4000   800     81270     321.5    0.3%
10000  1780  4000   800     80900     228.5    0.2%
10000  1800  4000   800     81570     344.6    0.4%
10000  1820  4000   800     80970     219.7    0.2%
10000  1840  4000   800     81320     447.6    0.5%
10000  1860  4000   800     81680       294    0.3%
10000  1880  4000   800     81330     404.3    0.4%
10000  1900  4000   800     81190       411    0.5%
10000  1920  4000   800     81270     335.2    0.4%
10000  1940  4000   800     81240     413.1    0.5%
10000  1960  4000   800     81540     324.9    0.3%
10000  1980  4000   800     81200     200.2    0.2%
10000  2000  1000   800     47590     242.7    0.5%
10000  2000  1060   800     47560     217.2    0.4%
10000  2000  1120   800     48390     255.2    0.5%
10000  2000  1180   800     49540     383.2    0.7%
10000  2000  1240   800     49800     284.2    0.5%
10000  2000  1300   800     50290     297.6    0.5%
10000  2000  1360   800     51270     421.7    0.8%
10000  2000  1420   800     52470     403.3    0.7%
10000  2000  1480   800     53120     300.1    0.5%
10000  2000  1540   800     53350     245.1    0.4%
10000  2000  1600   800     54200     337.1    0.6%
10000  2000  1660   800     54740     154.2    0.2%
10000  2000  1720   800     55550     408.3    0.7%
10000  2000  1780   800     56310     289.3    0.5%
10000  2000  1840   800     57260     451.5    0.7%
10000  2000  1900   800     57630     302.3    0.5%
10000  2000  1960   800     58200     276.8    0.4%
10000  2000  2020   800     59480     377.8    0.6%
10000  2000  2080   800     59520     288.7    0.4%
10000  2000  2140   800     60180     351.5    0.5%
10000  2000  2200   800     60920     388.3    0.6%
10000  2000  2260   800     61570     407.9    0.6%
10000  2000  2320   800     61970     336.4    0.5%
10000  2000  2380   800     62950     434.6    0.6%
10000  2000  2440   800     63360     389.1    0.6%
10000  2000  2500   800     64150     397.7    0.6%
10000  2000  2560   800     64650     412.2    0.6%
10000  2000  2620   800     65150     177.3    0.2%
10000  2000  2680   800     65910     421.3    0.6%
10000  2000  2740   800     66480     393.7    0.5%
10000  2000  2800   800     67370     387.2    0.5%
10000  2000  2860   800     69180     215.8    0.3%
10000  2000  2920   800     69730     413.8    0.5%
10000  2000  2980   800     69950     349.8    0.5%
10000  2000  3040   800     70990     374.8    0.5%
10000  2000  3100   800     71510     349.6    0.4%
10000  2000  3160   800     71980     402.7    0.5%
10000  2000  3220   800     72490     192.4    0.2%
10000  2000  3280   800     73350     300.7    0.4%
10000  2000  3340   800     74250     270.3    0.3%
10000  2000  3400   800     74850     367.2    0.4%
10000  2000  3460   800     75690     421.7    0.5%
10000  2000  3520   800     76080     365.4    0.4%
10000  2000  3580   800     77520     123.9    0.1%
10000  2000  3640   800     77490       321    0.4%
10000  2000  3700   800     78480     436.7    0.5%
10000  2000  3760   800     78530     185.3    0.2%
10000  2000  3820   800     79570     410.4    0.5%
10000  2000  3880   800     79890     282.5    0.3%
10000  2000  3940   800     80620     218.8    0.2%
10000  2000  4000   400     79150       261    0.3%
10000  2000  4000   408     79360     347.2    0.4%
10000  2000  4000   416     79750     529.4    0.6%
10000  2000  4000   424     79780     285.6    0.3%
10000  2000  4000   432     80020     347.1    0.4%
10000  2000  4000   440     79830     423.5    0.5%
10000  2000  4000   448     80080       168    0.2%
10000  2000  4000   456     80040       416    0.5%
10000  2000  4000   464     80160     260.8    0.3%
10000  2000  4000   472     79590     269.9    0.3%
10000  2000  4000   480     79900     375.2    0.4%
10000  2000  4000   488     79620     269.2    0.3%
10000  2000  4000   496     79680     301.7    0.3%
10000  2000  4000   504     80070     471.4    0.5%
10000  2000  4000   512     79790     441.5    0.5%
10000  2000  4000   520     79400     274.2    0.3%
10000  2000  4000   528     79890     311.8    0.3%
10000  2000  4000   536     79740     250.8    0.3%
10000  2000  4000   544     79110     291.7    0.3%
10000  2000  4000   552     79880     360.7    0.4%
10000  2000  4000   560     79490     273.1    0.3%
10000  2000  4000   568     79800     393.7    0.4%
10000  2000  4000   576     79350     280.2    0.3%
10000  2000  4000   584     79990     359.6    0.4%
10000  2000  4000   592     79350     115.7    0.1%
10000  2000  4000   600     79530     249.9    0.3%
10000  2000  4000   608     79580     337.7    0.4%
10000  2000  4000   616     79930     356.9    0.4%
10000  2000  4000   624     80150     393.9    0.4%
10000  2000  4000   632     79710     365.6    0.4%
10000  2000  4000   640     80100     389.9    0.4%
10000  2000  4000   648     80310     337.3    0.4%
10000  2000  4000   656     80410     430.6    0.5%
10000  2000  4000   664     80060     236.7    0.2%
10000  2000  4000   672     80420     370.1    0.4%
10000  2000  4000   680     80400     360.9    0.4%
10000  2000  4000   688     80370     274.2    0.3%
10000  2000  4000   696     80920     303.2    0.3%
10000  2000  4000   704     80390       226    0.2%
10000  2000  4000   712     80930     500.8    0.6%
10000  2000  4000   720     80400     162.8    0.2%
10000  2000  4000   728     80730     344.8    0.4%
10000  2000  4000   736     80700     299.6    0.3%
10000  2000  4000   744     81310     193.2    0.2%
10000  2000  4000   752     81280     250.6    0.3%
10000  2000  4000   760     80870     291.9    0.3%
10000  2000  4000   768     81270     394.7    0.4%
10000  2000  4000   776     81510     460.5    0.5%
10000  2000  4000   784     81270     386.2    0.4%
10000  2000  4000   792     81290     461.8    0.5%
10000  2000  4000   800     81360     410.7    0.5%

Quality and confidence:
param     error
v         0.008
t         0.042
a         0.014
d         0.107

Model:
Time ~=        0
    + v    3.701
    + t    0.075
    + a    11.26
    + d    5.019
              µs

Reads = 7 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Writes = 1 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Pallet: "pallet_election_provider_multi_phase", Extrinsic: "feasibility_check", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + v    3.569
    + t     0.19
    + a    9.759
    + d    3.976
              µs

Reads = 4 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Writes = 0 + (0 * v) + (0 * t) + (0 * a) + (0 * d)
Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    v     t     a     d   mean µs  sigma µs       %
 5000  2000  4000   800     56200      84.9    0.1%
 5100  2000  4000   800     56490     44.79    0.0%
 5200  2000  4000   800     57080     73.85    0.1%
 5300  2000  4000   800     57390     46.65    0.0%
 5400  2000  4000   800     57650     53.88    0.0%
 5500  2000  4000   800     58100       192    0.3%
 5600  2000  4000   800     58440     114.8    0.1%
 5700  2000  4000   800     58740     44.83    0.0%
 5800  2000  4000   800     59030     59.82    0.1%
 5900  2000  4000   800     59390     87.09    0.1%
 6000  2000  4000   800     59890     58.76    0.0%
 6100  2000  4000   800     60320     68.68    0.1%
 6200  2000  4000   800     60640     100.1    0.1%
 6300  2000  4000   800     60940     45.32    0.0%
 6400  2000  4000   800     61260     89.63    0.1%
 6500  2000  4000   800     61700     88.57    0.1%
 6600  2000  4000   800     62070     146.9    0.2%
 6700  2000  4000   800     62230     47.69    0.0%
 6800  2000  4000   800     62850     148.9    0.2%
 6900  2000  4000   800     63150     58.93    0.0%
 7000  2000  4000   800     63400     63.87    0.1%
 7100  2000  4000   800     63740     59.48    0.0%
 7200  2000  4000   800     64100     55.62    0.0%
 7300  2000  4000   800     64490     80.92    0.1%
 7400  2000  4000   800     64920     92.39    0.1%
 7500  2000  4000   800     65010     43.85    0.0%
 7600  2000  4000   800     65690     91.02    0.1%
 7700  2000  4000   800     65980      71.4    0.1%
 7800  2000  4000   800     66270     50.19<truncated>...

… kiz-mem-aware-benchmarks

…/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/election-provider-multi-phase/src/weights.rs --template=./.maintain/frame-weight-template.hbs

kianenigma · 2021-07-08T11:23:58Z

frame/election-provider-multi-phase/src/weights.rs

+			.saturating_add((1_957_000 as Weight).saturating_mul(a as Weight))
+			// Standard Error: 18_000
+			.saturating_add((588_000 as Weight).saturating_mul(d as Weight))
+			.saturating_add(T::DbWeight::get().reads(6 as Weight))


extra read is expected, we need to read snapshot_metadata now as well.

shawntabrizi · 2021-07-08T18:18:23Z

bin/node/runtime/src/lib.rs

+/// The numbers configured here should always be more than the the maximum limits of staking pallet
+/// to ensure election snapshot will not run out of memory.


I dont quite see how we can ensure this is always the case given the variability on chain.

I get if this is a temp solution, but i think we need to be more careful than to just add this code and forget about it later.

Actually, this comment no longer holds exactly. Given #9285, I had to #[ignore] these benchmarks again, so they are not always executed (which was my intention). My intention was that we always run them as we grow these parameters and it would automatically prevent us from making them too big.

Currently I am benchmarking only create_snapshot. If we do the same with on_initialize (which is only a smidgen different), and if it works here, I think then we can be pretty sure that the same success will hold on-chain as well.

how we can ensure this is always the case given the variability on chain.

Which part of it doesn't add up, and what vulnerability you mean?

The word was variability, that being that this 25_000 value can be changed on-chain, and nothing is really updating the value here in that case.

If we have some hardcoded value here, maybe we need to pass that hard-coded value through to the pallet and make sure the on-chain value is not set higher than this hard-coded value.

since staking's 25_000 is a storage item and it can be changed on the fly, this is not possible. This will be some manual process that we run to ensure some new value that we might want to set in staking's storage is safe.

My thought is that we can also place a limit to the value set in storage, so it does not reach past any limit we have not prepared benchmarks for

shawntabrizi · 2021-07-08T18:31:42Z

frame/election-provider-multi-phase/src/benchmarking.rs

+	// ONLY run this benchmark in isolation, and pass the `--extra` flag to enable it.
+	#[extra]
+	create_snapshot_memory {


do we have some process that will execute this when things change? or just internal knowledge

Not yet, internal knowledge.

shawntabrizi

the changes look okay, but i worry that all of this will be forgotten until it bites us again

kianenigma · 2021-07-08T18:41:20Z

the changes look okay, but i worry that all of this will be forgotten until it bites us again

given that the onchain limits are storage item and this is not, relying on this benchmark to set the boundaries of heap-safety will have to be manual.

Before setting any value to polkadot, we should re-run these benchmarks with the new numbers to get insight.

I think this, next to westend being populated more than polkadot, is as good as it gets for now.

kianenigma · 2021-07-08T19:01:31Z

bot merge

ghost · 2021-07-08T19:01:34Z

Waiting for commit status.

ghost · 2021-07-08T19:02:23Z

Merge aborted: Checks failed for 06982f6

kianenigma · 2021-07-09T07:22:12Z

bot merge

ghost · 2021-07-09T07:22:15Z

Waiting for commit status.

ghost · 2021-07-09T07:24:37Z

Merge aborted: Checks failed for 1d2d10e

kianenigma · 2021-07-09T08:03:49Z

bot merge

ghost · 2021-07-09T08:03:53Z

Waiting for commit status.

ghost · 2021-07-09T08:20:21Z

Merge failed: "Required status check "continuous-integration/gitlab-check-polkadot-companion-build" is failing."

ordian · 2021-07-09T19:55:26Z

bot merge

ghost · 2021-07-09T19:55:30Z

Trying merge.

kianenigma added 4 commits June 25, 2021 09:39

Make benchmarks a bit better with mem

2130f1a

Make election benchmarks more *memory-aware*

26be428

Master.into()

1c8c7e2

Fix a few errors

53f0378

kianenigma requested review from shawntabrizi and bkchr July 6, 2021 09:52

github-actions bot added the A0-please_review Pull request needs code review. label Jul 6, 2021

kianenigma added B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D2-notlive 💤 PR contains changes in a runtime directory that is not deployed to a chain that requires an audit. labels Jul 6, 2021

github-actions bot added the A7-needspolkadotpr label Jul 6, 2021

coriolinus approved these changes Jul 6, 2021

View reviewed changes

kianenigma changed the title ~~Make benchmarks a bit better with mem~~ Make election benchmarks more *memory-aware* Jul 6, 2021

Parity Bot added 2 commits July 7, 2021 04:43

Merge branch 'master' of https://github.com/paritytech/substrate into…

54b6ab1

… kiz-mem-aware-benchmarks

joao-paulo-parity mentioned this pull request Jul 7, 2021

Improve logging and error handling paritytech/bench-bot#46

Merged

kianenigma added 2 commits July 8, 2021 13:16

Manually fix the weights

dfd34f7

Master.into()

1f9d911

kianenigma commented Jul 8, 2021

View reviewed changes

Update lock file

bce640b

kianenigma mentioned this pull request Jul 8, 2021

Store election snapshot in a more memory-friendly way. #9275

Merged

shawntabrizi mentioned this pull request Jul 8, 2021

Companion for substrate/pull/9286 paritytech/polkadot#3443

Merged

remove dupe

4e9061b

kianenigma requested review from gui1117 and emostov July 8, 2021 18:14

shawntabrizi reviewed Jul 8, 2021

View reviewed changes

shawntabrizi approved these changes Jul 8, 2021

View reviewed changes

Fix tests

06982f6

github-actions bot removed the A7-needspolkadotpr label Jul 8, 2021

Master.into()

1d2d10e

cargo update pwasm

1cc4a0b

github-actions bot added the A7-needspolkadotpr label Jul 9, 2021

ghost merged commit deac632 into master Jul 9, 2021

ghost deleted the kiz-mem-aware-benchmarks branch July 9, 2021 19:55

MRamanenkau mentioned this pull request Oct 17, 2022

Feature/update to polkadot v0 9 9 Cerebellum-Network/pos-network-node#159

Merged

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make election benchmarks more memory-aware #9286

Make election benchmarks more memory-aware #9286

kianenigma commented Jul 6, 2021 •

edited

Loading

kianenigma commented Jul 6, 2021

parity-benchapp bot commented Jul 6, 2021

shawntabrizi commented Jul 7, 2021

parity-benchapp bot commented Jul 7, 2021

shawntabrizi commented Jul 7, 2021

parity-benchapp bot commented Jul 7, 2021

joao-paulo-parity commented Jul 7, 2021

parity-benchapp bot commented Jul 7, 2021 •

edited

Loading

kianenigma Jul 8, 2021

shawntabrizi Jul 8, 2021

kianenigma Jul 8, 2021

shawntabrizi Jul 8, 2021

kianenigma Jul 8, 2021

shawntabrizi Jul 8, 2021

shawntabrizi Jul 8, 2021

kianenigma Jul 8, 2021

shawntabrizi left a comment

kianenigma commented Jul 8, 2021

kianenigma commented Jul 8, 2021

ghost commented Jul 8, 2021

ghost commented Jul 8, 2021

kianenigma commented Jul 9, 2021

ghost commented Jul 9, 2021

ghost commented Jul 9, 2021

kianenigma commented Jul 9, 2021

ghost commented Jul 9, 2021

ghost commented Jul 9, 2021

ordian commented Jul 9, 2021

ghost commented Jul 9, 2021

		/// The numbers configured here should always be more than the the maximum limits of staking pallet
		/// to ensure election snapshot will not run out of memory.

Make election benchmarks more *memory-aware* #9286

Make election benchmarks more *memory-aware* #9286

Conversation

kianenigma commented Jul 6, 2021 • edited Loading

kianenigma commented Jul 6, 2021

parity-benchapp bot commented Jul 6, 2021

shawntabrizi commented Jul 7, 2021

parity-benchapp bot commented Jul 7, 2021

shawntabrizi commented Jul 7, 2021

parity-benchapp bot commented Jul 7, 2021

joao-paulo-parity commented Jul 7, 2021

parity-benchapp bot commented Jul 7, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shawntabrizi left a comment

Choose a reason for hiding this comment

kianenigma commented Jul 8, 2021

kianenigma commented Jul 8, 2021

ghost commented Jul 8, 2021

ghost commented Jul 8, 2021

kianenigma commented Jul 9, 2021

ghost commented Jul 9, 2021

ghost commented Jul 9, 2021

kianenigma commented Jul 9, 2021

ghost commented Jul 9, 2021

ghost commented Jul 9, 2021

ordian commented Jul 9, 2021

ghost commented Jul 9, 2021

Make election benchmarks more memory-aware #9286

Make election benchmarks more memory-aware #9286

kianenigma commented Jul 6, 2021 •

edited

Loading

parity-benchapp bot commented Jul 7, 2021 •

edited

Loading