Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: enable debug apis by default #6824

Merged
merged 1 commit into from
Jun 7, 2024
Merged

feat: enable debug apis by default #6824

merged 1 commit into from
Jun 7, 2024

Conversation

nflaig
Copy link
Member

@nflaig nflaig commented May 28, 2024

Motivation

Description

Enable debug apis by default, see referenced issue for rationale.

Closes #6822

@nflaig nflaig requested a review from a team as a code owner May 28, 2024 15:37
@codecov Codecov
Copy link

codecov bot commented May 28, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 62.19%. Comparing base (95ce044) to head (601fa51).
Report is 6 commits behind head on unstable.

Additional details and impacted files 
@@             Coverage Diff              @@
##           unstable    #6824      +/-   ##
============================================
- Coverage     62.21%   62.19%   -0.02%     
============================================
  Files           571      572       +1     
  Lines         60017    60043      +26     
  Branches       1975     1976       +1     
============================================
+ Hits          37338    37346       +8     
- Misses        22636    22654      +18     
  Partials         43       43

@github-actions GitHub Actions
Copy link
Contributor

⚠️ Performance Alert ⚠️

Possible performance regression was detected for some benchmarks.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold.

Benchmark suite Current: e902a1e Previous: 1831d47 Ratio
Map get x1000 6.9940 ns/op 0.82900 ns/op 8.44
Full benchmark results
Benchmark suite Current: e902a1e Previous: 1831d47 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 714.54 us/op 1.0257 ms/op 0.70
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 68.536 us/op 91.813 us/op 0.75
BLS verify - blst-native 1.2032 ms/op 1.1944 ms/op 1.01
BLS verifyMultipleSignatures 3 - blst-native 2.5358 ms/op 2.5098 ms/op 1.01
BLS verifyMultipleSignatures 8 - blst-native 5.4282 ms/op 5.5071 ms/op 0.99
BLS verifyMultipleSignatures 32 - blst-native 19.869 ms/op 20.004 ms/op 0.99
BLS verifyMultipleSignatures 64 - blst-native 38.902 ms/op 39.836 ms/op 0.98
BLS verifyMultipleSignatures 128 - blst-native 82.028 ms/op 77.788 ms/op 1.05
BLS deserializing 10000 signatures 900.57 ms/op 890.53 ms/op 1.01
BLS deserializing 100000 signatures 8.9343 s/op 8.9152 s/op 1.00
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.3172 ms/op 1.2771 ms/op 1.03
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.4668 ms/op 1.4502 ms/op 1.01
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.3490 ms/op 2.2828 ms/op 1.03
BLS verifyMultipleSignatures - same message - 64 - blst-native 3.7479 ms/op 3.3959 ms/op 1.10
BLS verifyMultipleSignatures - same message - 128 - blst-native 6.3260 ms/op 5.6055 ms/op 1.13
BLS aggregatePubkeys 32 - blst-native 33.479 us/op 28.328 us/op 1.18
BLS aggregatePubkeys 128 - blst-native 110.50 us/op 110.16 us/op 1.00
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 73.673 ms/op 63.617 ms/op 1.16
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 97.374 ms/op 51.832 ms/op 1.88
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 47.591 ms/op 31.658 ms/op 1.50
getSlashingsAndExits - default max 116.07 us/op 206.36 us/op 0.56
getSlashingsAndExits - 2k 307.80 us/op 526.77 us/op 0.58
proposeBlockBody type=full, size=empty 5.6247 ms/op 5.6800 ms/op 0.99
isKnown best case - 1 super set check 311.00 ns/op 469.00 ns/op 0.66
isKnown normal case - 2 super set checks 296.00 ns/op 401.00 ns/op 0.74
isKnown worse case - 16 super set checks 293.00 ns/op 403.00 ns/op 0.73
InMemoryCheckpointStateCache - add get delete 4.6180 us/op 6.4700 us/op 0.71
validate api signedAggregateAndProof - struct 2.5465 ms/op 2.5282 ms/op 1.01
validate gossip signedAggregateAndProof - struct 2.5432 ms/op 2.5318 ms/op 1.00
validate gossip attestation - vc 640000 1.2007 ms/op 1.2302 ms/op 0.98
batch validate gossip attestation - vc 640000 - chunk 32 157.72 us/op 169.82 us/op 0.93
batch validate gossip attestation - vc 640000 - chunk 64 140.73 us/op 154.43 us/op 0.91
batch validate gossip attestation - vc 640000 - chunk 128 135.26 us/op 147.94 us/op 0.91
batch validate gossip attestation - vc 640000 - chunk 256 135.41 us/op 138.31 us/op 0.98
pickEth1Vote - no votes 1.0968 ms/op 1.3061 ms/op 0.84
pickEth1Vote - max votes 7.7398 ms/op 7.5759 ms/op 1.02
pickEth1Vote - Eth1Data hashTreeRoot value x2048 13.249 ms/op 17.163 ms/op 0.77
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 17.104 ms/op 18.817 ms/op 0.91
pickEth1Vote - Eth1Data fastSerialize value x2048 523.59 us/op 629.05 us/op 0.83
pickEth1Vote - Eth1Data fastSerialize tree x2048 4.9173 ms/op 4.0735 ms/op 1.21
bytes32 toHexString 597.00 ns/op 454.00 ns/op 1.31
bytes32 Buffer.toString(hex) 255.00 ns/op 280.00 ns/op 0.91
bytes32 Buffer.toString(hex) from Uint8Array 456.00 ns/op 410.00 ns/op 1.11
bytes32 Buffer.toString(hex) + 0x 254.00 ns/op 282.00 ns/op 0.90
Object access 1 prop 0.16700 ns/op 0.16000 ns/op 1.04
Map access 1 prop 0.13300 ns/op 0.14400 ns/op 0.92
Object get x1000 6.1900 ns/op 7.3090 ns/op 0.85
Map get x1000 6.9940 ns/op 0.82900 ns/op 8.44
Object set x1000 49.054 ns/op 46.375 ns/op 1.06
Map set x1000 34.568 ns/op 26.952 ns/op 1.28
Return object 10000 times 0.30550 ns/op 0.25260 ns/op 1.21
Throw Error 10000 times 3.5174 us/op 3.5376 us/op 0.99
fastMsgIdFn sha256 / 200 bytes 2.3970 us/op 2.4420 us/op 0.98
fastMsgIdFn h32 xxhash / 200 bytes 285.00 ns/op 316.00 ns/op 0.90
fastMsgIdFn h64 xxhash / 200 bytes 285.00 ns/op 365.00 ns/op 0.78
fastMsgIdFn sha256 / 1000 bytes 7.5850 us/op 7.7160 us/op 0.98
fastMsgIdFn h32 xxhash / 1000 bytes 415.00 ns/op 452.00 ns/op 0.92
fastMsgIdFn h64 xxhash / 1000 bytes 365.00 ns/op 439.00 ns/op 0.83
fastMsgIdFn sha256 / 10000 bytes 66.546 us/op 66.973 us/op 0.99
fastMsgIdFn h32 xxhash / 10000 bytes 4.2590 us/op 1.9620 us/op 2.17
fastMsgIdFn h64 xxhash / 10000 bytes 3.6550 us/op 1.3320 us/op 2.74
send data - 1000 256B messages 15.300 ms/op 15.178 ms/op 1.01
send data - 1000 512B messages 19.875 ms/op 19.989 ms/op 0.99
send data - 1000 1024B messages 31.758 ms/op 27.936 ms/op 1.14
send data - 1000 1200B messages 27.963 ms/op 21.489 ms/op 1.30
send data - 1000 2048B messages 33.778 ms/op 38.635 ms/op 0.87
send data - 1000 4096B messages 32.888 ms/op 38.485 ms/op 0.85
send data - 1000 16384B messages 78.246 ms/op 84.417 ms/op 0.93
send data - 1000 65536B messages 228.45 ms/op 322.70 ms/op 0.71
enrSubnets - fastDeserialize 64 bits 1.3500 us/op 1.2680 us/op 1.06
enrSubnets - ssz BitVector 64 bits 455.00 ns/op 513.00 ns/op 0.89
enrSubnets - fastDeserialize 4 bits 211.00 ns/op 204.00 ns/op 1.03
enrSubnets - ssz BitVector 4 bits 462.00 ns/op 506.00 ns/op 0.91
prioritizePeers score -10:0 att 32-0.1 sync 2-0 201.23 us/op 202.45 us/op 0.99
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 189.73 us/op 257.98 us/op 0.74
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 320.43 us/op 340.46 us/op 0.94
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 433.94 us/op 519.15 us/op 0.84
prioritizePeers score 0:0 att 64-1 sync 4-1 815.76 us/op 620.46 us/op 1.31
array of 16000 items push then shift 1.7234 us/op 1.7221 us/op 1.00
LinkedList of 16000 items push then shift 8.1060 ns/op 8.4480 ns/op 0.96
array of 16000 items push then pop 145.32 ns/op 143.64 ns/op 1.01
LinkedList of 16000 items push then pop 7.8200 ns/op 7.4310 ns/op 1.05
array of 24000 items push then shift 2.5526 us/op 2.7020 us/op 0.94
LinkedList of 24000 items push then shift 8.0920 ns/op 7.5680 ns/op 1.07
array of 24000 items push then pop 197.51 ns/op 178.29 ns/op 1.11
LinkedList of 24000 items push then pop 7.8860 ns/op 6.7180 ns/op 1.17
intersect bitArray bitLen 8 6.9670 ns/op 6.0160 ns/op 1.16
intersect array and set length 8 71.760 ns/op 57.157 ns/op 1.26
intersect bitArray bitLen 128 32.058 ns/op 36.208 ns/op 0.89
intersect array and set length 128 1.2308 us/op 811.92 ns/op 1.52
bitArray.getTrueBitIndexes() bitLen 128 2.1780 us/op 1.3180 us/op 1.65
bitArray.getTrueBitIndexes() bitLen 248 3.9100 us/op 2.0960 us/op 1.87
bitArray.getTrueBitIndexes() bitLen 512 8.4130 us/op 4.1650 us/op 2.02
Buffer.concat 32 items 1.1070 us/op 860.00 ns/op 1.29
Uint8Array.set 32 items 2.1310 us/op 1.7580 us/op 1.21
Buffer.copy 2.1500 us/op 1.9350 us/op 1.11
Uint8Array.set - with subarray 4.2290 us/op 2.4580 us/op 1.72
Uint8Array.set - without subarray 2.2080 us/op 1.7020 us/op 1.30
Set add up to 64 items then delete first 3.3821 us/op 2.4258 us/op 1.39
OrderedSet add up to 64 items then delete first 5.5606 us/op 3.3601 us/op 1.65
Set add up to 64 items then delete last 3.8073 us/op 2.5285 us/op 1.51
OrderedSet add up to 64 items then delete last 6.2577 us/op 3.7789 us/op 1.66
Set add up to 64 items then delete middle 3.8690 us/op 2.6221 us/op 1.48
OrderedSet add up to 64 items then delete middle 7.8021 us/op 4.8645 us/op 1.60
Set add up to 128 items then delete first 7.5952 us/op 5.2343 us/op 1.45
OrderedSet add up to 128 items then delete first 12.820 us/op 8.4389 us/op 1.52
Set add up to 128 items then delete last 7.4876 us/op 6.4213 us/op 1.17
OrderedSet add up to 128 items then delete last 12.520 us/op 11.490 us/op 1.09
Set add up to 128 items then delete middle 7.3616 us/op 7.3960 us/op 1.00
OrderedSet add up to 128 items then delete middle 18.528 us/op 17.591 us/op 1.05
Set add up to 256 items then delete first 16.232 us/op 16.320 us/op 0.99
OrderedSet add up to 256 items then delete first 25.906 us/op 25.965 us/op 1.00
Set add up to 256 items then delete last 15.464 us/op 17.377 us/op 0.89
OrderedSet add up to 256 items then delete last 24.119 us/op 26.055 us/op 0.93
Set add up to 256 items then delete middle 14.731 us/op 12.880 us/op 1.14
OrderedSet add up to 256 items then delete middle 53.318 us/op 45.374 us/op 1.18
transfer serialized Status (84 B) 1.7240 us/op 1.9850 us/op 0.87
copy serialized Status (84 B) 1.5340 us/op 1.3290 us/op 1.15
transfer serialized SignedVoluntaryExit (112 B) 1.6730 us/op 2.0340 us/op 0.82
copy serialized SignedVoluntaryExit (112 B) 1.5910 us/op 1.6760 us/op 0.95
transfer serialized ProposerSlashing (416 B) 2.2840 us/op 2.4480 us/op 0.93
copy serialized ProposerSlashing (416 B) 2.6360 us/op 2.1600 us/op 1.22
transfer serialized Attestation (485 B) 2.1690 us/op 2.9640 us/op 0.73
copy serialized Attestation (485 B) 2.3580 us/op 1.9820 us/op 1.19
transfer serialized AttesterSlashing (33232 B) 4.0470 us/op 2.3240 us/op 1.74
copy serialized AttesterSlashing (33232 B) 10.947 us/op 11.485 us/op 0.95
transfer serialized Small SignedBeaconBlock (128000 B) 4.2840 us/op 3.1210 us/op 1.37
copy serialized Small SignedBeaconBlock (128000 B) 34.010 us/op 36.542 us/op 0.93
transfer serialized Avg SignedBeaconBlock (200000 B) 5.0300 us/op 4.0810 us/op 1.23
copy serialized Avg SignedBeaconBlock (200000 B) 47.851 us/op 51.741 us/op 0.92
transfer serialized BlobsSidecar (524380 B) 4.8970 us/op 5.3190 us/op 0.92
copy serialized BlobsSidecar (524380 B) 126.87 us/op 170.88 us/op 0.74
transfer serialized Big SignedBeaconBlock (1000000 B) 3.6590 us/op 5.5620 us/op 0.66
copy serialized Big SignedBeaconBlock (1000000 B) 265.26 us/op 391.14 us/op 0.68
pass gossip attestations to forkchoice per slot 3.4000 ms/op 3.5228 ms/op 0.97
forkChoice updateHead vc 100000 bc 64 eq 0 530.23 us/op 577.32 us/op 0.92
forkChoice updateHead vc 600000 bc 64 eq 0 3.7545 ms/op 5.9359 ms/op 0.63
forkChoice updateHead vc 1000000 bc 64 eq 0 6.9635 ms/op 7.2292 ms/op 0.96
forkChoice updateHead vc 600000 bc 320 eq 0 4.0447 ms/op 3.9336 ms/op 1.03
forkChoice updateHead vc 600000 bc 1200 eq 0 4.1698 ms/op 4.3957 ms/op 0.95
forkChoice updateHead vc 600000 bc 7200 eq 0 5.1434 ms/op 5.9561 ms/op 0.86
forkChoice updateHead vc 600000 bc 64 eq 1000 11.417 ms/op 11.262 ms/op 1.01
forkChoice updateHead vc 600000 bc 64 eq 10000 11.503 ms/op 11.498 ms/op 1.00
forkChoice updateHead vc 600000 bc 64 eq 300000 29.981 ms/op 27.811 ms/op 1.08
computeDeltas 500000 validators 300 proto nodes 4.1863 ms/op 4.7317 ms/op 0.88
computeDeltas 500000 validators 1200 proto nodes 5.4991 ms/op 4.4011 ms/op 1.25
computeDeltas 500000 validators 7200 proto nodes 6.1565 ms/op 4.3780 ms/op 1.41
computeDeltas 750000 validators 300 proto nodes 9.6519 ms/op 5.8905 ms/op 1.64
computeDeltas 750000 validators 1200 proto nodes 6.7948 ms/op 6.2860 ms/op 1.08
computeDeltas 750000 validators 7200 proto nodes 6.8433 ms/op 5.7938 ms/op 1.18
computeDeltas 1400000 validators 300 proto nodes 14.084 ms/op 10.666 ms/op 1.32
computeDeltas 1400000 validators 1200 proto nodes 14.044 ms/op 10.422 ms/op 1.35
computeDeltas 1400000 validators 7200 proto nodes 13.076 ms/op 9.8196 ms/op 1.33
computeDeltas 2100000 validators 300 proto nodes 17.735 ms/op 14.477 ms/op 1.23
computeDeltas 2100000 validators 1200 proto nodes 16.919 ms/op 14.634 ms/op 1.16
computeDeltas 2100000 validators 7200 proto nodes 16.934 ms/op 15.341 ms/op 1.10
altair processAttestation - 250000 vs - 7PWei normalcase 1.8706 ms/op 1.8005 ms/op 1.04
altair processAttestation - 250000 vs - 7PWei worstcase 2.9328 ms/op 3.3565 ms/op 0.87
altair processAttestation - setStatus - 1/6 committees join 94.574 us/op 138.49 us/op 0.68
altair processAttestation - setStatus - 1/3 committees join 194.59 us/op 279.08 us/op 0.70
altair processAttestation - setStatus - 1/2 committees join 283.58 us/op 403.35 us/op 0.70
altair processAttestation - setStatus - 2/3 committees join 368.36 us/op 478.24 us/op 0.77
altair processAttestation - setStatus - 4/5 committees join 551.07 us/op 663.75 us/op 0.83
altair processAttestation - setStatus - 100% committees join 634.96 us/op 828.05 us/op 0.77
altair processBlock - 250000 vs - 7PWei normalcase 5.7642 ms/op 9.1828 ms/op 0.63
altair processBlock - 250000 vs - 7PWei normalcase hashState 26.481 ms/op 38.930 ms/op 0.68
altair processBlock - 250000 vs - 7PWei worstcase 48.426 ms/op 33.732 ms/op 1.44
altair processBlock - 250000 vs - 7PWei worstcase hashState 84.093 ms/op 99.842 ms/op 0.84
phase0 processBlock - 250000 vs - 7PWei normalcase 2.3029 ms/op 2.8316 ms/op 0.81
phase0 processBlock - 250000 vs - 7PWei worstcase 29.948 ms/op 31.345 ms/op 0.96
altair processEth1Data - 250000 vs - 7PWei normalcase 434.70 us/op 575.72 us/op 0.76
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 6.7100 us/op 20.016 us/op 0.34
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 28.090 us/op 78.520 us/op 0.36
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 9.9490 us/op 24.386 us/op 0.41
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 7.1670 us/op 16.203 us/op 0.44
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 120.01 us/op 232.44 us/op 0.52
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 704.74 us/op 1.4294 ms/op 0.49
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 954.16 us/op 1.6577 ms/op 0.58
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 973.25 us/op 1.6281 ms/op 0.60
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 2.2868 ms/op 3.4342 ms/op 0.67
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.6890 ms/op 3.0323 ms/op 0.56
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 4.3164 ms/op 4.2436 ms/op 1.02
Tree 40 250000 create 265.00 ms/op 226.50 ms/op 1.17
Tree 40 250000 get(125000) 162.09 ns/op 151.80 ns/op 1.07
Tree 40 250000 set(125000) 844.89 ns/op 689.21 ns/op 1.23
Tree 40 250000 toArray() 19.579 ms/op 18.327 ms/op 1.07
Tree 40 250000 iterate all - toArray() + loop 17.157 ms/op 16.471 ms/op 1.04
Tree 40 250000 iterate all - get(i) 50.741 ms/op 59.284 ms/op 0.86
MutableVector 250000 create 8.2181 ms/op 8.3733 ms/op 0.98
MutableVector 250000 get(125000) 6.2240 ns/op 6.8270 ns/op 0.91
MutableVector 250000 set(125000) 195.89 ns/op 235.54 ns/op 0.83
MutableVector 250000 toArray() 3.3928 ms/op 4.1303 ms/op 0.82
MutableVector 250000 iterate all - toArray() + loop 3.8258 ms/op 4.9632 ms/op 0.77
MutableVector 250000 iterate all - get(i) 1.6663 ms/op 1.7527 ms/op 0.95
Array 250000 create 3.3203 ms/op 3.9852 ms/op 0.83
Array 250000 clone - spread 1.5283 ms/op 1.4409 ms/op 1.06
Array 250000 get(125000) 0.41300 ns/op 0.99800 ns/op 0.41
Array 250000 set(125000) 0.44200 ns/op 1.2290 ns/op 0.36
Array 250000 iterate all - loop 98.215 us/op 168.16 us/op 0.58
effectiveBalanceIncrements clone Uint8Array 300000 27.520 us/op 33.614 us/op 0.82
effectiveBalanceIncrements clone MutableVector 300000 118.00 ns/op 311.00 ns/op 0.38
effectiveBalanceIncrements rw all Uint8Array 300000 198.74 us/op 202.44 us/op 0.98
effectiveBalanceIncrements rw all MutableVector 300000 66.211 ms/op 77.851 ms/op 0.85
phase0 afterProcessEpoch - 250000 vs - 7PWei 89.319 ms/op 92.217 ms/op 0.97
phase0 beforeProcessEpoch - 250000 vs - 7PWei 40.522 ms/op 47.568 ms/op 0.85
altair processEpoch - mainnet_e81889 358.88 ms/op 407.61 ms/op 0.88
mainnet_e81889 - altair beforeProcessEpoch 65.921 ms/op 75.233 ms/op 0.88
mainnet_e81889 - altair processJustificationAndFinalization 12.937 us/op 17.173 us/op 0.75
mainnet_e81889 - altair processInactivityUpdates 5.6371 ms/op 5.6221 ms/op 1.00
mainnet_e81889 - altair processRewardsAndPenalties 34.380 ms/op 40.128 ms/op 0.86
mainnet_e81889 - altair processRegistryUpdates 1.8110 us/op 2.4800 us/op 0.73
mainnet_e81889 - altair processSlashings 344.00 ns/op 477.00 ns/op 0.72
mainnet_e81889 - altair processEth1DataReset 261.00 ns/op 931.00 ns/op 0.28
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.1428 ms/op 1.4703 ms/op 0.78
mainnet_e81889 - altair processSlashingsReset 4.9080 us/op 4.8730 us/op 1.01
mainnet_e81889 - altair processRandaoMixesReset 4.2280 us/op 6.2030 us/op 0.68
mainnet_e81889 - altair processHistoricalRootsUpdate 449.00 ns/op 983.00 ns/op 0.46
mainnet_e81889 - altair processParticipationFlagUpdates 1.7140 us/op 2.2730 us/op 0.75
mainnet_e81889 - altair processSyncCommitteeUpdates 651.00 ns/op 1.0150 us/op 0.64
mainnet_e81889 - altair afterProcessEpoch 93.410 ms/op 100.33 ms/op 0.93
capella processEpoch - mainnet_e217614 1.4780 s/op 1.5443 s/op 0.96
mainnet_e217614 - capella beforeProcessEpoch 273.75 ms/op 299.30 ms/op 0.91
mainnet_e217614 - capella processJustificationAndFinalization 24.377 us/op 18.902 us/op 1.29
mainnet_e217614 - capella processInactivityUpdates 18.806 ms/op 17.245 ms/op 1.09
mainnet_e217614 - capella processRewardsAndPenalties 257.32 ms/op 250.60 ms/op 1.03
mainnet_e217614 - capella processRegistryUpdates 34.055 us/op 31.322 us/op 1.09
mainnet_e217614 - capella processSlashings 2.9870 us/op 649.00 ns/op 4.60
mainnet_e217614 - capella processEth1DataReset 1.2450 us/op 430.00 ns/op 2.90
mainnet_e217614 - capella processEffectiveBalanceUpdates 18.452 ms/op 4.3153 ms/op 4.28
mainnet_e217614 - capella processSlashingsReset 8.4540 us/op 3.8000 us/op 2.22
mainnet_e217614 - capella processRandaoMixesReset 12.741 us/op 6.0340 us/op 2.11
mainnet_e217614 - capella processHistoricalRootsUpdate 1.8940 us/op 992.00 ns/op 1.91
mainnet_e217614 - capella processParticipationFlagUpdates 7.2800 us/op 1.9630 us/op 3.71
mainnet_e217614 - capella afterProcessEpoch 388.17 ms/op 275.25 ms/op 1.41
phase0 processEpoch - mainnet_e58758 750.11 ms/op 498.36 ms/op 1.51
mainnet_e58758 - phase0 beforeProcessEpoch 275.46 ms/op 156.95 ms/op 1.76
mainnet_e58758 - phase0 processJustificationAndFinalization 38.209 us/op 22.245 us/op 1.72
mainnet_e58758 - phase0 processRewardsAndPenalties 49.323 ms/op 22.159 ms/op 2.23
mainnet_e58758 - phase0 processRegistryUpdates 24.141 us/op 13.971 us/op 1.73
mainnet_e58758 - phase0 processSlashings 1.2920 us/op 588.00 ns/op 2.20
mainnet_e58758 - phase0 processEth1DataReset 1.4220 us/op 432.00 ns/op 3.29
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.7619 ms/op 1.1432 ms/op 2.42
mainnet_e58758 - phase0 processSlashingsReset 13.020 us/op 4.0090 us/op 3.25
mainnet_e58758 - phase0 processRandaoMixesReset 12.156 us/op 4.9280 us/op 2.47
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.5180 us/op 430.00 ns/op 3.53
mainnet_e58758 - phase0 processParticipationRecordUpdates 11.880 us/op 4.7500 us/op 2.50
mainnet_e58758 - phase0 afterProcessEpoch 109.72 ms/op 84.735 ms/op 1.29
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.0534 ms/op 1.3000 ms/op 1.58
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.8511 ms/op 5.3494 ms/op 0.72
altair processInactivityUpdates - 250000 normalcase 39.292 ms/op 17.397 ms/op 2.26
altair processInactivityUpdates - 250000 worstcase 38.571 ms/op 17.093 ms/op 2.26
phase0 processRegistryUpdates - 250000 normalcase 22.264 us/op 12.195 us/op 1.83
phase0 processRegistryUpdates - 250000 badcase_full_deposits 541.44 us/op 374.11 us/op 1.45
phase0 processRegistryUpdates - 250000 worstcase 0.5 152.76 ms/op 144.93 ms/op 1.05
altair processRewardsAndPenalties - 250000 normalcase 64.267 ms/op 45.625 ms/op 1.41
altair processRewardsAndPenalties - 250000 worstcase 52.514 ms/op 44.668 ms/op 1.18
phase0 getAttestationDeltas - 250000 normalcase 14.512 ms/op 8.6649 ms/op 1.67
phase0 getAttestationDeltas - 250000 worstcase 9.4267 ms/op 9.3644 ms/op 1.01
phase0 processSlashings - 250000 worstcase 117.93 us/op 90.393 us/op 1.30
altair processSyncCommitteeUpdates - 250000 147.72 ms/op 137.11 ms/op 1.08
BeaconState.hashTreeRoot - No change 372.00 ns/op 484.00 ns/op 0.77
BeaconState.hashTreeRoot - 1 full validator 120.88 us/op 139.48 us/op 0.87
BeaconState.hashTreeRoot - 32 full validator 1.7045 ms/op 1.5793 ms/op 1.08
BeaconState.hashTreeRoot - 512 full validator 15.511 ms/op 14.521 ms/op 1.07
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 183.00 us/op 154.61 us/op 1.18
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 3.1321 ms/op 2.1347 ms/op 1.47
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 37.211 ms/op 22.629 ms/op 1.64
BeaconState.hashTreeRoot - 1 balances 134.37 us/op 119.33 us/op 1.13
BeaconState.hashTreeRoot - 32 balances 1.1940 ms/op 1.3224 ms/op 0.90
BeaconState.hashTreeRoot - 512 balances 12.169 ms/op 10.348 ms/op 1.18
BeaconState.hashTreeRoot - 250000 balances 207.14 ms/op 188.35 ms/op 1.10
aggregationBits - 2048 els - zipIndexesInBitList 44.737 us/op 26.606 us/op 1.68
byteArrayEquals 32 58.702 ns/op 76.758 ns/op 0.76
Buffer.compare 32 52.019 ns/op 52.083 ns/op 1.00
byteArrayEquals 1024 1.6369 us/op 2.0902 us/op 0.78
Buffer.compare 1024 55.603 ns/op 53.031 ns/op 1.05
byteArrayEquals 16384 26.244 us/op 33.277 us/op 0.79
Buffer.compare 16384 260.69 ns/op 240.68 ns/op 1.08
byteArrayEquals 123687377 211.88 ms/op 258.62 ms/op 0.82
Buffer.compare 123687377 8.7188 ms/op 8.5620 ms/op 1.02
byteArrayEquals 32 - diff last byte 53.570 ns/op 76.409 ns/op 0.70
Buffer.compare 32 - diff last byte 46.593 ns/op 52.859 ns/op 0.88
byteArrayEquals 1024 - diff last byte 1.6788 us/op 2.2269 us/op 0.75
Buffer.compare 1024 - diff last byte 57.512 ns/op 56.932 ns/op 1.01
byteArrayEquals 16384 - diff last byte 28.081 us/op 33.224 us/op 0.85
Buffer.compare 16384 - diff last byte 260.47 ns/op 232.68 ns/op 1.12
byteArrayEquals 123687377 - diff last byte 219.70 ms/op 251.80 ms/op 0.87
Buffer.compare 123687377 - diff last byte 8.8905 ms/op 8.0219 ms/op 1.11
byteArrayEquals 32 - random bytes 5.4040 ns/op 5.7060 ns/op 0.95
Buffer.compare 32 - random bytes 50.130 ns/op 52.205 ns/op 0.96
byteArrayEquals 1024 - random bytes 5.6800 ns/op 5.5290 ns/op 1.03
Buffer.compare 1024 - random bytes 49.885 ns/op 51.235 ns/op 0.97
byteArrayEquals 16384 - random bytes 5.8870 ns/op 5.5270 ns/op 1.07
Buffer.compare 16384 - random bytes 51.090 ns/op 51.360 ns/op 0.99
byteArrayEquals 123687377 - random bytes 7.2800 ns/op 8.3900 ns/op 0.87
Buffer.compare 123687377 - random bytes 56.290 ns/op 55.650 ns/op 1.01
regular array get 100000 times 43.675 us/op 45.339 us/op 0.96
wrappedArray get 100000 times 35.490 us/op 45.599 us/op 0.78
arrayWithProxy get 100000 times 14.476 ms/op 14.268 ms/op 1.01
ssz.Root.equals 46.595 ns/op 57.033 ns/op 0.82
byteArrayEquals 45.666 ns/op 53.686 ns/op 0.85
Buffer.compare 10.563 ns/op 10.997 ns/op 0.96
shuffle list - 16384 els 6.5492 ms/op 6.6871 ms/op 0.98
shuffle list - 250000 els 95.335 ms/op 99.323 ms/op 0.96
processSlot - 1 slots 13.019 us/op 15.135 us/op 0.86
processSlot - 32 slots 2.6275 ms/op 3.4792 ms/op 0.76
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 37.150 ms/op 47.169 ms/op 0.79
getCommitteeAssignments - req 1 vs - 250000 vc 2.2801 ms/op 2.6829 ms/op 0.85
getCommitteeAssignments - req 100 vs - 250000 vc 4.6652 ms/op 3.8575 ms/op 1.21
getCommitteeAssignments - req 1000 vs - 250000 vc 4.6726 ms/op 4.2412 ms/op 1.10
findModifiedValidators - 10000 modified validators 317.48 ms/op 311.01 ms/op 1.02
findModifiedValidators - 1000 modified validators 206.72 ms/op 220.13 ms/op 0.94
findModifiedValidators - 100 modified validators 205.51 ms/op 217.60 ms/op 0.94
findModifiedValidators - 10 modified validators 210.39 ms/op 191.80 ms/op 1.10
findModifiedValidators - 1 modified validators 196.51 ms/op 184.59 ms/op 1.06
findModifiedValidators - no difference 182.30 ms/op 194.63 ms/op 0.94
compare ViewDUs 3.1590 s/op 4.1051 s/op 0.77
compare each validator Uint8Array 1.3317 s/op 1.5573 s/op 0.86
compare ViewDU to Uint8Array 1.3602 s/op 1.2889 s/op 1.06
migrate state 1000000 validators, 24 modified, 0 new 587.88 ms/op 642.52 ms/op 0.91
migrate state 1000000 validators, 1700 modified, 1000 new 844.76 ms/op 916.66 ms/op 0.92
migrate state 1000000 validators, 3400 modified, 2000 new 1.0183 s/op 1.1687 s/op 0.87
migrate state 1500000 validators, 24 modified, 0 new 705.01 ms/op 675.94 ms/op 1.04
migrate state 1500000 validators, 1700 modified, 1000 new 1.0144 s/op 877.72 ms/op 1.16
migrate state 1500000 validators, 3400 modified, 2000 new 1.3065 s/op 1.1449 s/op 1.14
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 5.5500 ns/op 4.0300 ns/op 1.38
state getBlockRootAtSlot - 250000 vs - 7PWei 690.75 ns/op 658.45 ns/op 1.05
computeProposers - vc 250000 10.050 ms/op 8.6700 ms/op 1.16
computeEpochShuffling - vc 250000 108.04 ms/op 97.930 ms/op 1.10
getNextSyncCommittee - vc 250000 172.55 ms/op 139.76 ms/op 1.23
computeSigningRoot for AttestationData 34.146 us/op 26.015 us/op 1.31
hash AttestationData serialized data then Buffer.toString(base64) 1.7441 us/op 1.6037 us/op 1.09
toHexString serialized data 1.5110 us/op 957.76 ns/op 1.58
Buffer.toString(base64) 235.73 ns/op 215.65 ns/op 1.09

by benchmarkbot/action

@philknows
Copy link
Member

Consensus on today's standup is that we would feel a lot more comfortable to enable by default only if we had DOS protection measures in place. Something like #6838 where we have protection measures and adding a CLI flag disabling certain endpoints/functionality.

It was also suggested that we should make our documentation note more prominent about having to manually turn on debug API. Argument is that experienced power users of Lodestar will be able to enable it via flag rather than burdening inexperienced users with exposing this risk by default.

@nflaig
Copy link
Member Author

nflaig commented Jun 5, 2024

Consensus on today's standup is that we would feel a lot more comfortable to enable by default only if we had DOS protection measures in place

After watching the recording and what was discussed there, it feels to me more like a misunderstanding of the current situation. Enabling debug routes by default does not introduce any new DoS vector as the same can already be achieved by calling getStateValidators api which is part of beacon namespace.

Something like #6838 where we have protection measures and adding a CLI flag disabling certain endpoints/functionality.

Whether or not we implement a solution for #6838 is not relevant for this PR. The issue is meant to address protecting public beacon nodes which have debug namespace enabled explicitly already.

rather than burdening inexperienced users with exposing this risk by default.

If operators / stakers expose their beacon api publicly due to misconfiguration, having their node go down due to oom is the least of their worries. Please see #6822 (comment). and just in general, enabling debug endpoints by default doesn't introduce any new risks, as mentioned multiple times (happy to be proven otherwise)

It was also suggested that we should make our documentation note more prominent about having to manually turn on debug API. Argument is that experienced power users of Lodestar will be able to enable it via flag

Our client should just work by default, people trying to use ethdo or other tooling that requires debug routes might not be advanced users that know about these different api namespaces. I've seen plenty of users facing this issue with Lodestar specifically because all other clients enable the debug namespace by default already. There was even discussion on the beacon-api repo to move it out of debug namespace (ethereum/beacon-APIs#357) but it was closed due to friction this would create changing the path, not because it would introduce a risk.

As a side note, a lot of tooling (e.g. eth-docker) already enables debug by default (ref) because of the UX degradation mentioned above of having the user figure this out themselves.

@matthewkeil
Copy link
Member

Consensus on today's standup is that we would feel a lot more comfortable to enable by default only if we had DOS protection measures in place

After watching the recording and what was discussed there, it feels to me more like a misunderstanding of the current situation. Enabling debug routes by default does not introduce any new DoS vector as the same can already be achieved by calling getStateValidators api which is part of beacon namespace.

Makes sense. I thought getting the state was the only route that causes an issue.

Something like #6838 where we have protection measures and adding a CLI flag disabling certain endpoints/functionality.

Whether or not we implement a solution for #6838 is not relevant for this PR. The issue is meant to address protecting public beacon nodes which have debug namespace enabled explicitly already.

Also makes sense.

rather than burdening inexperienced users with exposing this risk by default.

If operators / stakers expose their beacon api publicly due to misconfiguration, having their node go down due to oom is the least of their worries. Please see #6822 (comment). and just in general, enabling debug endpoints by default doesn't introduce any new risks, as mentioned multiple times (happy to be proven otherwise)

I tend to lean towards this as an issue but truth be told, it can be solved with documentation. I also tend to think that the single-threaded nature of JS causes more risk than just OOM. Large api requests halt the world for everything else so its also worth noting this in the docs if we turn this on by default.

It was also suggested that we should make our documentation note more prominent about having to manually turn on debug API. Argument is that experienced power users of Lodestar will be able to enable it via flag

Our client should just work by default, people trying to use ethdo or other tooling that requires debug routes might not be advanced users that know about these different api namespaces. I've seen plenty of users facing this issue with Lodestar specifically because all other clients enable the debug namespace by default already.

Agreed. I have noticed a lot of issues related to this and I suppose the UX of having it on by default may sway my opinion considering the discussion above.

There was even discussion on the beacon-api repo to move it out of debug namespace (ethereum/beacon-APIs#357) but it was closed due to friction this would create changing the path, not because it would introduce a risk.

Yes, but this was what lighthouse mentioned and this is very much not the case for us.... The node stops with a 400mb api response. ethereum/beacon-APIs#357 (comment)

As a side note, a lot of tooling (e.g. eth-docker) already enables debug by default (ref) because of the UX degradation mentioned above of having the user figure this out themselves.

I tend to see this in a new light but would like to put this thought forward, if the consumers are not seeing the note in the docs about turning on the debug namespace will they also miss the note about protecting one's node from attacks.

If we ensure that idea is front-and-center (even more prominently displayed in getting-started, on the readme, etc), that open apis are risky (should be common sense but that is definitely not so common) I suppose this will be a good change from a DX perspective...

@nflaig
Copy link
Member Author

nflaig commented Jun 5, 2024 •
edited
Loading

Yes, but this was what lighthouse mentioned and this is very much not the case for us.... The node stops with a 400mb api response. ethereum/beacon-APIs#357 (comment)

I would argue it's worse in case of Lighthouse because their node will bring down the whole server, maybe they are streaming the data more efficiently but without proper rate limiting in place this is a huge DoS vector for them as well. that's why we wanna have something like #6838 ...the getStateValidators response is more than 600MB (json-serialized) on mainnet, no server can handle serialization this if there are 100x consecutive requests..not even rust

they also miss the note about protecting one's node from attacks.

Right now, a user has to go out of their way to expose the beacon api, you have to set --rest.address 0.0.0.0, ignore the warning logged in the console, configure port-forwarding (in case of home staker) and have no firewall set up.

and we have a warning in the docs already
image

and console

warn: REST API server is exposed, ensure untrusted traffic cannot reach this API

If there are further ideas how we can warn about the api being exposed I am all for it. We could push for having authentication (similar to keymanager api) enabled by default but it will be hard to get consensus on this on spec level, and I don't see it as useful either.

@wemeetagain
Copy link
Member

getStateValidators

Ah yes, forgot about that terrible endpoint.
Are there any efforts to deprecate the worst offenders?

Noting that an exposed API is already a DoS vector, it seems ok to add the debug namespace by default.

I guess the hope would be that we could disable the worst endpoints / variants by default.
The hope is that an exposed API would go from terrible DoS vector (where on the order of 1 bad request blows up a node) to less-terrible DoS vector (where on the order of 1000 requests blow up the node).

@nflaig
Copy link
Member Author

nflaig commented Jun 5, 2024

Are there any efforts to deprecate the worst offenders?

There are efforts but since this api is required by a lot of infrastructure / tooling around the CL (and the VC itself) it is required to provide an alternative, if all offenders support ssz, we could at least disable JSON on those, see ethereum/beacon-APIs#449 for potential solution.

The hope is that an exposed API would go from terrible DoS vector (where on the order of 1 bad request blows up a node) to less-terrible DoS vector (where on the order of 1000 requests blow up the node).

yeah that would the best outcome we can get if people wanna expose the beacon api directly. It will still be quite easy to DoS, if people wanna run a infrastructure provider like Infura for the CL, they will have to implement a caching layer and other protections either way.

@nflaig nflaig merged commit c153277 into unstable Jun 7, 2024
19 of 20 checks passed
@nflaig nflaig deleted the nflaig/enable-debug branch June 7, 2024 16:56
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.20.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Consider enabling debug namespace by default
4 participants
@nflaig @philknows @matthewkeil @wemeetagain