Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: prune invalid ssz objects #6875

Merged
merged 4 commits into from
Jun 19, 2024
Merged

fix: prune invalid ssz objects #6875

merged 4 commits into from
Jun 19, 2024

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Jun 11, 2024

Motivation

We implemented a feature to prune invalid ssz objects but still see a lot of them there

Description

  • I think it's not triggered because we should get persistInvalidSszObjectsDir from chain options instead of args
  • Also add some logs to confirm

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jun 11, 2024 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 9a6f998 Previous: f37e50f Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 498.05 us/op 935.19 us/op 0.53
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 38.768 us/op 75.018 us/op 0.52
BLS verify - blst-native 1.1057 ms/op 1.3242 ms/op 0.83
BLS verifyMultipleSignatures 3 - blst-native 2.3571 ms/op 2.8662 ms/op 0.82
BLS verifyMultipleSignatures 8 - blst-native 5.2081 ms/op 6.2540 ms/op 0.83
BLS verifyMultipleSignatures 32 - blst-native 19.104 ms/op 22.723 ms/op 0.84
BLS verifyMultipleSignatures 64 - blst-native 37.172 ms/op 46.564 ms/op 0.80
BLS verifyMultipleSignatures 128 - blst-native 73.790 ms/op 93.196 ms/op 0.79
BLS deserializing 10000 signatures 782.13 ms/op 984.19 ms/op 0.79
BLS deserializing 100000 signatures 7.6462 s/op 10.164 s/op 0.75
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.0847 ms/op 1.4347 ms/op 0.76
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.2431 ms/op 1.7791 ms/op 0.70
BLS verifyMultipleSignatures - same message - 32 - blst-native 1.9502 ms/op 2.9663 ms/op 0.66
BLS verifyMultipleSignatures - same message - 64 - blst-native 3.3758 ms/op 4.0338 ms/op 0.84
BLS verifyMultipleSignatures - same message - 128 - blst-native 5.8682 ms/op 7.5617 ms/op 0.78
BLS aggregatePubkeys 32 - blst-native 22.981 us/op 28.784 us/op 0.80
BLS aggregatePubkeys 128 - blst-native 85.048 us/op 108.01 us/op 0.79
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 61.807 ms/op 94.486 ms/op 0.65
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 45.633 ms/op 76.690 ms/op 0.60
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 29.671 ms/op 43.001 ms/op 0.69
getSlashingsAndExits - default max 76.268 us/op 213.83 us/op 0.36
getSlashingsAndExits - 2k 276.53 us/op 475.00 us/op 0.58
proposeBlockBody type=full, size=empty 4.9999 ms/op 8.2561 ms/op 0.61
isKnown best case - 1 super set check 459.00 ns/op 889.00 ns/op 0.52
isKnown normal case - 2 super set checks 456.00 ns/op 943.00 ns/op 0.48
isKnown worse case - 16 super set checks 455.00 ns/op 863.00 ns/op 0.53
InMemoryCheckpointStateCache - add get delete 4.3010 us/op 7.1940 us/op 0.60
validate api signedAggregateAndProof - struct 2.2478 ms/op 3.1604 ms/op 0.71
validate gossip signedAggregateAndProof - struct 2.4512 ms/op 3.0473 ms/op 0.80
validate gossip attestation - vc 640000 1.1170 ms/op 1.5333 ms/op 0.73
batch validate gossip attestation - vc 640000 - chunk 32 136.97 us/op 194.35 us/op 0.70
batch validate gossip attestation - vc 640000 - chunk 64 122.65 us/op 156.96 us/op 0.78
batch validate gossip attestation - vc 640000 - chunk 128 112.30 us/op 149.87 us/op 0.75
batch validate gossip attestation - vc 640000 - chunk 256 105.19 us/op 150.60 us/op 0.70
pickEth1Vote - no votes 830.81 us/op 1.2908 ms/op 0.64
pickEth1Vote - max votes 8.5935 ms/op 11.580 ms/op 0.74
pickEth1Vote - Eth1Data hashTreeRoot value x2048 14.608 ms/op 16.975 ms/op 0.86
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 19.849 ms/op 25.875 ms/op 0.77
pickEth1Vote - Eth1Data fastSerialize value x2048 404.97 us/op 642.48 us/op 0.63
pickEth1Vote - Eth1Data fastSerialize tree x2048 3.1896 ms/op 5.3289 ms/op 0.60
bytes32 toHexString 574.00 ns/op 705.00 ns/op 0.81
bytes32 Buffer.toString(hex) 433.00 ns/op 295.00 ns/op 1.47
bytes32 Buffer.toString(hex) from Uint8Array 537.00 ns/op 502.00 ns/op 1.07
bytes32 Buffer.toString(hex) + 0x 447.00 ns/op 291.00 ns/op 1.54
Object access 1 prop 0.32900 ns/op 0.20100 ns/op 1.64
Map access 1 prop 0.32600 ns/op 0.16000 ns/op 2.04
Object get x1000 5.2970 ns/op 6.8240 ns/op 0.78
Map get x1000 6.0240 ns/op 7.4650 ns/op 0.81
Object set x1000 26.314 ns/op 59.981 ns/op 0.44
Map set x1000 19.370 ns/op 40.271 ns/op 0.48
Return object 10000 times 0.30460 ns/op 0.34060 ns/op 0.89
Throw Error 10000 times 2.7911 us/op 4.0613 us/op 0.69
fastMsgIdFn sha256 / 200 bytes 1.9650 us/op 2.4270 us/op 0.81
fastMsgIdFn h32 xxhash / 200 bytes 400.00 ns/op 342.00 ns/op 1.17
fastMsgIdFn h64 xxhash / 200 bytes 438.00 ns/op 319.00 ns/op 1.37
fastMsgIdFn sha256 / 1000 bytes 6.1800 us/op 8.1040 us/op 0.76
fastMsgIdFn h32 xxhash / 1000 bytes 546.00 ns/op 469.00 ns/op 1.16
fastMsgIdFn h64 xxhash / 1000 bytes 530.00 ns/op 387.00 ns/op 1.37
fastMsgIdFn sha256 / 10000 bytes 51.210 us/op 70.556 us/op 0.73
fastMsgIdFn h32 xxhash / 10000 bytes 1.9700 us/op 2.1040 us/op 0.94
fastMsgIdFn h64 xxhash / 10000 bytes 1.3630 us/op 1.3410 us/op 1.02
send data - 1000 256B messages 10.073 ms/op 15.474 ms/op 0.65
send data - 1000 512B messages 14.317 ms/op 20.113 ms/op 0.71
send data - 1000 1024B messages 21.803 ms/op 27.707 ms/op 0.79
send data - 1000 1200B messages 25.313 ms/op 28.918 ms/op 0.88
send data - 1000 2048B messages 32.168 ms/op 35.026 ms/op 0.92
send data - 1000 4096B messages 33.426 ms/op 36.592 ms/op 0.91
send data - 1000 16384B messages 70.987 ms/op 79.416 ms/op 0.89
send data - 1000 65536B messages 293.21 ms/op 260.79 ms/op 1.12
enrSubnets - fastDeserialize 64 bits 1.1550 us/op 1.4750 us/op 0.78
enrSubnets - ssz BitVector 64 bits 623.00 ns/op 470.00 ns/op 1.33
enrSubnets - fastDeserialize 4 bits 397.00 ns/op 223.00 ns/op 1.78
enrSubnets - ssz BitVector 4 bits 590.00 ns/op 448.00 ns/op 1.32
prioritizePeers score -10:0 att 32-0.1 sync 2-0 161.35 us/op 171.12 us/op 0.94
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 158.69 us/op 185.23 us/op 0.86
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 287.95 us/op 375.65 us/op 0.77
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 520.01 us/op 499.85 us/op 1.04
prioritizePeers score 0:0 att 64-1 sync 4-1 892.28 us/op 833.57 us/op 1.07
array of 16000 items push then shift 1.3036 us/op 1.7639 us/op 0.74
LinkedList of 16000 items push then shift 8.3360 ns/op 8.4020 ns/op 0.99
array of 16000 items push then pop 133.71 ns/op 142.17 ns/op 0.94
LinkedList of 16000 items push then pop 7.3250 ns/op 7.9120 ns/op 0.93
array of 24000 items push then shift 1.8847 us/op 2.5975 us/op 0.73
LinkedList of 24000 items push then shift 7.8650 ns/op 7.9050 ns/op 0.99
array of 24000 items push then pop 176.93 ns/op 188.25 ns/op 0.94
LinkedList of 24000 items push then pop 7.6390 ns/op 7.7800 ns/op 0.98
intersect bitArray bitLen 8 5.8140 ns/op 7.1350 ns/op 0.81
intersect array and set length 8 78.983 ns/op 55.600 ns/op 1.42
intersect bitArray bitLen 128 27.103 ns/op 32.227 ns/op 0.84
intersect array and set length 128 922.42 ns/op 855.71 ns/op 1.08
bitArray.getTrueBitIndexes() bitLen 128 2.7900 us/op 2.5660 us/op 1.09
bitArray.getTrueBitIndexes() bitLen 248 5.5560 us/op 4.3570 us/op 1.28
bitArray.getTrueBitIndexes() bitLen 512 9.5620 us/op 10.622 us/op 0.90
Buffer.concat 32 items 1.4440 us/op 1.1430 us/op 1.26
Uint8Array.set 32 items 2.3390 us/op 2.4490 us/op 0.96
Buffer.copy 2.2980 us/op 2.7660 us/op 0.83
Uint8Array.set - with subarray 3.8280 us/op 4.2830 us/op 0.89
Uint8Array.set - without subarray 2.0650 us/op 2.4570 us/op 0.84
Set add up to 64 items then delete first 2.7360 us/op 3.0651 us/op 0.89
OrderedSet add up to 64 items then delete first 4.7271 us/op 3.9120 us/op 1.21
Set add up to 64 items then delete last 2.3425 us/op 3.0198 us/op 0.78
OrderedSet add up to 64 items then delete last 4.2683 us/op 5.0422 us/op 0.85
Set add up to 64 items then delete middle 2.9581 us/op 3.1271 us/op 0.95
OrderedSet add up to 64 items then delete middle 6.7468 us/op 6.5077 us/op 1.04
Set add up to 128 items then delete first 6.0767 us/op 6.3505 us/op 0.96
OrderedSet add up to 128 items then delete first 9.4077 us/op 9.2761 us/op 1.01
Set add up to 128 items then delete last 6.0970 us/op 6.9510 us/op 0.88
OrderedSet add up to 128 items then delete last 9.8198 us/op 10.349 us/op 0.95
Set add up to 128 items then delete middle 5.6488 us/op 7.1654 us/op 0.79
OrderedSet add up to 128 items then delete middle 15.981 us/op 16.855 us/op 0.95
Set add up to 256 items then delete first 9.2219 us/op 13.616 us/op 0.68
OrderedSet add up to 256 items then delete first 13.373 us/op 18.424 us/op 0.73
Set add up to 256 items then delete last 8.0584 us/op 12.841 us/op 0.63
OrderedSet add up to 256 items then delete last 12.945 us/op 18.819 us/op 0.69
Set add up to 256 items then delete middle 8.1947 us/op 12.536 us/op 0.65
OrderedSet add up to 256 items then delete middle 42.830 us/op 45.839 us/op 0.93
transfer serialized Status (84 B) 1.6500 us/op 1.5960 us/op 1.03
copy serialized Status (84 B) 1.4980 us/op 1.3820 us/op 1.08
transfer serialized SignedVoluntaryExit (112 B) 1.5300 us/op 1.7600 us/op 0.87
copy serialized SignedVoluntaryExit (112 B) 1.3620 us/op 1.4870 us/op 0.92
transfer serialized ProposerSlashing (416 B) 2.0950 us/op 2.5710 us/op 0.81
copy serialized ProposerSlashing (416 B) 2.2540 us/op 2.5830 us/op 0.87
transfer serialized Attestation (485 B) 1.6700 us/op 2.0250 us/op 0.82
copy serialized Attestation (485 B) 1.5440 us/op 2.0930 us/op 0.74
transfer serialized AttesterSlashing (33232 B) 2.1120 us/op 2.6180 us/op 0.81
copy serialized AttesterSlashing (33232 B) 4.1480 us/op 7.8560 us/op 0.53
transfer serialized Small SignedBeaconBlock (128000 B) 2.0940 us/op 2.9480 us/op 0.71
copy serialized Small SignedBeaconBlock (128000 B) 9.1630 us/op 20.067 us/op 0.46
transfer serialized Avg SignedBeaconBlock (200000 B) 2.2940 us/op 3.1420 us/op 0.73
copy serialized Avg SignedBeaconBlock (200000 B) 14.259 us/op 26.979 us/op 0.53
transfer serialized BlobsSidecar (524380 B) 2.8960 us/op 3.8530 us/op 0.75
copy serialized BlobsSidecar (524380 B) 77.291 us/op 93.550 us/op 0.83
transfer serialized Big SignedBeaconBlock (1000000 B) 3.4690 us/op 3.6750 us/op 0.94
copy serialized Big SignedBeaconBlock (1000000 B) 148.43 us/op 174.92 us/op 0.85
pass gossip attestations to forkchoice per slot 2.8314 ms/op 4.3228 ms/op 0.65
forkChoice updateHead vc 100000 bc 64 eq 0 667.13 us/op 552.69 us/op 1.21
forkChoice updateHead vc 600000 bc 64 eq 0 2.7083 ms/op 3.6831 ms/op 0.74
forkChoice updateHead vc 1000000 bc 64 eq 0 4.3592 ms/op 5.8986 ms/op 0.74
forkChoice updateHead vc 600000 bc 320 eq 0 2.4781 ms/op 3.5521 ms/op 0.70
forkChoice updateHead vc 600000 bc 1200 eq 0 2.6487 ms/op 3.7376 ms/op 0.71
forkChoice updateHead vc 600000 bc 7200 eq 0 3.8064 ms/op 5.1100 ms/op 0.74
forkChoice updateHead vc 600000 bc 64 eq 1000 9.5384 ms/op 11.996 ms/op 0.80
forkChoice updateHead vc 600000 bc 64 eq 10000 9.4541 ms/op 12.049 ms/op 0.78
forkChoice updateHead vc 600000 bc 64 eq 300000 13.087 ms/op 26.092 ms/op 0.50
computeDeltas 500000 validators 300 proto nodes 3.1501 ms/op 4.4642 ms/op 0.71
computeDeltas 500000 validators 1200 proto nodes 3.0986 ms/op 4.7340 ms/op 0.65
computeDeltas 500000 validators 7200 proto nodes 3.0717 ms/op 4.7066 ms/op 0.65
computeDeltas 750000 validators 300 proto nodes 4.8760 ms/op 7.0706 ms/op 0.69
computeDeltas 750000 validators 1200 proto nodes 4.5677 ms/op 6.9498 ms/op 0.66
computeDeltas 750000 validators 7200 proto nodes 4.5295 ms/op 6.6924 ms/op 0.68
computeDeltas 1400000 validators 300 proto nodes 8.4264 ms/op 11.794 ms/op 0.71
computeDeltas 1400000 validators 1200 proto nodes 8.2773 ms/op 12.814 ms/op 0.65
computeDeltas 1400000 validators 7200 proto nodes 8.4308 ms/op 12.644 ms/op 0.67
computeDeltas 2100000 validators 300 proto nodes 12.989 ms/op 18.243 ms/op 0.71
computeDeltas 2100000 validators 1200 proto nodes 12.709 ms/op 17.998 ms/op 0.71
computeDeltas 2100000 validators 7200 proto nodes 12.753 ms/op 17.043 ms/op 0.75
altair processAttestation - 250000 vs - 7PWei normalcase 1.5847 ms/op 1.9023 ms/op 0.83
altair processAttestation - 250000 vs - 7PWei worstcase 2.4142 ms/op 2.9886 ms/op 0.81
altair processAttestation - setStatus - 1/6 committees join 70.697 us/op 107.31 us/op 0.66
altair processAttestation - setStatus - 1/3 committees join 142.99 us/op 206.95 us/op 0.69
altair processAttestation - setStatus - 1/2 committees join 210.64 us/op 296.64 us/op 0.71
altair processAttestation - setStatus - 2/3 committees join 268.14 us/op 363.44 us/op 0.74
altair processAttestation - setStatus - 4/5 committees join 384.12 us/op 522.69 us/op 0.73
altair processAttestation - setStatus - 100% committees join 474.99 us/op 639.97 us/op 0.74
altair processBlock - 250000 vs - 7PWei normalcase 4.0211 ms/op 4.8364 ms/op 0.83
altair processBlock - 250000 vs - 7PWei normalcase hashState 23.323 ms/op 25.969 ms/op 0.90
altair processBlock - 250000 vs - 7PWei worstcase 38.926 ms/op 44.672 ms/op 0.87
altair processBlock - 250000 vs - 7PWei worstcase hashState 73.055 ms/op 86.142 ms/op 0.85
phase0 processBlock - 250000 vs - 7PWei normalcase 1.9157 ms/op 2.3151 ms/op 0.83
phase0 processBlock - 250000 vs - 7PWei worstcase 24.826 ms/op 33.470 ms/op 0.74
altair processEth1Data - 250000 vs - 7PWei normalcase 253.52 us/op 380.85 us/op 0.67
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 5.8310 us/op 6.3780 us/op 0.91
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 17.759 us/op 26.047 us/op 0.68
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 7.0420 us/op 9.1580 us/op 0.77
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 6.0150 us/op 8.0830 us/op 0.74
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 75.357 us/op 116.38 us/op 0.65
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 761.29 us/op 891.75 us/op 0.85
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.0417 ms/op 1.0329 ms/op 1.01
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.0955 ms/op 1.1960 ms/op 0.92
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 2.0105 ms/op 2.5952 ms/op 0.77
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.0932 ms/op 1.9779 ms/op 0.55
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 2.9513 ms/op 3.9840 ms/op 0.74
Tree 40 250000 create 195.26 ms/op 247.19 ms/op 0.79
Tree 40 250000 get(125000) 123.17 ns/op 165.49 ns/op 0.74
Tree 40 250000 set(125000) 521.76 ns/op 715.58 ns/op 0.73
Tree 40 250000 toArray() 10.000 ms/op 17.925 ms/op 0.56
Tree 40 250000 iterate all - toArray() + loop 10.081 ms/op 17.789 ms/op 0.57
Tree 40 250000 iterate all - get(i) 41.858 ms/op 56.845 ms/op 0.74
MutableVector 250000 create 8.8760 ms/op 9.7648 ms/op 0.91
MutableVector 250000 get(125000) 5.8110 ns/op 6.3220 ns/op 0.92
MutableVector 250000 set(125000) 171.17 ns/op 210.45 ns/op 0.81
MutableVector 250000 toArray() 3.7220 ms/op 3.7321 ms/op 1.00
MutableVector 250000 iterate all - toArray() + loop 4.1987 ms/op 3.9883 ms/op 1.05
MutableVector 250000 iterate all - get(i) 1.5522 ms/op 1.7620 ms/op 0.88
Array 250000 create 2.3520 ms/op 3.6402 ms/op 0.65
Array 250000 clone - spread 1.3392 ms/op 1.5348 ms/op 0.87
Array 250000 get(125000) 0.59300 ns/op 0.44500 ns/op 1.33
Array 250000 set(125000) 0.58900 ns/op 0.46400 ns/op 1.27
Array 250000 iterate all - loop 77.819 us/op 96.782 us/op 0.80
effectiveBalanceIncrements clone Uint8Array 300000 14.900 us/op 40.676 us/op 0.37
effectiveBalanceIncrements clone MutableVector 300000 311.00 ns/op 136.00 ns/op 2.29
effectiveBalanceIncrements rw all Uint8Array 300000 165.98 us/op 202.69 us/op 0.82
effectiveBalanceIncrements rw all MutableVector 300000 59.595 ms/op 63.801 ms/op 0.93
phase0 afterProcessEpoch - 250000 vs - 7PWei 75.369 ms/op 94.010 ms/op 0.80
phase0 beforeProcessEpoch - 250000 vs - 7PWei 39.366 ms/op 47.065 ms/op 0.84
altair processEpoch - mainnet_e81889 338.66 ms/op 406.99 ms/op 0.83
mainnet_e81889 - altair beforeProcessEpoch 59.037 ms/op 71.526 ms/op 0.83
mainnet_e81889 - altair processJustificationAndFinalization 11.506 us/op 20.334 us/op 0.57
mainnet_e81889 - altair processInactivityUpdates 6.1705 ms/op 6.9004 ms/op 0.89
mainnet_e81889 - altair processRewardsAndPenalties 52.382 ms/op 49.519 ms/op 1.06
mainnet_e81889 - altair processRegistryUpdates 2.3290 us/op 3.0540 us/op 0.76
mainnet_e81889 - altair processSlashings 766.00 ns/op 510.00 ns/op 1.50
mainnet_e81889 - altair processEth1DataReset 715.00 ns/op 499.00 ns/op 1.43
mainnet_e81889 - altair processEffectiveBalanceUpdates 884.40 us/op 1.4482 ms/op 0.61
mainnet_e81889 - altair processSlashingsReset 2.6640 us/op 3.8750 us/op 0.69
mainnet_e81889 - altair processRandaoMixesReset 2.8730 us/op 4.4940 us/op 0.64
mainnet_e81889 - altair processHistoricalRootsUpdate 726.00 ns/op 535.00 ns/op 1.36
mainnet_e81889 - altair processParticipationFlagUpdates 1.7460 us/op 3.1410 us/op 0.56
mainnet_e81889 - altair processSyncCommitteeUpdates 824.00 ns/op 527.00 ns/op 1.56
mainnet_e81889 - altair afterProcessEpoch 79.363 ms/op 103.14 ms/op 0.77
capella processEpoch - mainnet_e217614 1.2138 s/op 1.5957 s/op 0.76
mainnet_e217614 - capella beforeProcessEpoch 233.98 ms/op 308.29 ms/op 0.76
mainnet_e217614 - capella processJustificationAndFinalization 7.0120 us/op 17.223 us/op 0.41
mainnet_e217614 - capella processInactivityUpdates 14.154 ms/op 20.738 ms/op 0.68
mainnet_e217614 - capella processRewardsAndPenalties 265.00 ms/op 263.60 ms/op 1.01
mainnet_e217614 - capella processRegistryUpdates 6.5240 us/op 18.674 us/op 0.35
mainnet_e217614 - capella processSlashings 430.00 ns/op 635.00 ns/op 0.68
mainnet_e217614 - capella processEth1DataReset 391.00 ns/op 1.2410 us/op 0.32
mainnet_e217614 - capella processEffectiveBalanceUpdates 3.7758 ms/op 5.5554 ms/op 0.68
mainnet_e217614 - capella processSlashingsReset 2.6920 us/op 5.3760 us/op 0.50
mainnet_e217614 - capella processRandaoMixesReset 3.3180 us/op 5.4220 us/op 0.61
mainnet_e217614 - capella processHistoricalRootsUpdate 689.00 ns/op 710.00 ns/op 0.97
mainnet_e217614 - capella processParticipationFlagUpdates 4.4500 us/op 2.3720 us/op 1.88
mainnet_e217614 - capella afterProcessEpoch 195.44 ms/op 311.20 ms/op 0.63
phase0 processEpoch - mainnet_e58758 354.59 ms/op 446.74 ms/op 0.79
mainnet_e58758 - phase0 beforeProcessEpoch 103.01 ms/op 115.94 ms/op 0.89
mainnet_e58758 - phase0 processJustificationAndFinalization 12.277 us/op 21.102 us/op 0.58
mainnet_e58758 - phase0 processRewardsAndPenalties 30.124 ms/op 40.966 ms/op 0.74
mainnet_e58758 - phase0 processRegistryUpdates 6.6830 us/op 9.3390 us/op 0.72
mainnet_e58758 - phase0 processSlashings 728.00 ns/op 404.00 ns/op 1.80
mainnet_e58758 - phase0 processEth1DataReset 651.00 ns/op 423.00 ns/op 1.54
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.0735 ms/op 1.2609 ms/op 0.85
mainnet_e58758 - phase0 processSlashingsReset 2.3740 us/op 4.8910 us/op 0.49
mainnet_e58758 - phase0 processRandaoMixesReset 2.7940 us/op 6.1590 us/op 0.45
mainnet_e58758 - phase0 processHistoricalRootsUpdate 708.00 ns/op 447.00 ns/op 1.58
mainnet_e58758 - phase0 processParticipationRecordUpdates 1.3730 us/op 3.2560 us/op 0.42
mainnet_e58758 - phase0 afterProcessEpoch 66.826 ms/op 83.647 ms/op 0.80
phase0 processEffectiveBalanceUpdates - 250000 normalcase 745.06 us/op 1.5414 ms/op 0.48
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.0102 ms/op 2.3453 ms/op 0.43
altair processInactivityUpdates - 250000 normalcase 18.308 ms/op 20.864 ms/op 0.88
altair processInactivityUpdates - 250000 worstcase 17.156 ms/op 22.420 ms/op 0.77
phase0 processRegistryUpdates - 250000 normalcase 3.3650 us/op 7.9640 us/op 0.42
phase0 processRegistryUpdates - 250000 badcase_full_deposits 293.90 us/op 325.11 us/op 0.90
phase0 processRegistryUpdates - 250000 worstcase 0.5 113.20 ms/op 122.92 ms/op 0.92
altair processRewardsAndPenalties - 250000 normalcase 30.417 ms/op 42.781 ms/op 0.71
altair processRewardsAndPenalties - 250000 worstcase 41.483 ms/op 49.137 ms/op 0.84
phase0 getAttestationDeltas - 250000 normalcase 6.1241 ms/op 7.8760 ms/op 0.78
phase0 getAttestationDeltas - 250000 worstcase 5.8452 ms/op 8.6937 ms/op 0.67
phase0 processSlashings - 250000 worstcase 88.077 us/op 104.97 us/op 0.84
altair processSyncCommitteeUpdates - 250000 97.982 ms/op 128.87 ms/op 0.76
BeaconState.hashTreeRoot - No change 502.00 ns/op 304.00 ns/op 1.65
BeaconState.hashTreeRoot - 1 full validator 105.77 us/op 106.89 us/op 0.99
BeaconState.hashTreeRoot - 32 full validator 1.1059 ms/op 1.5973 ms/op 0.69
BeaconState.hashTreeRoot - 512 full validator 12.135 ms/op 14.429 ms/op 0.84
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 159.20 us/op 156.59 us/op 1.02
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.0013 ms/op 2.2244 ms/op 0.90
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 26.480 ms/op 24.371 ms/op 1.09
BeaconState.hashTreeRoot - 1 balances 94.997 us/op 110.24 us/op 0.86
BeaconState.hashTreeRoot - 32 balances 988.36 us/op 1.3905 ms/op 0.71
BeaconState.hashTreeRoot - 512 balances 8.0650 ms/op 13.120 ms/op 0.61
BeaconState.hashTreeRoot - 250000 balances 166.69 ms/op 216.16 ms/op 0.77
aggregationBits - 2048 els - zipIndexesInBitList 19.189 us/op 33.208 us/op 0.58
byteArrayEquals 32 47.797 ns/op 55.638 ns/op 0.86
Buffer.compare 32 38.468 ns/op 47.129 ns/op 0.82
byteArrayEquals 1024 1.2566 us/op 1.6463 us/op 0.76
Buffer.compare 1024 45.807 ns/op 54.729 ns/op 0.84
byteArrayEquals 16384 19.982 us/op 26.112 us/op 0.77
Buffer.compare 16384 223.42 ns/op 255.50 ns/op 0.87
byteArrayEquals 123687377 150.53 ms/op 208.39 ms/op 0.72
Buffer.compare 123687377 3.7106 ms/op 10.315 ms/op 0.36
byteArrayEquals 32 - diff last byte 47.153 ns/op 57.818 ns/op 0.82
Buffer.compare 32 - diff last byte 38.839 ns/op 52.631 ns/op 0.74
byteArrayEquals 1024 - diff last byte 1.2526 us/op 1.7339 us/op 0.72
Buffer.compare 1024 - diff last byte 45.917 ns/op 61.560 ns/op 0.75
byteArrayEquals 16384 - diff last byte 19.883 us/op 27.545 us/op 0.72
Buffer.compare 16384 - diff last byte 196.55 ns/op 269.01 ns/op 0.73
byteArrayEquals 123687377 - diff last byte 150.09 ms/op 205.76 ms/op 0.73
Buffer.compare 123687377 - diff last byte 3.7655 ms/op 8.3503 ms/op 0.45
byteArrayEquals 32 - random bytes 4.8280 ns/op 5.6130 ns/op 0.86
Buffer.compare 32 - random bytes 40.387 ns/op 51.918 ns/op 0.78
byteArrayEquals 1024 - random bytes 4.8110 ns/op 5.6650 ns/op 0.85
Buffer.compare 1024 - random bytes 39.580 ns/op 48.823 ns/op 0.81
byteArrayEquals 16384 - random bytes 4.8350 ns/op 5.4930 ns/op 0.88
Buffer.compare 16384 - random bytes 39.068 ns/op 48.347 ns/op 0.81
byteArrayEquals 123687377 - random bytes 7.6600 ns/op 6.9700 ns/op 1.10
Buffer.compare 123687377 - random bytes 42.940 ns/op 50.100 ns/op 0.86
regular array get 100000 times 30.554 us/op 34.988 us/op 0.87
wrappedArray get 100000 times 30.563 us/op 35.217 us/op 0.87
arrayWithProxy get 100000 times 9.3420 ms/op 13.684 ms/op 0.68
ssz.Root.equals 43.071 ns/op 48.840 ns/op 0.88
byteArrayEquals 43.046 ns/op 49.143 ns/op 0.88
Buffer.compare 9.9900 ns/op 11.534 ns/op 0.87
shuffle list - 16384 els 5.4652 ms/op 7.1120 ms/op 0.77
shuffle list - 250000 els 81.194 ms/op 104.50 ms/op 0.78
processSlot - 1 slots 12.358 us/op 16.236 us/op 0.76
processSlot - 32 slots 2.9227 ms/op 3.7256 ms/op 0.78
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 44.536 ms/op 41.232 ms/op 1.08
getCommitteeAssignments - req 1 vs - 250000 vc 1.8013 ms/op 2.4224 ms/op 0.74
getCommitteeAssignments - req 100 vs - 250000 vc 3.4971 ms/op 4.6750 ms/op 0.75
getCommitteeAssignments - req 1000 vs - 250000 vc 3.7467 ms/op 5.1554 ms/op 0.73
findModifiedValidators - 10000 modified validators 240.08 ms/op 322.03 ms/op 0.75
findModifiedValidators - 1000 modified validators 148.95 ms/op 209.47 ms/op 0.71
findModifiedValidators - 100 modified validators 139.48 ms/op 194.96 ms/op 0.72
findModifiedValidators - 10 modified validators 139.34 ms/op 225.04 ms/op 0.62
findModifiedValidators - 1 modified validators 132.29 ms/op 206.22 ms/op 0.64
findModifiedValidators - no difference 142.55 ms/op 200.62 ms/op 0.71
compare ViewDUs 3.0358 s/op 3.3137 s/op 0.92
compare each validator Uint8Array 1.6163 s/op 1.4556 s/op 1.11
compare ViewDU to Uint8Array 724.60 ms/op 1.4147 s/op 0.51
migrate state 1000000 validators, 24 modified, 0 new 580.47 ms/op 782.01 ms/op 0.74
migrate state 1000000 validators, 1700 modified, 1000 new 818.62 ms/op 1.1296 s/op 0.72
migrate state 1000000 validators, 3400 modified, 2000 new 1.0009 s/op 1.3834 s/op 0.72
migrate state 1500000 validators, 24 modified, 0 new 582.13 ms/op 843.31 ms/op 0.69
migrate state 1500000 validators, 1700 modified, 1000 new 813.11 ms/op 1.2338 s/op 0.66
migrate state 1500000 validators, 3400 modified, 2000 new 1.0273 s/op 1.5087 s/op 0.68
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 5.8900 ns/op 5.9500 ns/op 0.99
state getBlockRootAtSlot - 250000 vs - 7PWei 764.36 ns/op 780.28 ns/op 0.98
computeProposers - vc 250000 5.4630 ms/op 12.413 ms/op 0.44
computeEpochShuffling - vc 250000 81.068 ms/op 117.84 ms/op 0.69
getNextSyncCommittee - vc 250000 101.51 ms/op 177.98 ms/op 0.57
computeSigningRoot for AttestationData 20.729 us/op 34.210 us/op 0.61
hash AttestationData serialized data then Buffer.toString(base64) 1.1659 us/op 2.0111 us/op 0.58
toHexString serialized data 691.55 ns/op 1.6255 us/op 0.43
Buffer.toString(base64) 132.16 ns/op 259.92 ns/op 0.51

by benchmarkbot/action

@twoeths
Copy link
Contributor Author

twoeths commented Jun 11, 2024

this worked fine on feat1

Screenshot 2024-06-11 at 18 08 12

it takes only 172kb for the last 5 days

@twoeths twoeths marked this pull request as ready for review June 11, 2024 11:14
@twoeths twoeths requested a review from a team as a code owner June 11, 2024 11:14
nflaig
nflaig previously approved these changes Jun 13, 2024
View reviewed changes
Copy link
Member

@nflaig nflaig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

ensi321
ensi321 previously approved these changes Jun 13, 2024
View reviewed changes
ensi321
ensi321 reviewed Jun 13, 2024
View reviewed changes
Copy link
Contributor

@ensi321 ensi321 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@twoeths Looks like these two tests are failing

test/unit/util/pruneOldFilesInDir.test.ts > pruneOldFilesInDir > should delete old files in nested directories
test/unit/util/pruneOldFilesInDir.test.ts > pruneOldFilesInDir > should handle empty directories

Can we fix it?

@nflaig
Copy link
Member

nflaig commented Jun 13, 2024 •
edited
Loading

unit tests are broken because we also delete the directory now if there are no files in it, anyways questionable to me, if we receive an new invalid ssz object the directory has to be created again, downside of just keeping it around is negligible imo as it's part of dataDir and always the same directory

@twoeths twoeths dismissed stale reviews from ensi321 and nflaig via c980c1a June 19, 2024 04:29
@nflaig nflaig enabled auto-merge (squash) June 19, 2024 07:50
@nflaig nflaig merged commit 85dc0ba into unstable Jun 19, 2024
19 of 20 checks passed
@nflaig nflaig deleted the te/prune_invalid_ssz_objects branch June 19, 2024 08:01
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.20.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ensi321 ensi321 ensi321 left review comments

@nflaig nflaig nflaig approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@twoeths @nflaig @wemeetagain @ensi321