Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Get the capella builder functional #5179

Merged
merged 1 commit into from
Feb 20, 2023
Merged

Get the capella builder functional #5179

merged 1 commit into from
Feb 20, 2023

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Feb 20, 2023

Capella builder was failing because of this extra check, however it works once this check was removed,

https://zhejiang.beaconcha.in/slot/135601
proposed by the lodestar post this fix

@g11tech g11tech requested a review from a team as a code owner February 20, 2023 11:03
@github-actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 06a875d Previous: 5396d1f Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 565.21 us/op 547.97 us/op 1.03
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 51.803 us/op 47.942 us/op 1.08
BLS verify - blst-native 1.2439 ms/op 1.2477 ms/op 1.00
BLS verifyMultipleSignatures 3 - blst-native 2.5306 ms/op 2.6445 ms/op 0.96
BLS verifyMultipleSignatures 8 - blst-native 5.4170 ms/op 5.5371 ms/op 0.98
BLS verifyMultipleSignatures 32 - blst-native 19.396 ms/op 19.771 ms/op 0.98
BLS aggregatePubkeys 32 - blst-native 25.842 us/op 27.482 us/op 0.94
BLS aggregatePubkeys 128 - blst-native 101.47 us/op 104.25 us/op 0.97
getAttestationsForBlock 61.144 ms/op 58.545 ms/op 1.04
isKnown best case - 1 super set check 276.00 ns/op 268.00 ns/op 1.03
isKnown normal case - 2 super set checks 274.00 ns/op 248.00 ns/op 1.10
isKnown worse case - 16 super set checks 266.00 ns/op 260.00 ns/op 1.02
CheckpointStateCache - add get delete 5.8550 us/op 5.2600 us/op 1.11
validate gossip signedAggregateAndProof - struct 2.8230 ms/op 2.7954 ms/op 1.01
validate gossip attestation - struct 1.3670 ms/op 1.3389 ms/op 1.02
pickEth1Vote - no votes 1.3592 ms/op 1.2589 ms/op 1.08
pickEth1Vote - max votes 12.366 ms/op 8.7731 ms/op 1.41
pickEth1Vote - Eth1Data hashTreeRoot value x2048 9.9483 ms/op 8.6634 ms/op 1.15
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 15.435 ms/op 15.707 ms/op 0.98
pickEth1Vote - Eth1Data fastSerialize value x2048 736.50 us/op 721.36 us/op 1.02
pickEth1Vote - Eth1Data fastSerialize tree x2048 5.4643 ms/op 5.0974 ms/op 1.07
bytes32 toHexString 548.00 ns/op 560.00 ns/op 0.98
bytes32 Buffer.toString(hex) 399.00 ns/op 401.00 ns/op 1.00
bytes32 Buffer.toString(hex) from Uint8Array 578.00 ns/op 634.00 ns/op 0.91
bytes32 Buffer.toString(hex) + 0x 347.00 ns/op 412.00 ns/op 0.84
Object access 1 prop 0.16400 ns/op 0.19400 ns/op 0.85
Map access 1 prop 0.17100 ns/op 0.16100 ns/op 1.06
Object get x1000 7.4320 ns/op 6.6660 ns/op 1.11
Map get x1000 0.62200 ns/op 0.56500 ns/op 1.10
Object set x1000 52.702 ns/op 62.372 ns/op 0.84
Map set x1000 43.819 ns/op 49.104 ns/op 0.89
Return object 10000 times 0.23790 ns/op 0.24150 ns/op 0.99
Throw Error 10000 times 4.2083 us/op 4.2421 us/op 0.99
fastMsgIdFn sha256 / 200 bytes 3.6400 us/op 3.5990 us/op 1.01
fastMsgIdFn h32 xxhash / 200 bytes 301.00 ns/op 310.00 ns/op 0.97
fastMsgIdFn h64 xxhash / 200 bytes 424.00 ns/op 445.00 ns/op 0.95
fastMsgIdFn sha256 / 1000 bytes 11.853 us/op 11.805 us/op 1.00
fastMsgIdFn h32 xxhash / 1000 bytes 438.00 ns/op 453.00 ns/op 0.97
fastMsgIdFn h64 xxhash / 1000 bytes 491.00 ns/op 519.00 ns/op 0.95
fastMsgIdFn sha256 / 10000 bytes 106.57 us/op 104.47 us/op 1.02
fastMsgIdFn h32 xxhash / 10000 bytes 1.9770 us/op 1.9990 us/op 0.99
fastMsgIdFn h64 xxhash / 10000 bytes 1.4340 us/op 1.4540 us/op 0.99
enrSubnets - fastDeserialize 64 bits 1.5780 us/op 1.6260 us/op 0.97
enrSubnets - ssz BitVector 64 bits 565.00 ns/op 601.00 ns/op 0.94
enrSubnets - fastDeserialize 4 bits 184.00 ns/op 180.00 ns/op 1.02
enrSubnets - ssz BitVector 4 bits 551.00 ns/op 491.00 ns/op 1.12
prioritizePeers score -10:0 att 32-0.1 sync 2-0 95.940 us/op 99.194 us/op 0.97
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 125.07 us/op 126.08 us/op 0.99
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 182.07 us/op 225.37 us/op 0.81
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 324.19 us/op 433.43 us/op 0.75
prioritizePeers score 0:0 att 64-1 sync 4-1 370.63 us/op 590.43 us/op 0.63
array of 16000 items push then shift 1.6725 us/op 2.4263 us/op 0.69
LinkedList of 16000 items push then shift 8.9520 ns/op 13.602 ns/op 0.66
array of 16000 items push then pop 97.693 ns/op 140.87 ns/op 0.69
LinkedList of 16000 items push then pop 8.5970 ns/op 12.736 ns/op 0.68
array of 24000 items push then shift 2.3938 us/op 4.0578 us/op 0.59
LinkedList of 24000 items push then shift 9.0990 ns/op 11.681 ns/op 0.78
array of 24000 items push then pop 78.630 ns/op 128.93 ns/op 0.61
LinkedList of 24000 items push then pop 8.5740 ns/op 11.932 ns/op 0.72
intersect bitArray bitLen 8 13.255 ns/op 22.990 ns/op 0.58
intersect array and set length 8 79.358 ns/op 85.260 ns/op 0.93
intersect bitArray bitLen 128 44.005 ns/op 45.385 ns/op 0.97
intersect array and set length 128 1.0672 us/op 1.2504 us/op 0.85
Buffer.concat 32 items 2.6860 us/op 2.8380 us/op 0.95
Uint8Array.set 32 items 2.1750 us/op 2.4650 us/op 0.88
pass gossip attestations to forkchoice per slot 4.0331 ms/op 2.3917 ms/op 1.69
computeDeltas 2.9877 ms/op 3.1230 ms/op 0.96
computeProposerBoostScoreFromBalances 1.7970 ms/op 1.8272 ms/op 0.98
altair processAttestation - 250000 vs - 7PWei normalcase 2.1712 ms/op 2.3020 ms/op 0.94
altair processAttestation - 250000 vs - 7PWei worstcase 3.5477 ms/op 3.3980 ms/op 1.04
altair processAttestation - setStatus - 1/6 committees join 140.25 us/op 143.57 us/op 0.98
altair processAttestation - setStatus - 1/3 committees join 276.82 us/op 276.48 us/op 1.00
altair processAttestation - setStatus - 1/2 committees join 375.43 us/op 369.79 us/op 1.02
altair processAttestation - setStatus - 2/3 committees join 471.50 us/op 466.32 us/op 1.01
altair processAttestation - setStatus - 4/5 committees join 659.47 us/op 653.57 us/op 1.01
altair processAttestation - setStatus - 100% committees join 778.94 us/op 767.37 us/op 1.02
altair processBlock - 250000 vs - 7PWei normalcase 16.036 ms/op 20.925 ms/op 0.77
altair processBlock - 250000 vs - 7PWei normalcase hashState 27.598 ms/op 27.413 ms/op 1.01
altair processBlock - 250000 vs - 7PWei worstcase 47.306 ms/op 46.664 ms/op 1.01
altair processBlock - 250000 vs - 7PWei worstcase hashState 66.169 ms/op 70.602 ms/op 0.94
phase0 processBlock - 250000 vs - 7PWei normalcase 2.0939 ms/op 1.9295 ms/op 1.09
phase0 processBlock - 250000 vs - 7PWei worstcase 27.870 ms/op 28.021 ms/op 0.99
altair processEth1Data - 250000 vs - 7PWei normalcase 438.46 us/op 468.46 us/op 0.94
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 6.7550 us/op 7.1680 us/op 0.94
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 19.377 us/op 34.102 us/op 0.57
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 8.2380 us/op 14.924 us/op 0.55
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 6.7610 us/op 12.633 us/op 0.54
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 75.550 us/op 120.73 us/op 0.63
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 629.74 us/op 696.64 us/op 0.90
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 900.25 us/op 1.0383 ms/op 0.87
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 880.01 us/op 918.45 us/op 0.96
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 2.1783 ms/op 2.7196 ms/op 0.80
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 1.5293 ms/op 1.7447 ms/op 0.88
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 3.7891 ms/op 4.1177 ms/op 0.92
Tree 40 250000 create 316.79 ms/op 472.11 ms/op 0.67
Tree 40 250000 get(125000) 188.72 ns/op 201.60 ns/op 0.94
Tree 40 250000 set(125000) 975.01 ns/op 1.1367 us/op 0.86
Tree 40 250000 toArray() 21.378 ms/op 23.332 ms/op 0.92
Tree 40 250000 iterate all - toArray() + loop 20.977 ms/op 22.891 ms/op 0.92
Tree 40 250000 iterate all - get(i) 74.716 ms/op 77.639 ms/op 0.96
MutableVector 250000 create 10.627 ms/op 13.969 ms/op 0.76
MutableVector 250000 get(125000) 6.1030 ns/op 6.6640 ns/op 0.92
MutableVector 250000 set(125000) 241.86 ns/op 393.34 ns/op 0.61
MutableVector 250000 toArray() 2.6814 ms/op 3.6263 ms/op 0.74
MutableVector 250000 iterate all - toArray() + loop 2.8389 ms/op 3.8407 ms/op 0.74
MutableVector 250000 iterate all - get(i) 1.4602 ms/op 1.5805 ms/op 0.92
Array 250000 create 2.4605 ms/op 3.3054 ms/op 0.74
Array 250000 clone - spread 1.1550 ms/op 1.1679 ms/op 0.99
Array 250000 get(125000) 0.54700 ns/op 0.65700 ns/op 0.83
Array 250000 set(125000) 0.63300 ns/op 0.86700 ns/op 0.73
Array 250000 iterate all - loop 78.904 us/op 99.447 us/op 0.79
effectiveBalanceIncrements clone Uint8Array 300000 25.388 us/op 46.122 us/op 0.55
effectiveBalanceIncrements clone MutableVector 300000 361.00 ns/op 391.00 ns/op 0.92
effectiveBalanceIncrements rw all Uint8Array 300000 161.10 us/op 184.32 us/op 0.87
effectiveBalanceIncrements rw all MutableVector 300000 78.738 ms/op 108.23 ms/op 0.73
phase0 afterProcessEpoch - 250000 vs - 7PWei 110.65 ms/op 128.97 ms/op 0.86
phase0 beforeProcessEpoch - 250000 vs - 7PWei 41.543 ms/op 48.462 ms/op 0.86
altair processEpoch - mainnet_e81889 324.10 ms/op 377.28 ms/op 0.86
mainnet_e81889 - altair beforeProcessEpoch 61.850 ms/op 75.949 ms/op 0.81
mainnet_e81889 - altair processJustificationAndFinalization 16.964 us/op 24.120 us/op 0.70
mainnet_e81889 - altair processInactivityUpdates 5.3994 ms/op 6.7647 ms/op 0.80
mainnet_e81889 - altair processRewardsAndPenalties 69.543 ms/op 53.762 ms/op 1.29
mainnet_e81889 - altair processRegistryUpdates 2.7460 us/op 5.2090 us/op 0.53
mainnet_e81889 - altair processSlashings 449.00 ns/op 883.00 ns/op 0.51
mainnet_e81889 - altair processEth1DataReset 539.00 ns/op 1.2980 us/op 0.42
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.1649 ms/op 1.3209 ms/op 0.88
mainnet_e81889 - altair processSlashingsReset 4.8660 us/op 8.1170 us/op 0.60
mainnet_e81889 - altair processRandaoMixesReset 4.3180 us/op 8.4440 us/op 0.51
mainnet_e81889 - altair processHistoricalRootsUpdate 587.00 ns/op 1.2270 us/op 0.48
mainnet_e81889 - altair processParticipationFlagUpdates 2.2620 us/op 5.5220 us/op 0.41
mainnet_e81889 - altair processSyncCommitteeUpdates 504.00 ns/op 821.00 ns/op 0.61
mainnet_e81889 - altair afterProcessEpoch 114.62 ms/op 150.55 ms/op 0.76
phase0 processEpoch - mainnet_e58758 315.94 ms/op 424.20 ms/op 0.74
mainnet_e58758 - phase0 beforeProcessEpoch 121.62 ms/op 173.41 ms/op 0.70
mainnet_e58758 - phase0 processJustificationAndFinalization 15.399 us/op 33.918 us/op 0.45
mainnet_e58758 - phase0 processRewardsAndPenalties 53.725 ms/op 64.245 ms/op 0.84
mainnet_e58758 - phase0 processRegistryUpdates 7.3350 us/op 14.315 us/op 0.51
mainnet_e58758 - phase0 processSlashings 529.00 ns/op 1.0890 us/op 0.49
mainnet_e58758 - phase0 processEth1DataReset 554.00 ns/op 828.00 ns/op 0.67
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 944.39 us/op 1.4798 ms/op 0.64
mainnet_e58758 - phase0 processSlashingsReset 2.7880 us/op 5.1760 us/op 0.54
mainnet_e58758 - phase0 processRandaoMixesReset 4.4790 us/op 13.244 us/op 0.34
mainnet_e58758 - phase0 processHistoricalRootsUpdate 607.00 ns/op 1.1050 us/op 0.55
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.7810 us/op 9.1410 us/op 0.41
mainnet_e58758 - phase0 afterProcessEpoch 95.690 ms/op 101.55 ms/op 0.94
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.1616 ms/op 1.4200 ms/op 0.82
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.4013 ms/op 1.8804 ms/op 0.75
altair processInactivityUpdates - 250000 normalcase 19.536 ms/op 23.239 ms/op 0.84
altair processInactivityUpdates - 250000 worstcase 23.001 ms/op 24.625 ms/op 0.93
phase0 processRegistryUpdates - 250000 normalcase 6.3340 us/op 9.8990 us/op 0.64
phase0 processRegistryUpdates - 250000 badcase_full_deposits 246.13 us/op 312.82 us/op 0.79
phase0 processRegistryUpdates - 250000 worstcase 0.5 114.15 ms/op 134.55 ms/op 0.85
altair processRewardsAndPenalties - 250000 normalcase 46.736 ms/op 65.147 ms/op 0.72
altair processRewardsAndPenalties - 250000 worstcase 58.711 ms/op 72.036 ms/op 0.82
phase0 getAttestationDeltas - 250000 normalcase 6.8692 ms/op 7.2587 ms/op 0.95
phase0 getAttestationDeltas - 250000 worstcase 6.7473 ms/op 9.4264 ms/op 0.72
phase0 processSlashings - 250000 worstcase 3.6807 ms/op 4.4091 ms/op 0.83
altair processSyncCommitteeUpdates - 250000 173.38 ms/op 200.77 ms/op 0.86
BeaconState.hashTreeRoot - No change 255.00 ns/op 331.00 ns/op 0.77
BeaconState.hashTreeRoot - 1 full validator 50.781 us/op 55.860 us/op 0.91
BeaconState.hashTreeRoot - 32 full validator 507.62 us/op 592.45 us/op 0.86
BeaconState.hashTreeRoot - 512 full validator 5.3050 ms/op 5.8117 ms/op 0.91
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 62.618 us/op 72.947 us/op 0.86
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 869.39 us/op 949.84 us/op 0.92
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 11.126 ms/op 12.783 ms/op 0.87
BeaconState.hashTreeRoot - 1 balances 48.431 us/op 54.835 us/op 0.88
BeaconState.hashTreeRoot - 32 balances 443.64 us/op 483.92 us/op 0.92
BeaconState.hashTreeRoot - 512 balances 4.5447 ms/op 4.7700 ms/op 0.95
BeaconState.hashTreeRoot - 250000 balances 74.938 ms/op 75.571 ms/op 0.99
aggregationBits - 2048 els - zipIndexesInBitList 14.850 us/op 19.561 us/op 0.76
regular array get 100000 times 31.392 us/op 34.002 us/op 0.92
wrappedArray get 100000 times 30.964 us/op 34.720 us/op 0.89
arrayWithProxy get 100000 times 14.815 ms/op 17.401 ms/op 0.85
ssz.Root.equals 522.00 ns/op 597.00 ns/op 0.87
byteArrayEquals 512.00 ns/op 591.00 ns/op 0.87
shuffle list - 16384 els 6.5357 ms/op 7.3056 ms/op 0.89
shuffle list - 250000 els 95.678 ms/op 105.64 ms/op 0.91
processSlot - 1 slots 8.5080 us/op 10.263 us/op 0.83
processSlot - 32 slots 1.3148 ms/op 1.4388 ms/op 0.91
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 190.08 us/op 219.02 us/op 0.87
getCommitteeAssignments - req 1 vs - 250000 vc 2.8923 ms/op 3.0026 ms/op 0.96
getCommitteeAssignments - req 100 vs - 250000 vc 4.1454 ms/op 4.3268 ms/op 0.96
getCommitteeAssignments - req 1000 vs - 250000 vc 4.4333 ms/op 4.6538 ms/op 0.95
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.4300 ns/op 5.4300 ns/op 0.82
state getBlockRootAtSlot - 250000 vs - 7PWei 932.82 ns/op 667.85 ns/op 1.40
computeProposers - vc 250000 10.168 ms/op 11.539 ms/op 0.88
computeEpochShuffling - vc 250000 100.53 ms/op 107.40 ms/op 0.94
getNextSyncCommittee - vc 250000 167.45 ms/op 196.99 ms/op 0.85

by benchmarkbot/action

@g11tech g11tech enabled auto-merge (squash) February 20, 2023 11:20
@philknows philknows added this to the v1.5.0 milestone Feb 20, 2023
@g11tech g11tech disabled auto-merge February 20, 2023 14:10
@g11tech g11tech enabled auto-merge (squash) February 20, 2023 15:41
@g11tech g11tech merged commit 1adb6e1 into unstable Feb 20, 2023
@g11tech g11tech deleted the g11tech/capella-builder branch February 20, 2023 15:50
@marioevz marioevz mentioned this pull request Feb 20, 2023
Merged
wemeetagain pushed a commit that referenced this pull request Feb 27, 2023
@g11tech @wemeetagain
Get the capella builder functional (#5179)
080b164
dapplion added a commit that referenced this pull request Feb 28, 2023
@dapplion @wemeetagain
@g11tech @nflaig
Merge stable to unstable no conflicts (#5216)
d4b9201 
* v1.5.0

* Add validatior option to specify builder block selection strategy (#5064)

* Add validatior option to specify builder block selection strategy

* cleanup and improvements

* server api fx

* fix test

* fix the testcase

* Re-gossip bls changes if submitted earlier than capella (#5049)

* Cache and retransmit bls changes if submitted early

* regossip when synced on/post capella

* fix tests

* modify publish vs cache condition

* add logging

* add error logging

* batchify the processing

* fix some of the cases

* fix log issues

* also delete included keys

* fix lint

* simplify gossip loop

* fix build

* refac to a versioned bls change store

* update

* logging improvements

* improve logging and finish behavior

* cleanup

* Rename eip4844 fork as Deneb (#5082)

* Fix and extract payload to header conversion (#5101)

* Remove unused eslint disable directives (#5046)

* Add execution payload header to light client header capella onwards (#5027)

* Add execution payload header to light client header capella onwards

* add more forks

* add allfork types

* make other types multifork

* make events multifork compatible

* make lightclient routes multifork

* update light-client transport

* upgrade the blocktype on lightclient server import block

* fixes

* fix test

* fix event serialization/deserilization type

* update proof constants

* fix payload index

* update the light client update generation to multi fork

* fix types

* add spec override to pass specs

* make event types allforks

* make req/resp multifork

* make types multifork throughout

* fix the update serialize/deserialize

* fix error log

* better fork determination for update

* fix the fork of finalized

* fix tests

* fix header upgradation

* Fix zero finalized header in lightclient update gen (#5121)

* Schedule capella on sepolia (#5127)

* Get the capella builder functional (#5179)

* Allow blsChange reprocess/republish via API submission (#5185)

* Fix gossiping blsChange on submit if post capella

* comment improvement

* Use new db bucket for light client updates (#5205)

---------

Co-authored-by: Cayman <caymannava@gmail.com>
Co-authored-by: g11tech <develop@g11tech.io>
Co-authored-by: Nico Flaig <nflaig@protonmail.com>
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.6.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@twoeths twoeths twoeths approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v1.5.0
Development

Successfully merging this pull request may close these issues.
4 participants
@g11tech @wemeetagain @twoeths @philknows