Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow blsChange reprocess/republish via API submission #5185

Merged
merged 2 commits into from
Feb 20, 2023
Merged

Allow blsChange reprocess/republish via API submission #5185

merged 2 commits into from
Feb 20, 2023

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Feb 20, 2023 •
edited by wemeetagain
Loading

Always process then publish a blsChange submitted through the API -- even if it's already in our local pool.
This helps UX of getting blsChanges not picked up by the network to be picked up.

@g11tech g11tech requested a review from a team as a code owner February 20, 2023 17:05
@github-actions
Copy link
Contributor

github-actions bot commented Feb 20, 2023 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: bea1aaa Previous: d1cddb7 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 624.96 us/op 921.33 us/op 0.68
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 59.739 us/op 44.953 us/op 1.33
BLS verify - blst-native 1.3480 ms/op 1.1882 ms/op 1.13
BLS verifyMultipleSignatures 3 - blst-native 2.7041 ms/op 2.4612 ms/op 1.10
BLS verifyMultipleSignatures 8 - blst-native 5.9310 ms/op 5.2841 ms/op 1.12
BLS verifyMultipleSignatures 32 - blst-native 21.219 ms/op 19.071 ms/op 1.11
BLS aggregatePubkeys 32 - blst-native 28.164 us/op 25.568 us/op 1.10
BLS aggregatePubkeys 128 - blst-native 107.40 us/op 99.603 us/op 1.08
getAttestationsForBlock 71.821 ms/op 52.476 ms/op 1.37
isKnown best case - 1 super set check 331.00 ns/op 264.00 ns/op 1.25
isKnown normal case - 2 super set checks 311.00 ns/op 259.00 ns/op 1.20
isKnown worse case - 16 super set checks 356.00 ns/op 257.00 ns/op 1.39
CheckpointStateCache - add get delete 6.4950 us/op 5.2510 us/op 1.24
validate gossip signedAggregateAndProof - struct 3.1165 ms/op 2.7454 ms/op 1.14
validate gossip attestation - struct 1.4901 ms/op 1.2943 ms/op 1.15
pickEth1Vote - no votes 1.5066 ms/op 1.2330 ms/op 1.22
pickEth1Vote - max votes 11.676 ms/op 10.586 ms/op 1.10
pickEth1Vote - Eth1Data hashTreeRoot value x2048 10.553 ms/op 8.9057 ms/op 1.18
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 17.267 ms/op 14.668 ms/op 1.18
pickEth1Vote - Eth1Data fastSerialize value x2048 796.14 us/op 628.88 us/op 1.27
pickEth1Vote - Eth1Data fastSerialize tree x2048 6.5508 ms/op 7.5048 ms/op 0.87
bytes32 toHexString 738.00 ns/op 488.00 ns/op 1.51
bytes32 Buffer.toString(hex) 473.00 ns/op 340.00 ns/op 1.39
bytes32 Buffer.toString(hex) from Uint8Array 691.00 ns/op 552.00 ns/op 1.25
bytes32 Buffer.toString(hex) + 0x 469.00 ns/op 334.00 ns/op 1.40
Object access 1 prop 0.20500 ns/op 0.16400 ns/op 1.25
Map access 1 prop 0.18000 ns/op 0.15800 ns/op 1.14
Object get x1000 7.7570 ns/op 6.2400 ns/op 1.24
Map get x1000 0.77200 ns/op 0.53600 ns/op 1.44
Object set x1000 76.001 ns/op 52.662 ns/op 1.44
Map set x1000 56.735 ns/op 42.497 ns/op 1.34
Return object 10000 times 0.25970 ns/op 0.23590 ns/op 1.10
Throw Error 10000 times 4.4602 us/op 4.1765 us/op 1.07
fastMsgIdFn sha256 / 200 bytes 3.8360 us/op 3.5900 us/op 1.07
fastMsgIdFn h32 xxhash / 200 bytes 324.00 ns/op 281.00 ns/op 1.15
fastMsgIdFn h64 xxhash / 200 bytes 498.00 ns/op 381.00 ns/op 1.31
fastMsgIdFn sha256 / 1000 bytes 12.481 us/op 11.423 us/op 1.09
fastMsgIdFn h32 xxhash / 1000 bytes 469.00 ns/op 403.00 ns/op 1.16
fastMsgIdFn h64 xxhash / 1000 bytes 573.00 ns/op 450.00 ns/op 1.27
fastMsgIdFn sha256 / 10000 bytes 114.48 us/op 103.19 us/op 1.11
fastMsgIdFn h32 xxhash / 10000 bytes 2.2450 us/op 1.8740 us/op 1.20
fastMsgIdFn h64 xxhash / 10000 bytes 1.5890 us/op 1.3300 us/op 1.19
enrSubnets - fastDeserialize 64 bits 1.8710 us/op 1.2560 us/op 1.49
enrSubnets - ssz BitVector 64 bits 671.00 ns/op 477.00 ns/op 1.41
enrSubnets - fastDeserialize 4 bits 235.00 ns/op 170.00 ns/op 1.38
enrSubnets - ssz BitVector 4 bits 712.00 ns/op 483.00 ns/op 1.47
prioritizePeers score -10:0 att 32-0.1 sync 2-0 107.16 us/op 92.062 us/op 1.16
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 158.61 us/op 120.85 us/op 1.31
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 218.73 us/op 170.80 us/op 1.28
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 371.88 us/op 330.79 us/op 1.12
prioritizePeers score 0:0 att 64-1 sync 4-1 409.37 us/op 375.61 us/op 1.09
array of 16000 items push then shift 1.8422 us/op 1.6778 us/op 1.10
LinkedList of 16000 items push then shift 9.6600 ns/op 9.2050 ns/op 1.05
array of 16000 items push then pop 122.85 ns/op 112.89 ns/op 1.09
LinkedList of 16000 items push then pop 9.3980 ns/op 8.6160 ns/op 1.09
array of 24000 items push then shift 2.7103 us/op 2.3245 us/op 1.17
LinkedList of 24000 items push then shift 10.016 ns/op 8.8380 ns/op 1.13
array of 24000 items push then pop 91.020 ns/op 87.126 ns/op 1.04
LinkedList of 24000 items push then pop 10.103 ns/op 8.5650 ns/op 1.18
intersect bitArray bitLen 8 15.540 ns/op 13.338 ns/op 1.17
intersect array and set length 8 114.16 ns/op 78.475 ns/op 1.45
intersect bitArray bitLen 128 46.517 ns/op 43.942 ns/op 1.06
intersect array and set length 128 1.2319 us/op 1.0597 us/op 1.16
Buffer.concat 32 items 3.0500 us/op 2.7490 us/op 1.11
Uint8Array.set 32 items 3.0500 us/op 2.4020 us/op 1.27
pass gossip attestations to forkchoice per slot 2.3632 ms/op 4.0145 ms/op 0.59
computeDeltas 3.0826 ms/op 3.3486 ms/op 0.92
computeProposerBoostScoreFromBalances 1.7958 ms/op 1.8254 ms/op 0.98
altair processAttestation - 250000 vs - 7PWei normalcase 2.0928 ms/op 3.4777 ms/op 0.60
altair processAttestation - 250000 vs - 7PWei worstcase 3.2942 ms/op 4.4096 ms/op 0.75
altair processAttestation - setStatus - 1/6 committees join 142.69 us/op 146.91 us/op 0.97
altair processAttestation - setStatus - 1/3 committees join 262.35 us/op 286.26 us/op 0.92
altair processAttestation - setStatus - 1/2 committees join 373.08 us/op 376.61 us/op 0.99
altair processAttestation - setStatus - 2/3 committees join 440.96 us/op 482.01 us/op 0.91
altair processAttestation - setStatus - 4/5 committees join 647.51 us/op 660.93 us/op 0.98
altair processAttestation - setStatus - 100% committees join 751.28 us/op 773.14 us/op 0.97
altair processBlock - 250000 vs - 7PWei normalcase 19.044 ms/op 18.046 ms/op 1.06
altair processBlock - 250000 vs - 7PWei normalcase hashState 25.658 ms/op 26.638 ms/op 0.96
altair processBlock - 250000 vs - 7PWei worstcase 52.066 ms/op 52.388 ms/op 0.99
altair processBlock - 250000 vs - 7PWei worstcase hashState 68.950 ms/op 70.749 ms/op 0.97
phase0 processBlock - 250000 vs - 7PWei normalcase 2.0129 ms/op 2.3551 ms/op 0.85
phase0 processBlock - 250000 vs - 7PWei worstcase 27.903 ms/op 29.480 ms/op 0.95
altair processEth1Data - 250000 vs - 7PWei normalcase 459.93 us/op 532.38 us/op 0.86
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 6.8000 us/op 9.8170 us/op 0.69
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 19.629 us/op 25.603 us/op 0.77
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 8.4370 us/op 13.332 us/op 0.63
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 6.5130 us/op 9.5570 us/op 0.68
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 74.243 us/op 117.86 us/op 0.63
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 612.85 us/op 667.80 us/op 0.92
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 884.23 us/op 964.39 us/op 0.92
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 873.61 us/op 890.73 us/op 0.98
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 2.3088 ms/op 2.3957 ms/op 0.96
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 1.4715 ms/op 1.7409 ms/op 0.85
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 3.7086 ms/op 4.6821 ms/op 0.79
Tree 40 250000 create 306.85 ms/op 319.89 ms/op 0.96
Tree 40 250000 get(125000) 173.39 ns/op 188.47 ns/op 0.92
Tree 40 250000 set(125000) 832.54 ns/op 979.19 ns/op 0.85
Tree 40 250000 toArray() 16.497 ms/op 21.185 ms/op 0.78
Tree 40 250000 iterate all - toArray() + loop 16.299 ms/op 20.331 ms/op 0.80
Tree 40 250000 iterate all - get(i) 64.243 ms/op 75.791 ms/op 0.85
MutableVector 250000 create 9.7755 ms/op 10.160 ms/op 0.96
MutableVector 250000 get(125000) 7.4170 ns/op 6.5480 ns/op 1.13
MutableVector 250000 set(125000) 259.76 ns/op 303.23 ns/op 0.86
MutableVector 250000 toArray() 2.8552 ms/op 3.6428 ms/op 0.78
MutableVector 250000 iterate all - toArray() + loop 2.9912 ms/op 3.6405 ms/op 0.82
MutableVector 250000 iterate all - get(i) 1.5248 ms/op 1.5902 ms/op 0.96
Array 250000 create 2.5134 ms/op 2.8512 ms/op 0.88
Array 250000 clone - spread 1.0692 ms/op 1.1862 ms/op 0.90
Array 250000 get(125000) 0.60800 ns/op 0.56200 ns/op 1.08
Array 250000 set(125000) 0.69600 ns/op 0.64700 ns/op 1.08
Array 250000 iterate all - loop 101.69 us/op 86.274 us/op 1.18
effectiveBalanceIncrements clone Uint8Array 300000 22.888 us/op 43.033 us/op 0.53
effectiveBalanceIncrements clone MutableVector 300000 295.00 ns/op 331.00 ns/op 0.89
effectiveBalanceIncrements rw all Uint8Array 300000 159.60 us/op 180.90 us/op 0.88
effectiveBalanceIncrements rw all MutableVector 300000 72.685 ms/op 92.533 ms/op 0.79
phase0 afterProcessEpoch - 250000 vs - 7PWei 109.82 ms/op 128.04 ms/op 0.86
phase0 beforeProcessEpoch - 250000 vs - 7PWei 41.778 ms/op 48.447 ms/op 0.86
altair processEpoch - mainnet_e81889 302.59 ms/op 380.09 ms/op 0.80
mainnet_e81889 - altair beforeProcessEpoch 65.210 ms/op 70.824 ms/op 0.92
mainnet_e81889 - altair processJustificationAndFinalization 17.328 us/op 20.147 us/op 0.86
mainnet_e81889 - altair processInactivityUpdates 5.0105 ms/op 6.0544 ms/op 0.83
mainnet_e81889 - altair processRewardsAndPenalties 64.161 ms/op 56.972 ms/op 1.13
mainnet_e81889 - altair processRegistryUpdates 2.2300 us/op 4.7180 us/op 0.47
mainnet_e81889 - altair processSlashings 558.00 ns/op 1.1530 us/op 0.48
mainnet_e81889 - altair processEth1DataReset 939.00 ns/op 633.00 ns/op 1.48
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.2742 ms/op 1.2716 ms/op 1.00
mainnet_e81889 - altair processSlashingsReset 4.8290 us/op 7.7460 us/op 0.62
mainnet_e81889 - altair processRandaoMixesReset 4.2160 us/op 7.0700 us/op 0.60
mainnet_e81889 - altair processHistoricalRootsUpdate 569.00 ns/op 1.5290 us/op 0.37
mainnet_e81889 - altair processParticipationFlagUpdates 2.4470 us/op 4.4650 us/op 0.55
mainnet_e81889 - altair processSyncCommitteeUpdates 565.00 ns/op 851.00 ns/op 0.66
mainnet_e81889 - altair afterProcessEpoch 120.42 ms/op 137.27 ms/op 0.88
phase0 processEpoch - mainnet_e58758 314.96 ms/op 431.62 ms/op 0.73
mainnet_e58758 - phase0 beforeProcessEpoch 132.23 ms/op 160.52 ms/op 0.82
mainnet_e58758 - phase0 processJustificationAndFinalization 16.268 us/op 25.625 us/op 0.63
mainnet_e58758 - phase0 processRewardsAndPenalties 54.988 ms/op 68.393 ms/op 0.80
mainnet_e58758 - phase0 processRegistryUpdates 7.3810 us/op 12.035 us/op 0.61
mainnet_e58758 - phase0 processSlashings 495.00 ns/op 994.00 ns/op 0.50
mainnet_e58758 - phase0 processEth1DataReset 495.00 ns/op 851.00 ns/op 0.58
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.0293 ms/op 1.5095 ms/op 0.68
mainnet_e58758 - phase0 processSlashingsReset 2.9620 us/op 7.9780 us/op 0.37
mainnet_e58758 - phase0 processRandaoMixesReset 4.1940 us/op 8.1650 us/op 0.51
mainnet_e58758 - phase0 processHistoricalRootsUpdate 535.00 ns/op 1.1090 us/op 0.48
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.9720 us/op 5.8440 us/op 0.68
mainnet_e58758 - phase0 afterProcessEpoch 100.12 ms/op 102.42 ms/op 0.98
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.2567 ms/op 1.2862 ms/op 0.98
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.5137 ms/op 1.6351 ms/op 0.93
altair processInactivityUpdates - 250000 normalcase 23.181 ms/op 26.574 ms/op 0.87
altair processInactivityUpdates - 250000 worstcase 20.480 ms/op 20.726 ms/op 0.99
phase0 processRegistryUpdates - 250000 normalcase 6.0500 us/op 7.0600 us/op 0.86
phase0 processRegistryUpdates - 250000 badcase_full_deposits 257.71 us/op 267.87 us/op 0.96
phase0 processRegistryUpdates - 250000 worstcase 0.5 124.66 ms/op 123.51 ms/op 1.01
altair processRewardsAndPenalties - 250000 normalcase 46.091 ms/op 63.162 ms/op 0.73
altair processRewardsAndPenalties - 250000 worstcase 53.123 ms/op 69.250 ms/op 0.77
phase0 getAttestationDeltas - 250000 normalcase 6.5487 ms/op 6.3368 ms/op 1.03
phase0 getAttestationDeltas - 250000 worstcase 6.3774 ms/op 6.4128 ms/op 0.99
phase0 processSlashings - 250000 worstcase 3.4885 ms/op 3.5288 ms/op 0.99
altair processSyncCommitteeUpdates - 250000 170.57 ms/op 173.73 ms/op 0.98
BeaconState.hashTreeRoot - No change 249.00 ns/op 258.00 ns/op 0.97
BeaconState.hashTreeRoot - 1 full validator 50.860 us/op 53.096 us/op 0.96
BeaconState.hashTreeRoot - 32 full validator 494.22 us/op 528.07 us/op 0.94
BeaconState.hashTreeRoot - 512 full validator 5.0325 ms/op 5.1867 ms/op 0.97
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 62.444 us/op 62.498 us/op 1.00
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 909.34 us/op 863.39 us/op 1.05
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 11.022 ms/op 11.572 ms/op 0.95
BeaconState.hashTreeRoot - 1 balances 48.721 us/op 48.590 us/op 1.00
BeaconState.hashTreeRoot - 32 balances 430.99 us/op 455.40 us/op 0.95
BeaconState.hashTreeRoot - 512 balances 4.2147 ms/op 4.2369 ms/op 0.99
BeaconState.hashTreeRoot - 250000 balances 73.815 ms/op 73.022 ms/op 1.01
aggregationBits - 2048 els - zipIndexesInBitList 15.861 us/op 15.817 us/op 1.00
regular array get 100000 times 41.139 us/op 31.995 us/op 1.29
wrappedArray get 100000 times 41.143 us/op 31.954 us/op 1.29
arrayWithProxy get 100000 times 18.186 ms/op 14.991 ms/op 1.21
ssz.Root.equals 702.00 ns/op 573.00 ns/op 1.23
byteArrayEquals 692.00 ns/op 522.00 ns/op 1.33
shuffle list - 16384 els 7.0179 ms/op 6.6387 ms/op 1.06
shuffle list - 250000 els 102.47 ms/op 97.562 ms/op 1.05
processSlot - 1 slots 8.5690 us/op 8.5140 us/op 1.01
processSlot - 32 slots 1.3592 ms/op 1.3131 ms/op 1.04
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 194.00 us/op 188.59 us/op 1.03
getCommitteeAssignments - req 1 vs - 250000 vc 2.9031 ms/op 2.8349 ms/op 1.02
getCommitteeAssignments - req 100 vs - 250000 vc 4.1291 ms/op 4.0464 ms/op 1.02
getCommitteeAssignments - req 1000 vs - 250000 vc 4.4318 ms/op 4.3507 ms/op 1.02
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 5.2800 ns/op 4.5600 ns/op 1.16
state getBlockRootAtSlot - 250000 vs - 7PWei 1.0245 us/op 941.74 ns/op 1.09
computeProposers - vc 250000 10.818 ms/op 10.456 ms/op 1.03
computeEpochShuffling - vc 250000 104.47 ms/op 100.80 ms/op 1.04
getNextSyncCommittee - vc 250000 174.30 ms/op 168.17 ms/op 1.04

by benchmarkbot/action

@wemeetagain wemeetagain changed the title Fix gossiping blsChange on submit if post capella Allow blsChange reprocess/republish via API submission Feb 20, 2023
@wemeetagain wemeetagain merged commit 2c16369 into unstable Feb 20, 2023
@wemeetagain wemeetagain deleted the g11tech/fix-bls-gossip branch February 20, 2023 17:48
@philknows philknows added this to the v1.5.0 milestone Feb 20, 2023
wemeetagain pushed a commit that referenced this pull request Feb 27, 2023
@g11tech @wemeetagain
Allow blsChange reprocess/republish via API submission (#5185)
1c69f97 
* Fix gossiping blsChange on submit if post capella

* comment improvement
dapplion added a commit that referenced this pull request Feb 28, 2023
@dapplion @wemeetagain
@g11tech @nflaig
Merge stable to unstable no conflicts (#5216)
d4b9201 
* v1.5.0

* Add validatior option to specify builder block selection strategy (#5064)

* Add validatior option to specify builder block selection strategy

* cleanup and improvements

* server api fx

* fix test

* fix the testcase

* Re-gossip bls changes if submitted earlier than capella (#5049)

* Cache and retransmit bls changes if submitted early

* regossip when synced on/post capella

* fix tests

* modify publish vs cache condition

* add logging

* add error logging

* batchify the processing

* fix some of the cases

* fix log issues

* also delete included keys

* fix lint

* simplify gossip loop

* fix build

* refac to a versioned bls change store

* update

* logging improvements

* improve logging and finish behavior

* cleanup

* Rename eip4844 fork as Deneb (#5082)

* Fix and extract payload to header conversion (#5101)

* Remove unused eslint disable directives (#5046)

* Add execution payload header to light client header capella onwards (#5027)

* Add execution payload header to light client header capella onwards

* add more forks

* add allfork types

* make other types multifork

* make events multifork compatible

* make lightclient routes multifork

* update light-client transport

* upgrade the blocktype on lightclient server import block

* fixes

* fix test

* fix event serialization/deserilization type

* update proof constants

* fix payload index

* update the light client update generation to multi fork

* fix types

* add spec override to pass specs

* make event types allforks

* make req/resp multifork

* make types multifork throughout

* fix the update serialize/deserialize

* fix error log

* better fork determination for update

* fix the fork of finalized

* fix tests

* fix header upgradation

* Fix zero finalized header in lightclient update gen (#5121)

* Schedule capella on sepolia (#5127)

* Get the capella builder functional (#5179)

* Allow blsChange reprocess/republish via API submission (#5185)

* Fix gossiping blsChange on submit if post capella

* comment improvement

* Use new db bucket for light client updates (#5205)

---------

Co-authored-by: Cayman <caymannava@gmail.com>
Co-authored-by: g11tech <develop@g11tech.io>
Co-authored-by: Nico Flaig <nflaig@protonmail.com>
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.6.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v1.5.0
Development

Successfully merging this pull request may close these issues.
3 participants
@g11tech @wemeetagain @philknows