Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove all intermediary ChainEvents #4973

Merged
merged 2 commits into from
Jan 5, 2023
Merged

Conversation

dapplion
Copy link
Contributor

@dapplion dapplion commented Jan 4, 2023

Motivation

BeaconNode source emits events meant for the API through intermediary ChainEvents. That's not necessary since the few users of the internal events can just listen to the events defined as in the API.

Description

Remove all intermediary ChainEvents

@dapplion dapplion requested a review from a team as a code owner January 4, 2023 05:44
@github-actions
Copy link
Contributor

github-actions bot commented Jan 4, 2023

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: c504d62 Previous: 278c2ee Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.0702 ms/op 1.3943 ms/op 0.77
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 70.756 us/op 79.028 us/op 0.90
BLS verify - blst-native 1.8607 ms/op 2.1275 ms/op 0.87
BLS verifyMultipleSignatures 3 - blst-native 3.8063 ms/op 4.3652 ms/op 0.87
BLS verifyMultipleSignatures 8 - blst-native 8.2017 ms/op 9.5053 ms/op 0.86
BLS verifyMultipleSignatures 32 - blst-native 30.104 ms/op 34.562 ms/op 0.87
BLS aggregatePubkeys 32 - blst-native 39.411 us/op 47.257 us/op 0.83
BLS aggregatePubkeys 128 - blst-native 154.37 us/op 178.05 us/op 0.87
getAttestationsForBlock 89.890 ms/op 105.61 ms/op 0.85
isKnown best case - 1 super set check 414.00 ns/op 505.00 ns/op 0.82
isKnown normal case - 2 super set checks 408.00 ns/op 498.00 ns/op 0.82
isKnown worse case - 16 super set checks 410.00 ns/op 493.00 ns/op 0.83
CheckpointStateCache - add get delete 8.1340 us/op 9.5480 us/op 0.85
validate gossip signedAggregateAndProof - struct 4.2520 ms/op 5.0023 ms/op 0.85
validate gossip attestation - struct 2.0284 ms/op 2.3823 ms/op 0.85
pickEth1Vote - no votes 2.1513 ms/op 2.4055 ms/op 0.89
pickEth1Vote - max votes 16.938 ms/op 18.783 ms/op 0.90
pickEth1Vote - Eth1Data hashTreeRoot value x2048 12.090 ms/op 13.930 ms/op 0.87
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 21.797 ms/op 24.486 ms/op 0.89
pickEth1Vote - Eth1Data fastSerialize value x2048 1.2193 ms/op 1.4340 ms/op 0.85
pickEth1Vote - Eth1Data fastSerialize tree x2048 9.6911 ms/op 11.023 ms/op 0.88
bytes32 toHexString 1.0410 us/op 1.1430 us/op 0.91
bytes32 Buffer.toString(hex) 534.00 ns/op 635.00 ns/op 0.84
bytes32 Buffer.toString(hex) from Uint8Array 904.00 ns/op 1.0620 us/op 0.85
bytes32 Buffer.toString(hex) + 0x 552.00 ns/op 649.00 ns/op 0.85
Object access 1 prop 0.27500 ns/op 0.30500 ns/op 0.90
Map access 1 prop 0.31600 ns/op 0.34700 ns/op 0.91
Object get x1000 16.117 ns/op 19.106 ns/op 0.84
Map get x1000 0.98700 ns/op 1.1760 ns/op 0.84
Object set x1000 114.28 ns/op 132.86 ns/op 0.86
Map set x1000 68.421 ns/op 82.081 ns/op 0.83
Return object 10000 times 0.36390 ns/op 0.44370 ns/op 0.82
Throw Error 10000 times 6.7481 us/op 8.1149 us/op 0.83
fastMsgIdFn sha256 / 200 bytes 4.4200 us/op 5.3760 us/op 0.82
fastMsgIdFn h32 xxhash / 200 bytes 448.00 ns/op 527.00 ns/op 0.85
fastMsgIdFn h64 xxhash / 200 bytes 605.00 ns/op 715.00 ns/op 0.85
fastMsgIdFn sha256 / 1000 bytes 14.131 us/op 15.730 us/op 0.90
fastMsgIdFn h32 xxhash / 1000 bytes 601.00 ns/op 705.00 ns/op 0.85
fastMsgIdFn h64 xxhash / 1000 bytes 704.00 ns/op 823.00 ns/op 0.86
fastMsgIdFn sha256 / 10000 bytes 122.10 us/op 137.62 us/op 0.89
fastMsgIdFn h32 xxhash / 10000 bytes 2.3110 us/op 2.6880 us/op 0.86
fastMsgIdFn h64 xxhash / 10000 bytes 1.6710 us/op 1.8450 us/op 0.91
enrSubnets - fastDeserialize 64 bits 2.3290 us/op 2.8920 us/op 0.81
enrSubnets - ssz BitVector 64 bits 785.00 ns/op 912.00 ns/op 0.86
enrSubnets - fastDeserialize 4 bits 282.00 ns/op 338.00 ns/op 0.83
enrSubnets - ssz BitVector 4 bits 830.00 ns/op 957.00 ns/op 0.87
prioritizePeers score -10:0 att 32-0.1 sync 2-0 168.76 us/op 215.69 us/op 0.78
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 197.10 us/op 246.16 us/op 0.80
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 320.26 us/op 362.77 us/op 0.88
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 549.31 us/op 609.39 us/op 0.90
prioritizePeers score 0:0 att 64-1 sync 4-1 616.64 us/op 707.48 us/op 0.87
array of 16000 items push then shift 2.8006 us/op 3.2655 us/op 0.86
LinkedList of 16000 items push then shift 19.169 ns/op 19.710 ns/op 0.97
array of 16000 items push then pop 222.53 ns/op 252.72 ns/op 0.88
LinkedList of 16000 items push then pop 17.900 ns/op 19.798 ns/op 0.90
array of 24000 items push then shift 3.9692 us/op 4.5780 us/op 0.87
LinkedList of 24000 items push then shift 22.118 ns/op 21.364 ns/op 1.04
array of 24000 items push then pop 208.46 ns/op 221.36 ns/op 0.94
LinkedList of 24000 items push then pop 19.322 ns/op 20.165 ns/op 0.96
intersect bitArray bitLen 8 29.196 ns/op 34.108 ns/op 0.86
intersect array and set length 8 164.68 ns/op 175.67 ns/op 0.94
intersect bitArray bitLen 128 119.08 ns/op 125.40 ns/op 0.95
intersect array and set length 128 2.2628 us/op 2.6103 us/op 0.87
Buffer.concat 32 items 5.1980 us/op 6.0120 us/op 0.86
Uint8Array.set 32 items 4.1330 us/op 4.6750 us/op 0.88
pass gossip attestations to forkchoice per slot 4.3506 ms/op 4.8104 ms/op 0.90
computeDeltas 6.2403 ms/op 7.7510 ms/op 0.81
computeProposerBoostScoreFromBalances 2.8406 ms/op 3.0912 ms/op 0.92
altair processAttestation - 250000 vs - 7PWei normalcase 3.8402 ms/op 4.2220 ms/op 0.91
altair processAttestation - 250000 vs - 7PWei worstcase 6.0687 ms/op 6.8632 ms/op 0.88
altair processAttestation - setStatus - 1/6 committees join 217.46 us/op 239.83 us/op 0.91
altair processAttestation - setStatus - 1/3 committees join 416.57 us/op 461.07 us/op 0.90
altair processAttestation - setStatus - 1/2 committees join 574.74 us/op 666.75 us/op 0.86
altair processAttestation - setStatus - 2/3 committees join 742.20 us/op 927.78 us/op 0.80
altair processAttestation - setStatus - 4/5 committees join 1.0255 ms/op 1.1455 ms/op 0.90
altair processAttestation - setStatus - 100% committees join 1.2032 ms/op 1.3524 ms/op 0.89
altair processBlock - 250000 vs - 7PWei normalcase 27.285 ms/op 30.587 ms/op 0.89
altair processBlock - 250000 vs - 7PWei normalcase hashState 45.105 ms/op 42.345 ms/op 1.07
altair processBlock - 250000 vs - 7PWei worstcase 74.541 ms/op 84.577 ms/op 0.88
altair processBlock - 250000 vs - 7PWei worstcase hashState 98.719 ms/op 115.57 ms/op 0.85
phase0 processBlock - 250000 vs - 7PWei normalcase 3.6037 ms/op 3.8996 ms/op 0.92
phase0 processBlock - 250000 vs - 7PWei worstcase 45.775 ms/op 52.446 ms/op 0.87
altair processEth1Data - 250000 vs - 7PWei normalcase 833.92 us/op 854.93 us/op 0.98
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 14.128 us/op 19.559 us/op 0.72
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 44.106 us/op 47.269 us/op 0.93
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 18.760 us/op 24.150 us/op 0.78
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 14.213 us/op 17.578 us/op 0.81
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 169.22 us/op 142.54 us/op 1.19
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 1.1228 ms/op 1.1988 ms/op 0.94
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 1.5442 ms/op 1.7129 ms/op 0.90
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 1.5321 ms/op 1.7119 ms/op 0.89
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 4.5459 ms/op 5.0834 ms/op 0.89
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 3.1544 ms/op 3.3573 ms/op 0.94
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 7.2036 ms/op 8.1126 ms/op 0.89
Tree 40 250000 create 746.22 ms/op 757.98 ms/op 0.98
Tree 40 250000 get(125000) 296.07 ns/op 332.14 ns/op 0.89
Tree 40 250000 set(125000) 2.3357 us/op 2.3744 us/op 0.98
Tree 40 250000 toArray() 32.279 ms/op 35.799 ms/op 0.90
Tree 40 250000 iterate all - toArray() + loop 32.779 ms/op 36.499 ms/op 0.90
Tree 40 250000 iterate all - get(i) 109.00 ms/op 124.75 ms/op 0.87
MutableVector 250000 create 16.596 ms/op 20.363 ms/op 0.82
MutableVector 250000 get(125000) 13.154 ns/op 15.633 ns/op 0.84
MutableVector 250000 set(125000) 643.57 ns/op 634.82 ns/op 1.01
MutableVector 250000 toArray() 6.2600 ms/op 7.0007 ms/op 0.89
MutableVector 250000 iterate all - toArray() + loop 6.3281 ms/op 6.9674 ms/op 0.91
MutableVector 250000 iterate all - get(i) 3.4788 ms/op 4.1120 ms/op 0.85
Array 250000 create 6.0580 ms/op 6.3707 ms/op 0.95
Array 250000 clone - spread 2.7795 ms/op 3.2014 ms/op 0.87
Array 250000 get(125000) 1.1300 ns/op 1.2910 ns/op 0.88
Array 250000 set(125000) 1.1590 ns/op 1.3030 ns/op 0.89
Array 250000 iterate all - loop 167.68 us/op 196.21 us/op 0.85
effectiveBalanceIncrements clone Uint8Array 300000 73.916 us/op 74.733 us/op 0.99
effectiveBalanceIncrements clone MutableVector 300000 849.00 ns/op 908.00 ns/op 0.94
effectiveBalanceIncrements rw all Uint8Array 300000 252.71 us/op 293.40 us/op 0.86
effectiveBalanceIncrements rw all MutableVector 300000 195.60 ms/op 209.46 ms/op 0.93
phase0 afterProcessEpoch - 250000 vs - 7PWei 192.39 ms/op 218.78 ms/op 0.88
phase0 beforeProcessEpoch - 250000 vs - 7PWei 79.759 ms/op 112.51 ms/op 0.71
altair processEpoch - mainnet_e81889 596.40 ms/op 666.33 ms/op 0.90
mainnet_e81889 - altair beforeProcessEpoch 145.67 ms/op 158.53 ms/op 0.92
mainnet_e81889 - altair processJustificationAndFinalization 37.229 us/op 45.689 us/op 0.81
mainnet_e81889 - altair processInactivityUpdates 11.768 ms/op 12.793 ms/op 0.92
mainnet_e81889 - altair processRewardsAndPenalties 94.656 ms/op 103.46 ms/op 0.91
mainnet_e81889 - altair processRegistryUpdates 5.5640 us/op 7.3330 us/op 0.76
mainnet_e81889 - altair processSlashings 1.3520 us/op 1.8790 us/op 0.72
mainnet_e81889 - altair processEth1DataReset 1.2190 us/op 2.2760 us/op 0.54
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.7496 ms/op 2.8445 ms/op 0.97
mainnet_e81889 - altair processSlashingsReset 8.3670 us/op 16.748 us/op 0.50
mainnet_e81889 - altair processRandaoMixesReset 10.452 us/op 13.567 us/op 0.77
mainnet_e81889 - altair processHistoricalRootsUpdate 1.4880 us/op 1.8480 us/op 0.81
mainnet_e81889 - altair processParticipationFlagUpdates 4.2490 us/op 7.2810 us/op 0.58
mainnet_e81889 - altair processSyncCommitteeUpdates 1.1230 us/op 1.6650 us/op 0.67
mainnet_e81889 - altair afterProcessEpoch 202.09 ms/op 227.71 ms/op 0.89
phase0 processEpoch - mainnet_e58758 551.17 ms/op 603.95 ms/op 0.91
mainnet_e58758 - phase0 beforeProcessEpoch 245.47 ms/op 257.19 ms/op 0.95
mainnet_e58758 - phase0 processJustificationAndFinalization 33.875 us/op 48.852 us/op 0.69
mainnet_e58758 - phase0 processRewardsAndPenalties 137.83 ms/op 145.61 ms/op 0.95
mainnet_e58758 - phase0 processRegistryUpdates 15.805 us/op 25.450 us/op 0.62
mainnet_e58758 - phase0 processSlashings 1.3450 us/op 1.6970 us/op 0.79
mainnet_e58758 - phase0 processEth1DataReset 1.1390 us/op 2.0530 us/op 0.55
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.4792 ms/op 2.5866 ms/op 0.96
mainnet_e58758 - phase0 processSlashingsReset 8.5300 us/op 12.060 us/op 0.71
mainnet_e58758 - phase0 processRandaoMixesReset 10.330 us/op 16.108 us/op 0.64
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.4460 us/op 2.4200 us/op 0.60
mainnet_e58758 - phase0 processParticipationRecordUpdates 8.5070 us/op 13.901 us/op 0.61
mainnet_e58758 - phase0 afterProcessEpoch 166.37 ms/op 191.64 ms/op 0.87
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.8761 ms/op 3.4031 ms/op 0.85
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.4936 ms/op 3.9372 ms/op 0.89
altair processInactivityUpdates - 250000 normalcase 50.922 ms/op 42.656 ms/op 1.19
altair processInactivityUpdates - 250000 worstcase 40.008 ms/op 40.710 ms/op 0.98
phase0 processRegistryUpdates - 250000 normalcase 14.526 us/op 20.021 us/op 0.73
phase0 processRegistryUpdates - 250000 badcase_full_deposits 526.78 us/op 619.62 us/op 0.85
phase0 processRegistryUpdates - 250000 worstcase 0.5 230.05 ms/op 244.72 ms/op 0.94
altair processRewardsAndPenalties - 250000 normalcase 83.271 ms/op 120.20 ms/op 0.69
altair processRewardsAndPenalties - 250000 worstcase 117.03 ms/op 94.355 ms/op 1.24
phase0 getAttestationDeltas - 250000 normalcase 12.289 ms/op 14.675 ms/op 0.84
phase0 getAttestationDeltas - 250000 worstcase 12.424 ms/op 15.200 ms/op 0.82
phase0 processSlashings - 250000 worstcase 6.3460 ms/op 6.8274 ms/op 0.93
altair processSyncCommitteeUpdates - 250000 270.57 ms/op 313.24 ms/op 0.86
BeaconState.hashTreeRoot - No change 453.00 ns/op 537.00 ns/op 0.84
BeaconState.hashTreeRoot - 1 full validator 65.076 us/op 74.567 us/op 0.87
BeaconState.hashTreeRoot - 32 full validator 653.54 us/op 869.44 us/op 0.75
BeaconState.hashTreeRoot - 512 full validator 6.1939 ms/op 6.8243 ms/op 0.91
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 77.433 us/op 88.854 us/op 0.87
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2139 ms/op 1.3313 ms/op 0.91
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 15.951 ms/op 18.547 ms/op 0.86
BeaconState.hashTreeRoot - 1 balances 54.753 us/op 71.886 us/op 0.76
BeaconState.hashTreeRoot - 32 balances 668.08 us/op 674.14 us/op 0.99
BeaconState.hashTreeRoot - 512 balances 5.7224 ms/op 6.5310 ms/op 0.88
BeaconState.hashTreeRoot - 250000 balances 93.791 ms/op 103.02 ms/op 0.91
aggregationBits - 2048 els - zipIndexesInBitList 30.932 us/op 35.900 us/op 0.86
regular array get 100000 times 67.379 us/op 81.441 us/op 0.83
wrappedArray get 100000 times 67.311 us/op 78.584 us/op 0.86
arrayWithProxy get 100000 times 27.740 ms/op 33.684 ms/op 0.82
ssz.Root.equals 1.0530 us/op 1.4660 us/op 0.72
byteArrayEquals 1.0490 us/op 1.2210 us/op 0.86
shuffle list - 16384 els 11.309 ms/op 13.031 ms/op 0.87
shuffle list - 250000 els 173.20 ms/op 191.50 ms/op 0.90
processSlot - 1 slots 12.573 us/op 12.949 us/op 0.97
processSlot - 32 slots 1.8236 ms/op 2.0218 ms/op 0.90
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 344.35 us/op 393.18 us/op 0.88
getCommitteeAssignments - req 1 vs - 250000 vc 5.5968 ms/op 6.5063 ms/op 0.86
getCommitteeAssignments - req 100 vs - 250000 vc 7.6083 ms/op 8.9200 ms/op 0.85
getCommitteeAssignments - req 1000 vs - 250000 vc 8.1593 ms/op 9.5871 ms/op 0.85
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 9.5700 ns/op 11.630 ns/op 0.82
state getBlockRootAtSlot - 250000 vs - 7PWei 1.1587 us/op 1.3154 us/op 0.88
computeProposers - vc 250000 16.492 ms/op 18.382 ms/op 0.90
computeEpochShuffling - vc 250000 195.89 ms/op 194.33 ms/op 1.01
getNextSyncCommittee - vc 250000 269.47 ms/op 303.06 ms/op 0.89

by benchmarkbot/action

commit 40b0630
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Wed Jan 4 20:50:58 2023 +0800

    Update tests

commit a6df872
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Wed Jan 4 12:55:07 2023 +0800

    Make enum const

commit c13f17c
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Wed Jan 4 12:40:17 2023 +0800

    Remove middleware ChainEvents

commit 41d11aa
Author: dapplion <35266934+dapplion@users.noreply.github.com>
Date:   Wed Jan 4 11:28:10 2023 +0800

    Remove intermediary event ChainEvent.forkChoiceReorg
@dapplion dapplion force-pushed the dapplion/direct-events branch from caa0d3a to 1bbe1ec Compare January 5, 2023 08:54
@@ -350,3 +349,26 @@ describe("executionEngine / ExecutionEngineHttp", function () {
console.log("\n\nDone\n\n");
}
});

async function retrieveCanonicalWithdrawals(bn: BeaconNode, fromSlot: Slot, toSlot: Slot): Promise<number> {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@g11tech Can you review if this assertion is good for your test? Re-wrote the previous event based, into an after the fact one

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes looks good

@@ -217,6 +219,9 @@ describe("executionEngine / ExecutionEngineHttp", function () {
// delay a bit so regular sync sees it's up to date and sync is completed from the beginning
const genesisSlotsDelay = 8;

// TODO for g11tech: Why 4? Provide rationale for the number
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@g11tech: Why 4? Provide rationale for the number

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from the test length run i noted that if there is no issue, withdrawal blocks come out to be 6, so i chose 4 as safe number incase there are any skipped slots for whatever reasons (sometimes have seen it happening)

Copy link
Contributor

@g11tech g11tech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@dapplion dapplion merged commit 65ce97f into unstable Jan 5, 2023
@dapplion dapplion deleted the dapplion/direct-events branch January 5, 2023 11:07
@@ -5,7 +5,7 @@ import {RouteDef, TypeJson} from "../../utils/index.js";

// See /packages/api/src/routes/index.ts for reasoning and instructions to add new routes

export enum EventType {
export const enum EventType {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dapplion What was the reason for changing this to a const enum besides just the minor reduction in generated code? The problem with const enums are that they are completely removed during compilation which means others using the library can not reuse them, see discord

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this should be reinstated as an enum

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also re-exported as necessary to allow for downstream reuse

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed in #5042, seems to be re-exported already so we should be fine there

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potentially unwanted side-effect from attempting to make EventType from API + ChainEvent work for the same handlers. Happy to revert it to enum

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants