Add ExecutionPayloadHeader to LC data
#3151
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| # Capella Light Client -- Fork Logic | ||
|
|
||
| ## Table of contents | ||
|
|
||
| <!-- TOC --> | ||
| <!-- START doctoc generated TOC please keep comment here to allow auto update --> | ||
| <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> | ||
|
|
||
| - [Introduction](#introduction) | ||
| - [Upgrading light client data](#upgrading-light-client-data) | ||
| - [Upgrading the store](#upgrading-the-store) | ||
|
|
||
| <!-- END doctoc generated TOC please keep comment here to allow auto update --> | ||
| <!-- /TOC --> | ||
|
|
||
| ## Introduction | ||
|
|
||
| This document describes how to upgrade existing light client objects based on the [Altair specification](../../altair/light-client/sync-protocol.md) to Capella. This is necessary when processing pre-Capella data with a post-Capella `LightClientStore`. Note that the data being exchanged over the network protocols uses the original format. | ||
|
|
||
| ### Upgrading light client data | ||
|
|
||
| A Capella `LightClientStore` can still process earlier light client data. In order to do so, that pre-Capella data needs to be locally upgraded to Capella before processing. | ||
|
|
||
| ```python | ||
| def upgrade_lc_header_to_capella(pre: BeaconBlockHeader) -> LightClientHeader: | ||
| return LightClientHeader( | ||
| beacon=pre, | ||
| ) | ||
| ``` | ||
|
|
||
| ```python | ||
| def upgrade_lc_bootstrap_to_capella(pre: altair.LightClientBootstrap) -> LightClientBootstrap: | ||
| return LightClientBootstrap( | ||
| header=upgrade_lc_header_to_capella(pre.header), | ||
| current_sync_committee=pre.current_sync_committee, | ||
| current_sync_committee_branch=pre.current_sync_committee_branch, | ||
| ) | ||
| ``` | ||
|
|
||
| ```python | ||
| def upgrade_lc_update_to_capella(pre: altair.LightClientUpdate) -> LightClientUpdate: | ||
| return LightClientUpdate( | ||
| attested_header=upgrade_lc_header_to_capella(pre.attested_header), | ||
| next_sync_committee=pre.next_sync_committee, | ||
| next_sync_committee_branch=pre.next_sync_committee_branch, | ||
| finalized_header=upgrade_lc_header_to_capella(pre.finalized_header), | ||
| finality_branch=pre.finality_branch, | ||
| sync_aggregate=pre.sync_aggregate, | ||
| signature_slot=pre.signature_slot, | ||
| ) | ||
| ``` | ||
|
|
||
| ```python | ||
| def upgrade_lc_finality_update_to_capella(pre: altair.LightClientFinalityUpdate) -> LightClientFinalityUpdate: | ||
| return LightClientFinalityUpdate( | ||
| attested_header=upgrade_lc_header_to_capella(pre.attested_header), | ||
| finalized_header=upgrade_lc_header_to_capella(pre.finalized_header), | ||
| finality_branch=pre.finality_branch, | ||
| sync_aggregate=pre.sync_aggregate, | ||
| signature_slot=pre.signature_slot, | ||
| ) | ||
| ``` | ||
|
|
||
| ```python | ||
| def upgrade_lc_optimistic_update_to_capella(pre: altair.LightClientOptimisticUpdate) -> LightClientOptimisticUpdate: | ||
| return LightClientOptimisticUpdate( | ||
| attested_header=upgrade_lc_header_to_capella(pre.attested_header), | ||
| sync_aggregate=pre.sync_aggregate, | ||
| signature_slot=pre.signature_slot, | ||
| ) | ||
| ``` | ||
|
|
||
| ### Upgrading the store | ||
|
|
||
| Existing `LightClientStore` objects based on Altair MUST be upgraded to Capella before Capella based light client data can be processed. The `LightClientStore` upgrade MAY be performed before `CAPELLA_FORK_EPOCH`. | ||
|
|
||
| ```python | ||
| def upgrade_lc_store_to_capella(pre: altair.LightClientStore) -> LightClientStore: | ||
| if pre.best_valid_update is None: | ||
| best_valid_update = None | ||
| else: | ||
| best_valid_update = upgrade_lc_update_to_capella(pre.best_valid_update) | ||
| return LightClientStore( | ||
| finalized_header=upgrade_lc_header_to_capella(pre.finalized_header), | ||
| current_sync_committee=pre.current_sync_committee, | ||
| next_sync_committee=pre.next_sync_committee, | ||
| best_valid_update=best_valid_update, | ||
| optimistic_header=upgrade_lc_header_to_capella(pre.optimistic_header), | ||
| previous_max_active_participants=pre.previous_max_active_participants, | ||
| current_max_active_participants=pre.current_max_active_participants, | ||
| ) | ||
| ``` | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| # Capella Light Client -- Full Node | ||
|
|
||
| **Notice**: This document is a work-in-progress for researchers and implementers. | ||
|
|
||
| ## Table of contents | ||
|
|
||
| <!-- TOC --> | ||
| <!-- START doctoc generated TOC please keep comment here to allow auto update --> | ||
| <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> | ||
|
|
||
| - [Introduction](#introduction) | ||
| - [Helper functions](#helper-functions) | ||
| - [`compute_merkle_proof_for_block_body`](#compute_merkle_proof_for_block_body) | ||
| - [Modified `block_to_light_client_header`](#modified-block_to_light_client_header) | ||
|
|
||
| <!-- END doctoc generated TOC please keep comment here to allow auto update --> | ||
| <!-- /TOC --> | ||
|
|
||
| ## Introduction | ||
|
|
||
| This upgrade adds information about the execution payload to light client data as part of the Capella upgrade. | ||
|
|
||
| ## Helper functions | ||
|
|
||
| ### `compute_merkle_proof_for_block_body` | ||
|
|
||
| ```python | ||
| def compute_merkle_proof_for_block_body(body: BeaconBlockBody, | ||
| index: GeneralizedIndex) -> Sequence[Bytes32]: | ||
| ... | ||
| ``` | ||
|
|
||
| ### Modified `block_to_light_client_header` | ||
|
|
||
| ```python | ||
| def block_to_light_client_header(block: SignedBeaconBlock) -> LightClientHeader: | ||
| epoch = compute_epoch_at_slot(block.message.slot) | ||
|
|
||
| if epoch >= CAPELLA_FORK_EPOCH: | ||
|
Comment on lines
+36
to
+39
There was a problem hiding this comment. Is it because There was a problem hiding this comment. Yes, exactly (or, in theory, even all the way back to Altair or Phase0, if bellatrix / altair never finalized, and finalized header is still back in those ancient forks). Added a comment to clarify that. |
||
| payload = block.message.body.execution_payload | ||
| execution_header = ExecutionPayloadHeader( | ||
| parent_hash=payload.parent_hash, | ||
| fee_recipient=payload.fee_recipient, | ||
| state_root=payload.state_root, | ||
| receipts_root=payload.receipts_root, | ||
| logs_bloom=payload.logs_bloom, | ||
| prev_randao=payload.prev_randao, | ||
| block_number=payload.block_number, | ||
| gas_limit=payload.gas_limit, | ||
| gas_used=payload.gas_used, | ||
| timestamp=payload.timestamp, | ||
| extra_data=payload.extra_data, | ||
| base_fee_per_gas=payload.base_fee_per_gas, | ||
| block_hash=payload.block_hash, | ||
| transactions_root=hash_tree_root(payload.transactions), | ||
| withdrawals_root=hash_tree_root(payload.withdrawals), | ||
| ) | ||
| execution_branch = compute_merkle_proof_for_block_body(block.message.body, EXECUTION_PAYLOAD_INDEX) | ||
| else: | ||
| # Note that during fork transitions, `finalized_header` may still point to earlier forks. | ||
| # While Bellatrix blocks also contain an `ExecutionPayload` (minus `withdrawals_root`), | ||
| # it was not included in the corresponding light client data. To ensure compatibility | ||
| # with legacy data going through `upgrade_lc_header_to_capella`, leave out execution data. | ||
| execution_header = ExecutionPayloadHeader() | ||
| execution_branch = [Bytes32() for _ in range(floorlog2(EXECUTION_PAYLOAD_INDEX))] | ||
|
|
||
| return LightClientHeader( | ||
| beacon=BeaconBlockHeader( | ||
| slot=block.message.slot, | ||
| proposer_index=block.message.proposer_index, | ||
| parent_root=block.message.parent_root, | ||
| state_root=block.message.state_root, | ||
| body_root=hash_tree_root(block.message.body), | ||
| ), | ||
| execution=execution_header, | ||
| execution_branch=execution_branch, | ||
| ) | ||
| ``` | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| # Capella Light Client -- Networking | ||
|
|
||
| **Notice**: This document is a work-in-progress for researchers and implementers. | ||
|
|
||
| ## Table of contents | ||
|
|
||
| <!-- TOC --> | ||
| <!-- START doctoc generated TOC please keep comment here to allow auto update --> | ||
| <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> | ||
|
|
||
| - [Networking](#networking) | ||
| - [The gossip domain: gossipsub](#the-gossip-domain-gossipsub) | ||
| - [Topics and messages](#topics-and-messages) | ||
| - [Global topics](#global-topics) | ||
| - [`light_client_finality_update`](#light_client_finality_update) | ||
| - [`light_client_optimistic_update`](#light_client_optimistic_update) | ||
| - [The Req/Resp domain](#the-reqresp-domain) | ||
| - [Messages](#messages) | ||
| - [GetLightClientBootstrap](#getlightclientbootstrap) | ||
| - [LightClientUpdatesByRange](#lightclientupdatesbyrange) | ||
| - [GetLightClientFinalityUpdate](#getlightclientfinalityupdate) | ||
| - [GetLightClientOptimisticUpdate](#getlightclientoptimisticupdate) | ||
|
|
||
| <!-- END doctoc generated TOC please keep comment here to allow auto update --> | ||
| <!-- /TOC --> | ||
|
|
||
| ## Networking | ||
|
|
||
| The [Altair light client networking specification](../../altair/light-client/p2p-interface.md) is extended to exchange [Capella light client data](./sync-protocol.md). | ||
|
|
||
| ### The gossip domain: gossipsub | ||
|
|
||
| #### Topics and messages | ||
|
|
||
| ##### Global topics | ||
|
|
||
| ###### `light_client_finality_update` | ||
|
|
||
| [0]: # (eth2spec: skip) | ||
|
|
||
| | `fork_version` | Message SSZ type | | ||
| | ------------------------------------------------------ | ------------------------------------- | | ||
| | `GENESIS_FORK_VERSION` | n/a | | ||
| | `ALTAIR_FORK_VERSION` through `BELLATRIX_FORK_VERSION` | `altair.LightClientFinalityUpdate` | | ||
| | `CAPELLA_FORK_VERSION` and later | `capella.LightClientFinalityUpdate` | | ||
|
|
||
| ###### `light_client_optimistic_update` | ||
|
|
||
| [0]: # (eth2spec: skip) | ||
|
|
||
| | `fork_version` | Message SSZ type | | ||
| | ------------------------------------------------------ | ------------------------------------- | | ||
| | `GENESIS_FORK_VERSION` | n/a | | ||
| | `ALTAIR_FORK_VERSION` through `BELLATRIX_FORK_VERSION` | `altair.LightClientOptimisticUpdate` | | ||
| | `CAPELLA_FORK_VERSION` and later | `capella.LightClientOptimisticUpdate` | | ||
|
|
||
| ### The Req/Resp domain | ||
|
|
||
| #### Messages | ||
|
|
||
| ##### GetLightClientBootstrap | ||
|
|
||
| [0]: # (eth2spec: skip) | ||
|
|
||
| | `fork_version` | Response SSZ type | | ||
| | ------------------------------------------------------ | ------------------------------------- | | ||
| | `GENESIS_FORK_VERSION` | n/a | | ||
| | `ALTAIR_FORK_VERSION` through `BELLATRIX_FORK_VERSION` | `altair.LightClientBootstrap` | | ||
| | `CAPELLA_FORK_VERSION` and later | `capella.LightClientBootstrap` | | ||
|
|
||
| ##### LightClientUpdatesByRange | ||
|
|
||
| [0]: # (eth2spec: skip) | ||
|
|
||
| | `fork_version` | Response chunk SSZ type | | ||
| | ------------------------------------------------------ | ------------------------------------- | | ||
| | `GENESIS_FORK_VERSION` | n/a | | ||
| | `ALTAIR_FORK_VERSION` through `BELLATRIX_FORK_VERSION` | `altair.LightClientUpdate` | | ||
| | `CAPELLA_FORK_VERSION` and later | `capella.LightClientUpdate` | | ||
|
|
||
| ##### GetLightClientFinalityUpdate | ||
|
|
||
| [0]: # (eth2spec: skip) | ||
|
|
||
| | `fork_version` | Response SSZ type | | ||
| | ------------------------------------------------------ | ------------------------------------- | | ||
| | `GENESIS_FORK_VERSION` | n/a | | ||
| | `ALTAIR_FORK_VERSION` through `BELLATRIX_FORK_VERSION` | `altair.LightClientFinalityUpdate` | | ||
| | `CAPELLA_FORK_VERSION` and later | `capella.LightClientFinalityUpdate` | | ||
|
|
||
| ##### GetLightClientOptimisticUpdate | ||
|
|
||
| [0]: # (eth2spec: skip) | ||
|
|
||
| | `fork_version` | Response SSZ type | | ||
| | ------------------------------------------------------ | ------------------------------------- | | ||
| | `GENESIS_FORK_VERSION` | n/a | | ||
| | `ALTAIR_FORK_VERSION` through `BELLATRIX_FORK_VERSION` | `altair.LightClientOptimisticUpdate` | | ||
| | `CAPELLA_FORK_VERSION` and later | `capella.LightClientOptimisticUpdate` | |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Although
altair.LightClientBootstrapSSZ fields are identicalbellatrix.LightClientBootstrap, I still think it should bebellatrix.LightClientBootstraphere for the fork transition. That is, we needbellatrix/light-client/fork.mdandupgrade_*_to_bellatrixhelpers.Moreover, should we make
bellatrix.LightClientHeaderincludeExecutionPayloadHeaderand the Merkle proof? It seems more natural since theExecutionPayloadHeaderfield was first introduced in Bellatrix. However, (1) it's less useful since Capella is coming soon, and (2) we are still discussing #3078. So I'd say it's fine to keepbellatrix.LightClientHeaderas simple asaltair.LightClientHeader.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, not sure about the nop
upgrade_*_to_bellatrix— this would then be required for any future fork, regardless of whether there are any real updates.Kinda like it with
altairfor readability (in practice, implementations will reuse the Altair definition for Bellatrix and not duplicate all the code with every fork). Agree though, that it is a bit hacky as proposed.Anyhow, if you insist, can update it (it works well as-is in pytests though).
About changing Bellatrix data: that one is already being cached in DBs and gossiped without the
ExecutionPayloadHeader. Changing that would be quite tricky (and in case of gossip may even need a version bump on those topics to avoid disconnecting peers on the previous version). Would therefore leave Bellatrix as is, also considering Capella is in the near future.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
About
upgrade_*_to_bellatrix, those are actually not needed. Theseupgradehelpers are only used when obtaining Altair formatted data and trying to feed it to an already-upgradedLightClientStore.In the
p2p-interface.mdfiles you can see that it is not possible to obtain abellatrix.LightClient*object.fork_versionGENESIS_FORK_VERSIONALTAIR_FORK_VERSIONthroughBELLATRIX_FORK_VERSIONaltair.LightClientUpdateCAPELLA_FORK_VERSIONcapella.LightClientUpdateEIP4844_FORK_VERSIONand latereip4844.LightClientUpdateThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And, with this, it is actually correct to have the upgrade convert directly from
altair.LightClient*tocapella.LightClient*.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(as in, there is no way to actually obtain a
bellatrix.LightClientUpdatein practice, so no upgrade facility is needed).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now I agree on no need to implement
upgrade_*_to_bellatrixand the wholebellatrix/light-client/fork.mdfile. 👍But I still think it's good to use the "previous fork" (bellatrix) in
upgrade_*_to_capellain specs.bellatrix.Xmeans the class/method version we have in Bellatrix.bellatrix.LightClientUpdateexists in pyspec usage, although it is just as same asaltair.LightClientUpdatedefinition.If client implementations actually skip the "unchanged-phase" objects, it's also acceptable since the results are the same.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good, will change the function signature to
bellatrix.Xbut omit theupgrade_*_to_bellatrixstub.