PeerDAS fork-choice, validator custody and parameter changes

configs/mainnet.yaml

@@ -157,11 +157,13 @@ WHISK_PROPOSER_SELECTION_GAP: 2
 # EIP7594
 NUMBER_OF_COLUMNS: 128
 MAX_CELLS_IN_EXTENDED_MATRIX: 768
-DATA_COLUMN_SIDECAR_SUBNET_COUNT: 32
+DATA_COLUMN_SIDECAR_SUBNET_COUNT: 128
 MAX_REQUEST_DATA_COLUMN_SIDECARS: 16384
-SAMPLES_PER_SLOT: 8
-CUSTODY_REQUIREMENT: 1
-TARGET_NUMBER_OF_PEERS: 70
+SAMPLES_PER_SLOT: 16
+CUSTODY_REQUIREMENT: 4
+VALIDATOR_CUSTODY_REQUIREMENT: 8
+BALANCE_PER_ADDITIONAL_CUSTODY_SUBNET: 32000000000 # 2**5 * 10**9 (= 32,000,000,000)
+TARGET_NUMBER_OF_PEERS: 100
 
 # [New in Electra:EIP7251]
 MIN_PER_EPOCH_CHURN_LIMIT_ELECTRA: 128000000000 # 2**7 * 10**9 (= 128,000,000,000)

configs/minimal.yaml

@@ -156,11 +156,13 @@ WHISK_PROPOSER_SELECTION_GAP: 1
 # EIP7594
 NUMBER_OF_COLUMNS: 128
 MAX_CELLS_IN_EXTENDED_MATRIX: 768
-DATA_COLUMN_SIDECAR_SUBNET_COUNT: 32
+DATA_COLUMN_SIDECAR_SUBNET_COUNT: 128
 MAX_REQUEST_DATA_COLUMN_SIDECARS: 16384
-SAMPLES_PER_SLOT: 8
-CUSTODY_REQUIREMENT: 1
-TARGET_NUMBER_OF_PEERS: 70
+SAMPLES_PER_SLOT: 16
+CUSTODY_REQUIREMENT: 4
+VALIDATOR_CUSTODY_REQUIREMENT: 8
+BALANCE_PER_ADDITIONAL_CUSTODY_SUBNET: 32000000000 # 2**5 * 10**9 (= 32,000,000,000)
+TARGET_NUMBER_OF_PEERS: 100
 
 # [New in Electra:EIP7251]
 MIN_PER_EPOCH_CHURN_LIMIT_ELECTRA: 64000000000 # 2**6 * 10**9 (= 64,000,000,000)

pysetup/spec_builders/eip7594.py

@@ -12,6 +12,15 @@ def imports(cls, preset_name: str):
         return f'''
 from eth2spec.deneb import {preset_name} as deneb
 '''
+
+
+    @classmethod
+    def sundry_functions(cls) -> str:
+        return """
+def retrieve_column_sidecars(beacon_block_root: Root,
+                             require_peer_sampling: bool) -> Sequence[DataColumnSidecar]:
+    return []
+"""
 
     @classmethod
     def hardcoded_custom_type_dep_constants(cls, spec_object) -> str:
@@ -27,3 +36,5 @@ def hardcoded_func_dep_presets(cls, spec_object) -> Dict[str, str]:
         return {
             'KZG_COMMITMENTS_INCLUSION_PROOF_DEPTH': spec_object.preset_vars['KZG_COMMITMENTS_INCLUSION_PROOF_DEPTH'].value,
         }
+
+

specs/_features/eip7594/das-core.md

@@ -73,15 +73,18 @@ We define the following Python custom types for type hinting and readability:
 
 | Name | Value | Description |
 | - | - | - |
-| `DATA_COLUMN_SIDECAR_SUBNET_COUNT` | `32` | The number of data column sidecar subnets used in the gossipsub protocol |
+| `DATA_COLUMN_SIDECAR_SUBNET_COUNT` | `128` | The number of data column sidecar subnets used in the gossipsub protocol |
 
 ### Custody setting
 
 | Name | Value | Description |
 | - | - | - |
-| `SAMPLES_PER_SLOT` | `8` | Number of `DataColumn` random samples a node queries per slot |
-| `CUSTODY_REQUIREMENT` | `1` | Minimum number of subnets an honest node custodies and serves samples from |
-| `TARGET_NUMBER_OF_PEERS` | `70` | Suggested minimum peer count |
+| `SAMPLES_PER_SLOT` | `16` | Number of `DataColumn` random samples a node queries per slot |
+| `CUSTODY_REQUIREMENT` | `4` | Minimum number of subnets an honest node custodies and serves samples from |
+| `VALIDATOR_CUSTODY_REQUIREMENT` | `8` | Minimum number of subnets an honest node with validators attached custodies and serves samples from |
+| `BALANCE_PER_ADDITIONAL_CUSTODY_SUBNET` | `Gwei(32 * 10**9)` | Balance increment corresponding to one additional subnet to custody |
+| `TARGET_NUMBER_OF_PEERS` | `100` | Suggested minimum peer count |
+
 
 ### Containers
 
@@ -201,7 +204,16 @@ def get_data_column_sidecars(signed_block: SignedBeaconBlock,
 
 ### Custody requirement
 
-Each node downloads and custodies a minimum of `CUSTODY_REQUIREMENT` subnets per slot. The particular subnets that the node is required to custody are selected pseudo-randomly (more on this below).
+Each node *without attached validators* downloads and custodies a minimum of `CUSTODY_REQUIREMENT` subnets per slot. A node with validators attached downloads and custodies a higher minimum of subnets per slot, determined by `get_validators_custody_requirement(state, validator_indices)`. Here, `state` is the current `BeaconState` and `validator_indices` is the list of indices corresponding to validators attached to the node. Any node with at least one validator attached, and with the sum of the balances of all attached validators being `total_node_balance`, downloads and custodies `total_node_balance // BALANCE_PER_ADDITIONAL_CUSTODY_SUBNET` subnets per slot, with a minimum of `VALIDATOR_CUSTODY_REQUIREMENT` and of course a maximum of `DATA_COLUMN_SIDECAR_SUBNET_COUNT`.
+
+```python
+def get_validators_custody_requirement(state: BeaconState, validator_indices: Sequence[ValidatorIndex]) -> uint64:
+    total_node_balance = sum(state.balances[index] for index in validator_indices)
+    count = total_node_balance // BALANCE_PER_ADDITIONAL_CUSTODY_SUBNET
+    return min(max(count, VALIDATOR_CUSTODY_REQUIREMENT), DATA_COLUMN_SIDECAR_SUBNET_COUNT)
+```
+
+The particular subnets that the node is required to custody are selected pseudo-randomly (more on this below).
 
 A node *may* choose to custody and serve more than the minimum honesty requirement. Such a node explicitly advertises a number greater than `CUSTODY_REQUIREMENT` via the peer discovery mechanism -- for example, in their ENR (e.g. `custody_subnet_count: 4` if the node custodies `4` subnets each slot) -- up to a `DATA_COLUMN_SIDECAR_SUBNET_COUNT` (i.e. a super-full node).
 

specs/_features/eip7594/fork-choice.md

@@ -0,0 +1,193 @@
+# EIP-7594 -- Fork Choice
+
+## Table of contents
+<!-- TOC -->
+<!-- START doctoc generated TOC please keep comment here to allow auto update -->
+<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
+
+- [Introduction](#introduction)
+  - [Helpers](#helpers)
+    - [`get_custody_parameters`](#get_custody_parameters)
+    - [`get_sampling_columns`](#get_sampling_columns)
+    - [`retrieve_column_sidecars`](#retrieve_column_sidecars)
+    - [`is_data_available`](#is_data_available)
+    - [`is_chain_available`](#is_chain_available)
+    - [`get_head`](#get_head)
+    - [`is_peer_sampling_required`](#is_peer_sampling_required)
+- [Updated fork-choice handlers](#updated-fork-choice-handlers)
+  - [`on_block`](#on_block)
+    - [Pull-up tip helpers](#pull-up-tip-helpers)
+      - [`compute_pulled_up_tip`](#compute_pulled_up_tip)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+<!-- /TOC -->
+
+## Introduction
+
+This is the modification of the fork choice accompanying EIP-7594
+
+### Helpers
+
+#### `get_custody_parameters`
+
+`def get_custody_parameters() -> Tuple[NodeID, uint64]`
+
+#### `get_sampling_columns`
+
+
+`def get_sampling_columns() -> Sequence[ColumnIndex]`
+
+
+#### `retrieve_column_sidecars`
+
+`def retrieve_column_sidecars(beacon_block_root: Root, columns_to_retrieve: Sequence[ColumnIndex]) -> Sequence[DataColumnSidecar]`
+
+#### `is_data_available`
+
+```python
+def is_data_available(beacon_block_root: Root, require_peer_sampling: bool=False) -> bool:
+    column_sidecars = retrieve_column_sidecars(beacon_block_root, require_peer_sampling)
+    return all(
+        verify_data_column_sidecar_kzg_proofs(column_sidecar)
+        for column_sidecar in column_sidecars
+    )
+```
+
+#### `is_chain_available`
+
+```python
+def is_chain_available(store: Store, beacon_block_root: Root) -> bool: 
+    if beacon_block_root not in store.blocks:
+        # Deal with Genesis edge cases, where current_justified_checkpoint and parent_root are not set
+        return True
+    block = store.blocks[beacon_block_root]
+    block_epoch = compute_epoch_at_slot(block.slot)
+    current_epoch = get_current_store_epoch(store)
+    if block_epoch + MIN_EPOCHS_FOR_DATA_COLUMN_SIDECARS_REQUESTS <= current_epoch:
+        return True
+    parent_root = block.parent_root
+    return (
+        is_data_available(beacon_block_root, require_peer_sampling=True) 
+        and is_chain_available(store, parent_root)
+    )
+
+```
+
+#### `get_head`
+
+```python
+def get_head(store: Store) -> Root:
+    # Get filtered block tree that only includes viable branches
+    blocks = get_filtered_block_tree(store)
+    # Execute the LMD-GHOST fork choice
+    head = store.justified_checkpoint.root
+    while True:
+        # Get available children for the current slot
+        children = [
+            root for (root, block) in blocks.items()
+            if (
+                block.parent_root == head
+                and is_data_available(
+                    root, 
+                    require_peer_sampling=is_peer_sampling_required(store, block.slot)
+                )
+            )
+        ]
+        if len(children) == 0:
+            return head
+        # Sort by latest attesting balance with ties broken lexicographically
+        # Ties broken by favoring block with lexicographically higher root
+        head = max(children, key=lambda root: (get_weight(store, root), root))
+```
+
+#### `is_peer_sampling_required`
+
+```python
+def is_peer_sampling_required(store: Store, slot: Slot):
+    return compute_epoch_at_slot(slot) + 2 <= get_current_store_epoch(store)
+```
+
+## Updated fork-choice handlers
+
+### `on_block`
+
+*Note*: The blob data availability check is removed and replaced with an availability
+check on the on the justified checkpoint in the "pulled up state" of the block, after
+applying `process_justification_and_finalization`.
+
+```python
+def on_block(store: Store, signed_block: SignedBeaconBlock) -> None:
+    """
+    Run ``on_block`` upon receiving a new block.
+    """
+    block = signed_block.message
+    # Parent block must be known
+    assert block.parent_root in store.block_states
+    # Make a copy of the state to avoid mutability issues
+    state = copy(store.block_states[block.parent_root])
+    # Blocks cannot be in the future. If they are, their consideration must be delayed until they are in the past.
+    assert get_current_slot(store) >= block.slot
+
+    # Check that block is later than the finalized epoch slot (optimization to reduce calls to get_ancestor)
+    finalized_slot = compute_start_slot_at_epoch(store.finalized_checkpoint.epoch)
+    assert block.slot > finalized_slot
+    # Check block is a descendant of the finalized block at the checkpoint finalized slot
+    finalized_checkpoint_block = get_checkpoint_block(
+        store,
+        block.parent_root,
+        store.finalized_checkpoint.epoch,
+    )
+    assert store.finalized_checkpoint.root == finalized_checkpoint_block
+
+    # Check the block is valid and compute the post-state
+    block_root = hash_tree_root(block)
+    state_transition(state, signed_block, True)
+
+    # [New in EIP7594] Do not import the block if its unrealized justified checkpoint is not available
+    pulled_up_state = state.copy()
+    process_justification_and_finalization(pulled_up_state)
+    assert is_chain_available(store, pulled_up_state.current_justified_checkpoint.root)
+
+    # Add new block to the store
+    store.blocks[block_root] = block
+    # Add new state for this block to the store
+    store.block_states[block_root] = state
+
+    # Add block timeliness to the store
+    time_into_slot = (store.time - store.genesis_time) % SECONDS_PER_SLOT
+    is_before_attesting_interval = time_into_slot < SECONDS_PER_SLOT // INTERVALS_PER_SLOT
+    is_timely = get_current_slot(store) == block.slot and is_before_attesting_interval
+    store.block_timeliness[hash_tree_root(block)] = is_timely
+
+    # Add proposer score boost if the block is timely and not conflicting with an existing block
+    is_first_block = store.proposer_boost_root == Root()
+    if is_timely and is_first_block:
+        store.proposer_boost_root = hash_tree_root(block)
+
+    # Update checkpoints in store if necessary
+    update_checkpoints(store, state.current_justified_checkpoint, state.finalized_checkpoint)
+
+    # Eagerly compute unrealized justification and finality.
+    compute_pulled_up_tip(store, pulled_up_state, block_root)
+```
+
+#### Pull-up tip helpers
+
+##### `compute_pulled_up_tip`
+
+Modified to take `pulled_up_state`, the block's state after applying `processing_justification_and_finalization`.
+The application of `processing_justification_and_finalization` now happens in `on_block`.
+
+```python
+def compute_pulled_up_tip(store: Store, pulled_up_state: BeaconState, block_root: Root) -> None:
+    store.unrealized_justifications[block_root] = pulled_up_state.current_justified_checkpoint
+    unrealized_justified = pulled_up_state.current_justified_checkpoint
+    unrealized_finalized = pulled_up_state.finalized_checkpoint
+    update_unrealized_checkpoints(store, unrealized_justified, unrealized_finalized)
+
+    # If the block is from a prior epoch, apply the realized values
+    block_epoch = compute_epoch_at_slot(store.blocks[block_root].slot)
+    current_epoch = get_current_store_epoch(store)
+    if block_epoch < current_epoch:
+        update_checkpoints(store, unrealized_justified, unrealized_finalized)
+```

tests/core/pyspec/eth2spec/test/deneb/fork_choice/test_on_block.py

@@ -2,9 +2,11 @@
 
 from eth2spec.test.context import (
     spec_state_test,
-    with_deneb_and_later,
+    with_phases,
 )
 
+from eth2spec.test.helpers.constants import DENEB
+
 from eth2spec.test.helpers.block import (
     build_empty_block_for_next_slot,
 )
@@ -34,7 +36,7 @@ def get_block_with_blob(spec, state, rng=None):
     return block, blobs, blob_kzg_proofs
 
 
-@with_deneb_and_later
+@with_phases([DENEB])
 @spec_state_test
 def test_simple_blob_data(spec, state):
     rng = Random(1234)
@@ -69,7 +71,7 @@ def test_simple_blob_data(spec, state):
     yield 'steps', test_steps
 
 
-@with_deneb_and_later
+@with_phases([DENEB])
 @spec_state_test
 def test_invalid_incorrect_proof(spec, state):
     rng = Random(1234)
@@ -97,7 +99,7 @@ def test_invalid_incorrect_proof(spec, state):
     yield 'steps', test_steps
 
 
-@with_deneb_and_later
+@with_phases([DENEB])
 @spec_state_test
 def test_invalid_data_unavailable(spec, state):
     rng = Random(1234)
@@ -125,7 +127,7 @@ def test_invalid_data_unavailable(spec, state):
     yield 'steps', test_steps
 
 
-@with_deneb_and_later
+@with_phases([DENEB])
 @spec_state_test
 def test_invalid_wrong_proofs_length(spec, state):
     rng = Random(1234)
@@ -153,7 +155,7 @@ def test_invalid_wrong_proofs_length(spec, state):
     yield 'steps', test_steps
 
 
-@with_deneb_and_later
+@with_phases([DENEB])
 @spec_state_test
 def test_invalid_wrong_blobs_length(spec, state):
     rng = Random(1234)