Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DRAFT] Increase MAX_EFFECTIVE_BALANCE minimal spec change #3

Closed
wants to merge 100 commits into from
Closed
Changes from 23 commits
Commits
Show all changes
100 commits
Select commit Hold shift + click to select a range
0b8bb11
mvp
michaelneuder May 22, 2023
6809439
partial withdrawals
michaelneuder May 22, 2023
2b1c7b8
top ups
michaelneuder May 22, 2023
6824f17
top ups
michaelneuder May 22, 2023
67cc34f
top ups
michaelneuder May 22, 2023
1403bdb
all top ups capped at 32 ETH
michaelneuder May 22, 2023
cb49250
simplify top-up
fradamt May 22, 2023
81f9080
update get_expected_withdrawals
michaelneuder May 24, 2023
3575804
Merge branch 'maxeb-pyspec-min-viable' of https://github.com/michaeln…
michaelneuder May 24, 2023
95a167d
ceiling function
michaelneuder May 25, 2023
604b876
remove proposer slashing modification –– permature optimizaiton IMO
michaelneuder May 25, 2023
552cb21
rm MIN_PROPOSER_SLASHING
michaelneuder May 25, 2023
fdcb6bd
balance - ceiling
michaelneuder May 25, 2023
2e536a4
justin round 1
michaelneuder May 26, 2023
2526b9d
remove virtual
michaelneuder May 26, 2023
b6c476f
remove whitespace
michaelneuder May 26, 2023
4681d67
min_balance_incrememts
michaelneuder May 26, 2023
921e898
min_balance_increments
michaelneuder May 26, 2023
b6d5f07
compute_weak_subjectivity_period
michaelneuder May 26, 2023
8e56d72
justin rd 2 comments
michaelneuder May 29, 2023
7b5ad56
justin rd 2 comments
michaelneuder May 29, 2023
7e1408c
new initiate_validator_exit logic
michaelneuder May 29, 2023
4b1ee30
new activation and exit logic
michaelneuder May 29, 2023
2ad9633
justin rd 3 :-)
michaelneuder May 30, 2023
d6db57c
rollback
michaelneuder May 30, 2023
bcd6164
update with pending deposits rate limited by churn
michaelneuder Jun 19, 2023
2955027
make pending balance deposit a list
michaelneuder Jun 19, 2023
1577082
dapplion response
michaelneuder Jun 19, 2023
6f8c3e1
small fix
michaelneuder Jun 19, 2023
9ae94fc
cache compute_activation_exit_epoch
michaelneuder Jun 19, 2023
1e487dc
custom ceilings
michaelneuder Jun 20, 2023
2cf9b52
dapplion response
michaelneuder Jun 20, 2023
6de5355
remove duplicate is_partially_withdrawable_validator
michaelneuder Jun 20, 2023
e7e8f20
adding topups that modify the balance ceiling
michaelneuder Jun 20, 2023
02e1bac
no ceilings
michaelneuder Jun 28, 2023
40432e8
pending withdrawals
michaelneuder Jul 3, 2023
bc0a782
pull exit queue stuff
michaelneuder Jul 3, 2023
24871ec
remove old state variable
michaelneuder Jul 3, 2023
b592974
adding withdrawal delay
michaelneuder Jul 3, 2023
1aaf390
get_expected_withdrawals
michaelneuder Jul 3, 2023
bb61d09
is_complete
michaelneuder Jul 3, 2023
587542e
withdrawable_epoch
michaelneuder Jul 3, 2023
5f52180
withdrawable_epoch
michaelneuder Jul 3, 2023
0e2228a
FAR_FUTURE_EPOCH
michaelneuder Jul 3, 2023
8a557ae
get_expected_withdrawals
michaelneuder Jul 3, 2023
c063473
remove unused local
michaelneuder Jul 3, 2023
9c141fc
fix typos
fradamt Jul 3, 2023
88f650b
- Fix typos and small errors
fradamt Jul 3, 2023
a39d578
cancel pending withdrawals
michaelneuder Jul 4, 2023
022deb9
Add process_execution_layer_withdraw_request
dapplion Jul 4, 2023
b57a7ce
Handle slashings in exit withdrawal queue
mkalinin Jul 5, 2023
ec33407
Remove unprocessed exit upon slashing
mkalinin Jul 5, 2023
63b3633
Do not break if withdrawable epoch is not yet come
mkalinin Jul 5, 2023
ccf5acf
Merge pull request #5 from michaelneuder/exit-withdrawal-queue
michaelneuder Jul 5, 2023
f73a996
two queues
michaelneuder Jul 5, 2023
82d49cf
two queues
michaelneuder Jul 5, 2023
986e6ad
three queue:
michaelneuder Jul 5, 2023
f4f6538
three queue
michaelneuder Jul 5, 2023
7801942
Implement exit churn with no physical queue, add partial withdrawals
mkalinin Jul 5, 2023
fd8aa04
Fix consumed update
mkalinin Jul 5, 2023
f23f77c
Fix expected withdrawals
mkalinin Jul 5, 2023
a6c62be
Polish names
mkalinin Jul 5, 2023
3e927cf
Skip partial withdrawals if validator exited
mkalinin Jul 5, 2023
f87def1
Remove unused fields and functions
mkalinin Jul 5, 2023
fa8f2ee
Cleanup get_expected_withdrawals
mkalinin Jul 5, 2023
b489655
Limit partial withdrawals to MAX_WITHDRAWALS_PER_PAYLOAD // 2
mkalinin Jul 5, 2023
6f4750d
Merge pull request #6 from michaelneuder/queueless-exit-churn
michaelneuder Jul 6, 2023
ccc0aae
compute_exit_epoch_and_update_churn tweak
michaelneuder Jul 6, 2023
f7b74c3
fix consumed increment indentation level
michaelneuder Jul 6, 2023
ff9197c
add decrease_balance
michaelneuder Jul 6, 2023
31e056e
rm decrease_balance
michaelneuder Jul 6, 2023
d253ead
Revert changes in slash_validator
mkalinin Jul 7, 2023
be4797b
remove unused field, `withdrawal_balance_to_consume`
fradamt Jul 7, 2023
29066f3
Constant penalty for proposer equivocations
dapplion Jul 12, 2023
4c12835
Split slashable status
dapplion Jul 13, 2023
dcf1b16
Add is_attester_slashable_validator
dapplion Jul 14, 2023
d81d771
Don't allow is_slashed_attester to propose blocks
dapplion Jul 20, 2023
2ebf61a
Merge pull request #7 from dapplion/proposer-equivocation-penalty
michaelneuder Aug 1, 2023
a278951
Update specs/_features/maxeb_increase/capella.py
michaelneuder Aug 1, 2023
f398b45
Update EB processing on epoch boundary
mkalinin Aug 15, 2023
6b9351d
Update specs/_features/maxeb_increase/capella.py
mkalinin Aug 18, 2023
48f4c4d
Merge pull request #11 from michaelneuder/mkalinin-patch-1
michaelneuder Aug 18, 2023
b9a0284
Apply suggestions from code review
mkalinin Dec 13, 2023
026be15
consolidation by moving balance upon exit
fradamt Jan 16, 2024
7e9292b
Revert "Constant penalty for proposer equivocations"
mkalinin Jan 17, 2024
c9f1f50
Merge pull request #15 from michaelneuder/revert-7-proposer-equivocat…
michaelneuder Jan 17, 2024
f577511
separate consolidation churn from exit churn
fradamt Jan 17, 2024
857b0e0
Make initial slashing penalty negligible
mkalinin Jan 19, 2024
d6cec95
Set consolidation churn to leftover from activation and exit churn
fradamt Jan 19, 2024
89f15df
Update specs/_features/maxeb_increase/capella.py
mkalinin Jan 23, 2024
13f93ee
Update specs/_features/maxeb_increase/capella.py
mkalinin Jan 23, 2024
71fd7a8
Set MAX_CONSOLIDATIONS to 1
mkalinin Jan 23, 2024
419baf2
Polish process_pending_consolidations
mkalinin Jan 23, 2024
53582d0
Merge pull request #14 from fradamt/consolidate-through-exit
mkalinin Jan 24, 2024
55c50d4
fix issue in apply_deposit
fradamt Jan 24, 2024
93096e2
Merge branch 'maxeb-pyspec-min-viable' into mkalinin-patch-2
mkalinin Jan 25, 2024
c242364
small fixes
fradamt Jan 25, 2024
e9ad4dc
Merge pull request #16 from michaelneuder/mkalinin-patch-2
mkalinin Jan 25, 2024
5d134a2
Apply suggestions from code review
mkalinin Jan 29, 2024
fe6d697
Update specs/_features/maxeb_increase/capella.py
mkalinin Jan 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 58 additions & 21 deletions specs/_features/maxeb_increase/capella.py
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,7 @@ def floorlog2(x: int) -> uint64:
ENDIANNESS: Final = 'little'
BLS_WITHDRAWAL_PREFIX = Bytes1('0x00')
ETH1_ADDRESS_WITHDRAWAL_PREFIX = Bytes1('0x01')
COMPOUNDING_WITHDRAWAL_PREFIX = Bytes1('0x02')
DOMAIN_BEACON_PROPOSER = DomainType('0x00000000')
DOMAIN_BEACON_ATTESTER = DomainType('0x01000000')
DOMAIN_RANDAO = DomainType('0x02000000')
Expand Down Expand Up @@ -190,7 +191,8 @@ def floorlog2(x: int) -> uint64:
HYSTERESIS_DOWNWARD_MULTIPLIER = uint64(1)
HYSTERESIS_UPWARD_MULTIPLIER = uint64(5)
MIN_DEPOSIT_AMOUNT = Gwei(1000000000)
MAX_EFFECTIVE_BALANCE = Gwei(32000000000)
MIN_ACTIVATION_BALANCE = Gwei(32000000000)
michaelneuder marked this conversation as resolved.
Show resolved Hide resolved
MAX_EFFECTIVE_BALANCE = Gwei(2048000000000)
EFFECTIVE_BALANCE_INCREMENT = Gwei(1000000000)
MIN_ATTESTATION_INCLUSION_DELAY = uint64(1)
SLOTS_PER_EPOCH = uint64(32)
Expand Down Expand Up @@ -656,6 +658,8 @@ class BeaconState(Container):
# Registry
validators: List[Validator, VALIDATOR_REGISTRY_LIMIT]
balances: List[Gwei, VALIDATOR_REGISTRY_LIMIT]
activation_validator_balance: Gwei
JustinDrake marked this conversation as resolved.
Show resolved Hide resolved
exit_queue_churn: Gwei
JustinDrake marked this conversation as resolved.
Show resolved Hide resolved
# Randomness
randao_mixes: Vector[Bytes32, EPOCHS_PER_HISTORICAL_VECTOR]
# Slashings
Expand Down Expand Up @@ -798,7 +802,7 @@ def is_eligible_for_activation_queue(validator: Validator) -> bool:
"""
return (
validator.activation_eligibility_epoch == FAR_FUTURE_EPOCH
and validator.effective_balance == MAX_EFFECTIVE_BALANCE
and validator.effective_balance >= MIN_ACTIVATION_BALANCE
)


Expand Down Expand Up @@ -1021,12 +1025,11 @@ def get_active_validator_indices(state: BeaconState, epoch: Epoch) -> Sequence[V
return [ValidatorIndex(i) for i, v in enumerate(state.validators) if is_active_validator(v, epoch)]


def get_validator_churn_limit(state: BeaconState) -> uint64:
def get_validator_churn_limit(state: BeaconState) -> Gwei:
"""
Return the validator churn limit for the current epoch.
"""
active_validator_indices = get_active_validator_indices(state, get_current_epoch(state))
return max(config.MIN_PER_EPOCH_CHURN_LIMIT, uint64(len(active_validator_indices)) // config.CHURN_LIMIT_QUOTIENT)
return max(config.MIN_PER_EPOCH_CHURN_LIMIT * MIN_ACTIVATION_BALANCE, get_total_active_balance(state) // config.CHURN_LIMIT_QUOTIENT)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the engineering on this on Lodestar, the churn not being align with EFFECTIVE_BALANCE_INCREMENT would break multiple optimizations we have. Due to JS 53 bit representation of stack integers we treat account weights denominated in EFFECTIVE_BALANCE_INCREMENT instead of GWei so the aggregated network balances fit. Aligning the churn to that value should produce a marginal accounting difference in the overall churn so I would really push for this extra line

Suggested change
return max(config.MIN_PER_EPOCH_CHURN_LIMIT * MIN_ACTIVATION_BALANCE, get_total_active_balance(state) // config.CHURN_LIMIT_QUOTIENT)
churn = max(config.MIN_PER_EPOCH_CHURN_LIMIT * MIN_ACTIVATION_BALANCE, get_total_active_balance(state) // config.CHURN_LIMIT_QUOTIENT)
return churn - churn % EFFECTIVE_BALANCE_INCREMENT

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done!



def get_seed(state: BeaconState, epoch: Epoch, domain_type: DomainType) -> Bytes32:
Expand Down Expand Up @@ -1146,10 +1149,17 @@ def initiate_validator_exit(state: BeaconState, index: ValidatorIndex) -> None:
# Compute exit queue epoch
exit_epochs = [v.exit_epoch for v in state.validators if v.exit_epoch != FAR_FUTURE_EPOCH]
exit_queue_epoch = max(exit_epochs + [compute_activation_exit_epoch(get_current_epoch(state))])
exit_queue_churn = len([v for v in state.validators if v.exit_epoch == exit_queue_epoch])
if exit_queue_churn >= get_validator_churn_limit(state):
exit_balance_to_consume = validator.effective_balance
per_epoch_churn_limit = get_validator_churn_limit(state)
if state.exit_queue_churn + exit_balance_to_consume <= per_epoch_churn_limit:
state.exit_queue_churn += exit_balance_to_consume
else: # Exit balance rolls over to subsequent epoch(s)
exit_balance_to_consume -= (per_epoch_churn_limit - state.exit_queue_churn)
exit_queue_epoch += Epoch(1)

while exit_balance_to_consume >= per_epoch_churn_limit:
exit_balance_to_consume -= per_epoch_churn_limit
exit_queue_epoch += Epoch(1)
state.exit_queue_churn = exit_balance_to_consume
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The while loop here looks unnecessary, why not do a integer division and modulus operation instead?

Suggested change
while exit_balance_to_consume >= per_epoch_churn_limit:
exit_balance_to_consume -= per_epoch_churn_limit
exit_queue_epoch += Epoch(1)
state.exit_queue_churn = exit_balance_to_consume
additional_epochs, state.exit_queue_churn = divmod(exit_balance_to_consume - (per_epoch_churn_limit - state.exit_queue_churn), per_epoch_churn_limit)
exit_queue_epoch += Epoch(additional_epochs + 1)

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good call! this is clearer thanks :-)

# Set validator exit epoch and withdrawable epoch
validator.exit_epoch = exit_queue_epoch
validator.withdrawable_epoch = Epoch(validator.exit_epoch + config.MIN_VALIDATOR_WITHDRAWABILITY_DELAY)
Expand Down Expand Up @@ -1209,7 +1219,7 @@ def initialize_beacon_state_from_eth1(eth1_block_hash: Hash32,
for index, validator in enumerate(state.validators):
balance = state.balances[index]
validator.effective_balance = min(balance - balance % EFFECTIVE_BALANCE_INCREMENT, MAX_EFFECTIVE_BALANCE)
if validator.effective_balance == MAX_EFFECTIVE_BALANCE:
if validator.effective_balance >= MIN_ACTIVATION_BALANCE:
validator.activation_eligibility_epoch = GENESIS_EPOCH
validator.activation_epoch = GENESIS_EPOCH

Expand Down Expand Up @@ -1548,9 +1558,17 @@ def process_registry_updates(state: BeaconState) -> None:
# Order by the sequence of activation_eligibility_epoch setting and then index
], key=lambda index: (state.validators[index].activation_eligibility_epoch, index))
# Dequeued validators for activation up to churn limit
for index in activation_queue[:get_validator_churn_limit(state)]:
activation_balance_to_consume = get_validator_churn_limit(state)
for index in activation_queue:
validator = state.validators[index]
validator.activation_epoch = compute_activation_exit_epoch(get_current_epoch(state))
# Validator can now be activated
if state.activation_validator_balance + activation_balance_to_consume >= validator.effective_balance:
activation_balance_to_consume -= (validator.effective_balance - state.activation_validator_balance)
state.activation_validator_balance = Gwei(0)
validator.activation_epoch = compute_activation_exit_epoch(get_current_epoch(state))
else:
state.activation_validator_balance += activation_balance_to_consume
break
michaelneuder marked this conversation as resolved.
Show resolved Hide resolved


def process_slashings(state: BeaconState) -> None:
Expand Down Expand Up @@ -1794,9 +1812,9 @@ def apply_deposit(state: BeaconState,
state.current_epoch_participation.append(ParticipationFlags(0b0000_0000))
state.inactivity_scores.append(uint64(0))
else:
# Increase balance by deposit amount
# Increase balance by deposit amount, up to MIN_ACTIVATION_BALANCE
index = ValidatorIndex(validator_pubkeys.index(pubkey))
increase_balance(state, index, amount)
increase_balance(state, index, min(amount, MIN_ACTIVATION_BALANCE - state.balances[index]))

JustinDrake marked this conversation as resolved.
Show resolved Hide resolved

def process_deposit(state: BeaconState, deposit: Deposit) -> None:
Expand Down Expand Up @@ -2334,9 +2352,14 @@ def get_slot_signature(state: BeaconState, slot: Slot, privkey: int) -> BLSSigna
return bls.Sign(privkey, signing_root)


def is_aggregator(state: BeaconState, slot: Slot, index: CommitteeIndex, slot_signature: BLSSignature) -> bool:
def is_aggregator(state: BeaconState, slot: Slot, index: CommitteeIndex, validator_index: ValidatorIndex, slot_signature: BLSSignature) -> bool:
validator = state.validators[validator_index]
committee = get_beacon_committee(state, slot, index)
modulo = max(1, len(committee) // TARGET_AGGREGATORS_PER_COMMITTEE)
min_balance_increments = validator.effective_balance // MIN_ACTIVATION_BALANCE
committee_balance = get_total_balance(state, set(committee))
denominator = committee_balance ** min_balance_increments
numerator = denominator - (committee_balance - TARGET_AGGREGATORS_PER_COMMITTEE * MIN_ACTIVATION_BALANCE) ** min_balance_increments
michaelneuder marked this conversation as resolved.
Show resolved Hide resolved
michaelneuder marked this conversation as resolved.
Show resolved Hide resolved
modulo = denominator // numerator
return bytes_to_uint64(hash(slot_signature)[0:8]) % modulo == 0


Expand Down Expand Up @@ -2378,7 +2401,7 @@ def compute_weak_subjectivity_period(state: BeaconState) -> uint64:
N = len(get_active_validator_indices(state, get_current_epoch(state)))
t = get_total_active_balance(state) // N // ETH_TO_GWEI
T = MAX_EFFECTIVE_BALANCE // ETH_TO_GWEI
delta = get_validator_churn_limit(state)
delta = get_validator_churn_limit(state) // MIN_ACTIVATION_BALANCE
Delta = MAX_DEPOSITS * SLOTS_PER_EPOCH
D = SAFETY_DECAY

Expand Down Expand Up @@ -3540,24 +3563,38 @@ def has_eth1_withdrawal_credential(validator: Validator) -> bool:
return validator.withdrawal_credentials[:1] == ETH1_ADDRESS_WITHDRAWAL_PREFIX


def has_compounding_withdrawal_credential(validator: Validator) -> bool:
"""
Check if ``validator`` has an 0x02 prefixed "compounding" withdrawal credential.
"""
return validator.withdrawal_credentials[:1] == COMPOUNDING_WITHDRAWAL_PREFIX


def is_fully_withdrawable_validator(validator: Validator, balance: Gwei, epoch: Epoch) -> bool:
"""
Check if ``validator`` is fully withdrawable.
"""
return (
has_eth1_withdrawal_credential(validator)
(has_eth1_withdrawal_credential(validator) or has_compounding_withdrawal_credential(validator))
and validator.withdrawable_epoch <= epoch
and balance > 0
)

def get_validator_excess_balance(validator: Validator, balance: Gwei) -> Gwei:
"""
Get excess balance for partial withdrawals for ``validator``.
"""
if has_compounding_withdrawal_credential(validator) and balance > MAX_EFFECTIVE_BALANCE:
return balance - MAX_EFFECTIVE_BALANCE
elif has_eth1_withdrawal_credential(validator) and balance > MIN_ACTIVATION_BALANCE:
return balance - MIN_ACTIVATION_BALANCE
return Gwei(0)

def is_partially_withdrawable_validator(validator: Validator, balance: Gwei) -> bool:
"""
Check if ``validator`` is partially withdrawable.
"""
has_max_effective_balance = validator.effective_balance == MAX_EFFECTIVE_BALANCE
Copy link

@avsetsin avsetsin Mar 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the check has_max_effective_balance was removed by mistake.

There may be a situation where a validator has lost 0.25 eth below MAX_EFFECTIVE_BALANCE resulting in its effective balance decreasing by 1 eth. The validator could then bring the balance up to MAX_EFFECTIVE_BALANCE or higher, but less than MAX_EFFECTIVE_BALANCE + 0.25 eth (the value needed to return the effective balance to MAX_EFFECTIVE_BALANCE).

The current code allows partial withdrawals of the validator balance when effective balance < MAX_EFFECTIVE_BALANCE, because of which it is very likely that the validator balance will not reach the necessary MAX_EFFECTIVE_BALANCE + 0.25 eth to return its effective balance to equal MAX_EFFECTIVE_BALANCE.

The same is true for 0x01 type and MIN_ACTIVATION_BALANCE

def is_partially_withdrawable_validator(validator: Validator, balance: Gwei) -> bool:
    """
    Check if ``validator`` is partially withdrawable.
    """
    if has_eth1_withdrawal_credential(validator) and validator.effective_balance == MIN_ACTIVATION_BALANCE:
        return get_validator_excess_balance(validator, balance) > 0
    if has_compounding_withdrawal_credential(validator) and validator.effective_balance == MAX_EFFECTIVE_BALANCE:
        return get_validator_excess_balance(validator, balance) > 0
    return False

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@avsetsin Please can you port your review comments to ethereum#3618 ?

has_excess_balance = balance > MAX_EFFECTIVE_BALANCE
return has_eth1_withdrawal_credential(validator) and has_max_effective_balance and has_excess_balance
return get_validator_excess_balance(validator, balance) > 0


def process_historical_summaries_update(state: BeaconState) -> None:
Expand Down Expand Up @@ -3593,7 +3630,7 @@ def get_expected_withdrawals(state: BeaconState) -> Sequence[Withdrawal]:
index=withdrawal_index,
validator_index=validator_index,
address=ExecutionAddress(validator.withdrawal_credentials[12:]),
amount=balance - MAX_EFFECTIVE_BALANCE,
amount=get_validator_excess_balance(validator, balance),
))
withdrawal_index += WithdrawalIndex(1)
if len(withdrawals) == MAX_WITHDRAWALS_PER_PAYLOAD:
Expand Down