Separation of Constant, Preset and Configuration variables

.gitignore

@@ -29,3 +29,6 @@ tests/core/pyspec/test-reports
 tests/core/pyspec/eth2spec/test_results.xml
 
 *.egg-info
+
+# TOC tool outputs temporary files
+*.tmp

Makefile

@@ -96,11 +96,11 @@ install_test:
 
 test: pyspec
 	. venv/bin/activate; cd $(PY_SPEC_DIR); \
-	python3 -m pytest -n 4 --disable-bls --cov=eth2spec.phase0.spec --cov=eth2spec.altair.spec --cov-report="html:$(COV_HTML_OUT)" --cov-branch eth2spec
+	python3 -m pytest -n 4 --disable-bls --cov=eth2spec.phase0.mainnet --cov=eth2spec.altair.mainnet --cov-report="html:$(COV_HTML_OUT)" --cov-branch eth2spec
 
 find_test: pyspec
 	. venv/bin/activate; cd $(PY_SPEC_DIR); \
-	python3 -m pytest -k=$(K) --disable-bls --cov=eth2spec.phase0.spec --cov=eth2spec.altair.spec --cov-report="html:$(COV_HTML_OUT)" --cov-branch eth2spec
+	python3 -m pytest -k=$(K) --disable-bls --cov=eth2spec.phase0.mainnet --cov=eth2spec.altair.mainnet --cov-report="html:$(COV_HTML_OUT)" --cov-branch eth2spec
 
 citest: pyspec
 	mkdir -p tests/core/pyspec/test-reports/eth2spec; . venv/bin/activate; cd $(PY_SPEC_DIR); \
@@ -167,7 +167,7 @@ define run_generator
 	if ! test -d venv; then python3 -m venv venv; fi; \
 	. venv/bin/activate; \
 	pip3 install -r requirements.txt; \
-	python3 main.py -o $(CURRENT_DIR)/$(TEST_VECTOR_DIR) -c $(CURRENT_DIR)/$(CONFIGS_DIR); \
+	python3 main.py -o $(CURRENT_DIR)/$(TEST_VECTOR_DIR); \
 	echo "generator $(1) finished"
 endef
 

configs/README.md

@@ -1,34 +1,39 @@
-# Configs
+# Configurations
 
-This directory contains a set of constants presets used for testing, testnets, and mainnet.
+This directory contains a set of configurations used for testing, testnets, and mainnet.
+A client binary may be compiled for a specific `PRESET_BASE`, 
+and then load different configurations around that preset to participate in different networks or tests.
 
-A preset file contains all the constants known for its target.
-Later-fork constants can be ignored, e.g. ignore Sharding constants as a client that only supports Phase 0 currently.
+Standard configs:
+- [`mainnet.yaml`](./mainnet.yaml): Mainnet configuration
+- [`minimal.yaml`](./minimal.yaml): Minimal configuration, used in spec-testing along with the [`minimal`](../presets/minimal) preset.
 
+Not all network configurations are in scope for the specification,
+see [`github.com/eth2-clients/eth2-networks`](https://github.com/eth2-clients/eth2-networks) for common networks,
+and additional testnet assets.
 
 ## Forking
 
-Configs are not replaced, but extended with forks. This is to support syncing from one state to the other over a fork boundary, without hot-swapping a config.
-Instead, for forks that introduce changes in a constant, the constant name is prefixed with a short abbreviation of the fork.
+Variables are not replaced, but extended with forks. This is to support syncing from one state to the other over a fork boundary, without hot-swapping a config.
+Instead, for forks that introduce changes in a variable, the variable name is suffixed with the fork name, e.g. `INACTIVITY_PENALTY_QUOTIENT_ALTAIR`.
+
+Future-fork variables can be ignored, e.g. ignore Sharding variables as a client that only supports Phase 0 currently.
 
 Over time, the need to sync an older state may be deprecated.
-In this case, the prefix on the new constant may be removed, and the old constant will keep a special name before completely being removed.
+In this case, the suffix on the new variable may be removed, and the old variable will keep a special name before completely being removed.
 
-A previous iteration of forking made use of "timelines", but this collides with the definitions used in the spec (constants for special forking slots, etc.), and was not integrated sufficiently in any of the spec tools or implementations.
+A previous iteration of forking made use of "timelines", but this collides with the definitions used in the spec (variables for special forking slots, etc.), and was not integrated sufficiently in any of the spec tools or implementations.
 Instead, the config essentially doubles as fork definition now, e.g. changing the value for `ALTAIR_FORK_EPOCH` changes the fork.
-
-Another reason to prefer forking through constants is the ability to program a forking moment based on context, instead of being limited to a static slot number.
 
 ## Format
 
-Each preset is a key-value mapping.
+Each preset and configuration is a key-value mapping.
 
-**Key**: an `UPPER_SNAKE_CASE` (a.k.a. "macro case") formatted string, name of the constant.
+**Key**: an `UPPER_SNAKE_CASE` (a.k.a. "macro case") formatted string, name of the variable.
 
 **Value** can be either:
  - an unsigned integer number, can be up to 64 bits (incl.)
  - a hexadecimal string, prefixed with `0x`
 
-Presets may contain comments to describe the values.
-
-See [`mainnet/phase0.yaml`](./mainnet/phase0.yaml) for a complete example.
+This format is fully YAML compatible.
+The presets and configurations may contain comments to describe the values.

configs/mainnet.yaml

@@ -0,0 +1,71 @@
+# Mainnet config
+
+# Extends the mainnet preset
+PRESET_BASE: 'mainnet'
+
+# Genesis
+# ---------------------------------------------------------------
+# `2**14` (= 16,384)
+MIN_GENESIS_ACTIVE_VALIDATOR_COUNT: 16384
+# Dec 1, 2020, 12pm UTC
+MIN_GENESIS_TIME: 1606824000
+# Mainnet initial fork version, recommend altering for testnets
+GENESIS_FORK_VERSION: 0x00000000
+# 604800 seconds (7 days)
+GENESIS_DELAY: 604800
+
+
+# Forking
+# ---------------------------------------------------------------
+# Some forks are disabled for now:
+#  - These may be re-assigned to another fork-version later
+#  - Temporarily set to max uint64 value: 2**64 - 1
+
+# Altair
+ALTAIR_FORK_VERSION: 0x01000000
+ALTAIR_FORK_EPOCH: 18446744073709551615
+# Merge
+MERGE_FORK_VERSION: 0x02000000
+MERGE_FORK_EPOCH: 18446744073709551615
+# Sharding
+SHARDING_FORK_VERSION: 0x03000000
+SHARDING_FORK_EPOCH: 18446744073709551615
+
+# TBD, 2**32 is a placeholder. Merge transition approach is in active R&D.
+TRANSITION_TOTAL_DIFFICULTY: 4294967296
+
+
+# Time parameters
+# ---------------------------------------------------------------
+# 12 seconds
+SECONDS_PER_SLOT: 12
+# 14 (estimate from Eth1 mainnet)
+SECONDS_PER_ETH1_BLOCK: 14
+# 2**8 (= 256) epochs ~27 hours
+MIN_VALIDATOR_WITHDRAWABILITY_DELAY: 256
+# 2**8 (= 256) epochs ~27 hours
+SHARD_COMMITTEE_PERIOD: 256
+# 2**11 (= 2,048) Eth1 blocks ~8 hours
+ETH1_FOLLOW_DISTANCE: 2048
+
+
+# Validator cycle
+# ---------------------------------------------------------------
+# 2**2 (= 4)
+INACTIVITY_SCORE_BIAS: 4
+# 2**4 (= 16)
+INACTIVITY_SCORE_RECOVERY_RATE: 16
+# 2**4 * 10**9 (= 16,000,000,000) Gwei
+EJECTION_BALANCE: 16000000000
+# 2**2 (= 4)
+MIN_PER_EPOCH_CHURN_LIMIT: 4
+# 2**16 (= 65,536)
+CHURN_LIMIT_QUOTIENT: 65536
+
+
+# Deposit contract
+# ---------------------------------------------------------------
+# Ethereum PoW Mainnet
+DEPOSIT_CHAIN_ID: 1
+DEPOSIT_NETWORK_ID: 1
+DEPOSIT_CONTRACT_ADDRESS: 0x00000000219ab540356cBB839Cbe05303d7705Fa

configs/minimal.yaml

@@ -0,0 +1,71 @@
+# Minimal config
+
+# Extends the minimal preset
+PRESET_BASE: 'minimal'
+
+# Genesis
+# ---------------------------------------------------------------
+# [customized]
+MIN_GENESIS_ACTIVE_VALIDATOR_COUNT: 64
+# Jan 3, 2020
+MIN_GENESIS_TIME: 1578009600
+# Highest byte set to 0x01 to avoid collisions with mainnet versioning
+GENESIS_FORK_VERSION: 0x00000001
+# [customized] Faster to spin up testnets, but does not give validator reasonable warning time for genesis
+GENESIS_DELAY: 300
+
+
+# Forking
+# ---------------------------------------------------------------
+# Values provided for illustrative purposes.
+# Individual tests/testnets may set different values.
+
+# Altair
+ALTAIR_FORK_VERSION: 0x01000001
+ALTAIR_FORK_EPOCH: 18446744073709551615
+# Merge
+MERGE_FORK_VERSION: 0x02000001
+MERGE_FORK_EPOCH: 18446744073709551615
+# Sharding
+SHARDING_FORK_VERSION: 0x03000001
+SHARDING_FORK_EPOCH: 18446744073709551615
+
+# TBD, 2**32 is a placeholder. Merge transition approach is in active R&D.
+TRANSITION_TOTAL_DIFFICULTY: 4294967296
+
+
+# Time parameters
+# ---------------------------------------------------------------
+# [customized] Faster for testing purposes
+SECONDS_PER_SLOT: 6
+# 14 (estimate from Eth1 mainnet)
+SECONDS_PER_ETH1_BLOCK: 14
+# 2**8 (= 256) epochs
+MIN_VALIDATOR_WITHDRAWABILITY_DELAY: 256
+# [customized] higher frequency of committee turnover and faster time to acceptable voluntary exit
+SHARD_COMMITTEE_PERIOD: 64
+# [customized] process deposits more quickly, but insecure
+ETH1_FOLLOW_DISTANCE: 16
+
+
+# Validator cycle
+# ---------------------------------------------------------------
+# 2**2 (= 4)
+INACTIVITY_SCORE_BIAS: 4
+# 2**4 (= 16)
+INACTIVITY_SCORE_RECOVERY_RATE: 16
+# 2**4 * 10**9 (= 16,000,000,000) Gwei
+EJECTION_BALANCE: 16000000000
+# 2**2 (= 4)
+MIN_PER_EPOCH_CHURN_LIMIT: 4
+# 2**16 (= 65,536)
+CHURN_LIMIT_QUOTIENT: 65536
+
+
+# Deposit contract
+# ---------------------------------------------------------------
+# Ethereum Goerli testnet
+DEPOSIT_CHAIN_ID: 5
+DEPOSIT_NETWORK_ID: 5
+# Configured on a per testnet basis
+DEPOSIT_CONTRACT_ADDRESS: 0x1234567890123456789012345678901234567890

presets/README.md

@@ -0,0 +1,26 @@
+## Presets
+
+Presets are more extensive than runtime configurations, and generally only applicable during compile-time.
+Each preset is defined as a directory, with YAML files per fork.
+
+Configurations can extend a preset by setting the `PRESET_BASE` variable.
+An implementation may choose to only support 1 preset per build-target and should validate
+the `PRESET_BASE` variable in the config matches the running build.
+
+Standard presets:
+- [`mainnet/`](./mainnet): Used in mainnet, mainnet-like testnets (e.g. Prater), and spec-testing
+- [`minimal/`](./minimal): Used in low-resource local dev testnets, and spec-testing
+
+Client implementers may opt to support additional presets, e.g. for extra large beacon states for benchmarking.
+See [`/configs/`](../configs) for run-time configuration, e.g. to configure a new testnet.
+
+## Forking
+
+Like the [config forking](../configs/README.md#forking),
+the preset extends with every fork, instead of overwriting previous values.
+An implementation can ignore preset files as a whole for future forks,
+and can thus implement stricter compile-time warnings on unrecognized or missing variables in current forks.
+
+## Format
+
+The preset format matches the [config format](../configs/README.md#format).