Test format docs

specs/test_formats/operations/README.md

@@ -21,8 +21,8 @@ Operations:
 
 | *`operation-name`*      | *`operation-object`* | *`input name`*       | *`processing call`*                                    |
 |-------------------------|----------------------|----------------------|--------------------------------------------------------|
-| `attestation`           | `Attestation`        | `attestation`        | `process_attestation(state, attestation)`                  |
-| `attester_slashing`     | `AttesterSlashing`   | `attester_slashing`  | `process_attester_slashing(state, attester_slashing)`            |
+| `attestation`           | `Attestation`        | `attestation`        | `process_attestation(state, attestation)`              |
+| `attester_slashing`     | `AttesterSlashing`   | `attester_slashing`  | `process_attester_slashing(state, attester_slashing)`  |
 | `block_header`          | `Block`              | `block`              | `process_block_header(state, block)`                   |
 | `deposit`               | `Deposit`            | `deposit`            | `process_deposit(state, deposit)`                      |
 | `proposer_slashing`     | `ProposerSlashing`   | `proposer_slashing`  | `process_proposer_slashing(state, proposer_slashing)`  |

specs/test_formats/shuffling/README.md

@@ -7,10 +7,10 @@ Clients may take different approaches to shuffling, for optimizing,
  and supporting advanced lookup behavior back in older history.
 
 For implementers, possible test runners implementing testing can include:
-1) Just test permute-index, run it for each index `i` in `range(count)`, and check against expected `output[i]` (default spec implementation).
-2) Test un-permute-index (the reverse lookup; implemented by running the shuffling rounds in reverse, from `round_count-1` to `0`).
-3) Test the optimized complete shuffle, where all indices are shuffled at once; test output in one go.
-4) Test complete shuffle in reverse (reverse rounds, same as #2).
+1) Just test `get_shuffled_index`.
+2) Test the reverse lookup; implemented by running the shuffling rounds in reverse, from `round_count-1` to `0`.
+3) Test the optimized complete shuffle, where all indices are shuffled at once
+4) Test complete shuffle in reverse (reverse rounds, similar to #2).
 
 ## Test case format
 
@@ -20,13 +20,19 @@ count: int
 shuffled: List[int]
 ```
 
-- The `bytes32` is encoded a string, hexadecimal encoding, prefixed with `0x`.
+- The `bytes32` is encoded as a string, hexadecimal encoding, prefixed with `0x`.
 - Integers are validator indices. These are `uint64`, but realistically they are not as big.
 
-The `count` specifies the validator registry size. One should compute the shuffling for indices `0, 1, 2, 3, ..., count (exclusive)`.
-Seed is the raw shuffling seed, passed to permute-index (or optimized shuffling approach). 
+The `count` specifies the count of validators being shuffled (i.e. active validators during committee computation).
 
-## Condition
+One should test the shuffling for indices `0, 1, 2, 3, ..., count (exclusive)`.
+
+`shuffled` is a mapping from `i` to `get_shuffled_index(i)`.
+- `i` here is the index within committee-partitioned space. `i...i+N (excl.)` is used to get the validators for a committee of size `N`.
+- `get_shuffled_index(i)` here is the index within the active-validators space. Pointing to the validator assigned to the committee corresponding to `i`.
 
-The resulting list should match the expected output `shuffled` after shuffling the implied input, using the given `seed`.
+Seed is the raw shuffling seed, passed to shuffling function. 
+
+## Condition
 
+For the `get_shuffled_index` implementation (or list-wise equivalent): `get_shuffled_index(i) == shuffled[i]`

specs/test_formats/ssz_generic/README.md

@@ -16,5 +16,11 @@ Test format documentation can be found here: [uint test format](./uint.md).
 *Note*: The current Phase 0 spec does not use larger uints, and uses byte vectors (fixed length) instead to represent roots etc.
 The exact uint lengths to support may be redefined in the future.
 
-Extension of the SSZ tests collection is planned, with an update to the new spec-maintained `minimal_ssz.py`;
- see CI/testing issues for progress tracking.
+## Recommendation
+
+For SSZ testing directly applicable to test-networks, refer to the [`ssz_static`](../ssz_static/README.md) tests,
+ which cover SSZ behavior for phase-0, for all container types.
+With serialization, hash-tree-root and signing-root all covered.
+Note that deserialization can be tested using this same data,
+ but be cautious about the serialized data only covering the valid-input side.
+SSZ implementations should be hardened against invalid offsets, invalid boolean values, etc. in a non-testing setting.

test_generators/README.md

@@ -4,8 +4,10 @@ This directory contains all the generators for YAML tests, consumed by Eth 2.0 c
 
 Any issues with the generators and/or generated tests should be filed in the repository that hosts the generator outputs, here: [ethereum/eth2.0-spec-tests](https://github.com/ethereum/eth2.0-spec-tests).
 
-Whenever a release is made, the new tests are automatically built, and
-[eth2TestGenBot](https://github.com/eth2TestGenBot) commits the changes to the test repository.
+Test-vectors are generated and published by the release-publishers for every release.
+
+Spec-tests are unified with the executable spec: these tests are PyTest compatible, run in CI, but also enable special settings.
+ Settings include BLS on/off, and the option to output the relevant data into a test-vector in "generator mode".
 
 ## How to run generators