-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement proposed test organization and structure improvements #1320
Conversation
…y of configuration in context of forks, and update docs
…n if it already exists.
Not sure what happened, but it doesn't show up as draft PR anymore. Still a work in progress (most testgen is updated, just need more documentation updates). |
…ven same runner dir
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
awesome! didn't quite make it through all of the modified generators but about to step AFK.
Just have some minor fixes along with a few rationale questions on little design decisions. great work :)
Co-Authored-By: Danny Ryan <dannyjryan@gmail.com>
Co-Authored-By: Danny Ryan <dannyjryan@gmail.com>
|
This PR implements the proposed changes to test vector generation in #1311
Change log:
tests/<config name>/<fork or phase name>/<test runner name>/<test handler name>/<test suite name>/<test case>/<output part>
. Background details can be found in Test orientation & distribution format issues #1311, and all the necessary information can be found in the new updatedspecs/test_formats/README.md
.ssz
file.pre
/post
/change
level. This enables clients to choose for SSZ encoding over YAML. For better testing speed, and no issues with YAML unmarshalling. This alleviates pains Prysmatic Labs was having with the legacy choice for YAML.ssz_generic
test generator is included, which utilizes the new ability for differently typed test parts, and starts to fill the previous gap of general-purposes SSZ tests.generator -> runner
anymore, any generator can output to any runner/handler/configs/suites it wishes too.-l <config name> [<config name> [...]]
flag.general
, a folder next to the real configurations. These general tests simply do not require any particular configuration for correctness.config.yaml
.PHASE_1_FORK_EPOCH
), and this allows for a less complicated forking setup in the future. As there are no forks yet, this should not affect clients directly. The configuration procedure for forks is more extensively described in the main test format doc.Note: before introducing an update to the generic-ssz format this PR actually reduced line count. The documentation and generation code however pulled it back. However, more testing and documentation is always welcome :)