Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nightly backward compatibility testing with e2e tests #2027

Open
7 tasks
jpraynaud opened this issue Oct 21, 2024 · 0 comments
Open
7 tasks

Nightly backward compatibility testing with e2e tests #2027

jpraynaud opened this issue Oct 21, 2024 · 0 comments
Labels

Comments

@jpraynaud
Copy link
Member

jpraynaud commented Oct 21, 2024

Why

We need to run backward compatibility tests to make sure that the current distribution we are working on is compatible (or not) with the previously released distributions. These tests should be run once a day.

What

Create a script that runs the e2e test on all the interesting combinations of signer/aggregator/client/(relay?) versions and records if the e2e test succeeds or fails.

How

  • Download the last N (2 or 3) stable distributions binaries
  • Create combination of the signer/aggregator/client/(relay?) where all except one are unstable
  • Run a e2e test on each combination and record if it succeeds or fails
  • Adjust the e2e test parameters for slower execution (slower epochs and slot lengths)
  • Create a GitHub Actions workflow with matrix configuration for the combinations
  • Schedule the workflow at night
  • Manual launch of the workflow with a specific combination and/or different number of previous stable versions?
@jpraynaud jpraynaud added the CI/CD 🏭 CI/CD label Oct 21, 2024
@jpraynaud jpraynaud changed the title NIghtly regression testing with e2e tests Nightly regression testing with e2e tests Oct 21, 2024
@jpraynaud jpraynaud changed the title Nightly regression testing with e2e tests Nightly backward compatibility testing with e2e tests Oct 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant