Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle multiple beacon types in Certificate #1601

Merged
merged 42 commits into from
Mar 29, 2024

Conversation

Alenar
Copy link
Collaborator

@Alenar Alenar commented Mar 27, 2024

Content

Warning

The hash computation of certificates is changed by this PR, we must run the certificate hash migrator on our aggregators after deployment.

This PR do the groundwork to allow multiple beacon types in our nodes.

  1. Certificate - replace the CardanoDbBeacon with a more generic SignedEntityType :
    • Add a SignedEntityType if the certificate sign a multi-signature
    • Add a epoch field since it's crucial data of the certificate and the signed entity type may not have one
    • Remove the beacon field
    • Add a network and immutable_file_number fields to the CertificateMetadata (the later is for retro-compatibility only, it's used to reconstruct a CardanoDbBeacon for certificate messages).
  2. Certificate messages, same kind of changes but with retro-compatibility in mind:
    • Add a epoch and signed_entity_type fields plus a network field to their metadata.
    • Mark their beacon fields as deprecated.
  3. New common type: TimePoint
    • Conceptually it's the current tip of the chain for all beacons that we handle now and in the future (ie: cardano transactions beacon data will be added to it).
    • Supersede CardanoDbBeacon for state machines pacing: cardano db beacon can only pace signed data by immutable file number and/or an epoch, TimePoint will be allow to pace all signed entity types.
    • BeaconProvider is replaced by a TimePointProvider

Pre-submit checklist

  • Branch
    • Tests are provided (if possible)
    • Crates versions are updated (if relevant)
    • Commit sequence broadly makes sense
    • Key commits have useful messages
  • PR
    • No clippy warnings in the CI
    • Self-reviewed the diff
    • Useful pull request description
    • Reviewer requested
  • Documentation
    • Update README file (if relevant)
    • Update documentation website (if relevant)
    • Add dev blog post (if relevant)

Comments

Also fix a minor issue of the fake aggregator data import script when it was run on a really young network (such as a run only e2e right after it is ready).

Issue(s)

Relates to #1562

Copy link

github-actions bot commented Mar 27, 2024

Test Results

    3 files  ±0     42 suites  ±0   8m 41s ⏱️ - 1m 8s
  943 tests  - 4    943 ✅  - 4  0 💤 ±0  0 ❌ ±0 
1 037 runs   - 4  1 037 ✅  - 4  0 💤 ±0  0 ❌ ±0 

Results for commit 92d877a. ± Comparison against base commit 46c7b79.

This pull request removes 15 and adds 11 tests. Note that renamed tests count towards both.
mithril-aggregator ‑ runtime::runner::tests::test_get_beacon_from_chain
mithril-aggregator ‑ runtime::state_machine::tests::ready_certificate_does_not_exist_for_beacon
mithril-aggregator ‑ tools::certificates_hash_migrator::test::recompute_hash_test_tool
mithril-common ‑ beacon_provider::tests::test_beacon_error
mithril-common ‑ beacon_provider::tests::test_beacon_ok
mithril-common ‑ entities::cardano_db_beacon::tests::test_beacon_compare_to_older_different_network
mithril-common ‑ entities::cardano_db_beacon::tests::test_beacon_compare_to_older_epochs_greater
mithril-common ‑ entities::cardano_db_beacon::tests::test_beacon_compare_to_older_epochs_less
mithril-common ‑ entities::cardano_db_beacon::tests::test_beacon_compare_to_older_equal
mithril-common ‑ entities::cardano_db_beacon::tests::test_beacon_compare_to_older_lower_epoch_greater_immutable
…
mithril-aggregator ‑ database::provider::certificate::tests::persisting_many_without_any_records_dont_crash
mithril-aggregator ‑ runtime::runner::tests::test_get_time_point_from_chain
mithril-aggregator ‑ runtime::state_machine::tests::ready_certificate_does_not_exist_for_time_point
mithril-aggregator ‑ tools::certificates_hash_migrator::test::ensure_test_framework_recompute_correct_hashes
mithril-common ‑ entities::time_point::tests::time_point_ord_cmp_epochs_less
mithril-common ‑ entities::time_point::tests::time_point_ord_equal
mithril-common ‑ entities::time_point::tests::time_point_ord_same_epoch_greater
mithril-common ‑ entities::time_point::tests::time_point_ord_same_epoch_less
mithril-common ‑ time_point_provider::tests::test_error_from_dependency
mithril-common ‑ time_point_provider::tests::test_happy_path
…

♻️ This comment has been updated with latest results.

mithril-signer/src/runtime/runner.rs Outdated Show resolved Hide resolved
mithril-aggregator/src/runtime/state_machine.rs Outdated Show resolved Hide resolved
mithril-aggregator/src/runtime/state_machine.rs Outdated Show resolved Hide resolved
mithril-aggregator/src/runtime/runner.rs Outdated Show resolved Hide resolved
mithril-aggregator/src/runtime/runner.rs Outdated Show resolved Hide resolved
@Alenar Alenar self-assigned this Mar 27, 2024
@Alenar Alenar force-pushed the ensemble/1562/multiple_beacon_types branch from 34b3906 to e7e1fb5 Compare March 27, 2024 17:05
@Alenar Alenar temporarily deployed to testing-sanchonet March 27, 2024 17:13 — with GitHub Actions Inactive
@Alenar Alenar force-pushed the ensemble/1562/multiple_beacon_types branch from e7e1fb5 to 52a31df Compare March 28, 2024 09:43
@Alenar Alenar temporarily deployed to testing-sanchonet March 28, 2024 09:50 — with GitHub Actions Inactive
As one can always be deduced from the certificate since a genesis
certificate is equivalent to a MithrilStakeDistribution
SignedEntityType.
This means that bleeding edge signers won't be compatible with our
until we update our aggregators.
This type will be used for state machines states as its role is to
encompass all beacon values.
Allowing it to know the network on which the aggregator operate.
Since they don't use the structure content it can be fully eluded.
Copy link
Member

@jpraynaud jpraynaud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good 👍

I left few comments and minor typos to fix

mithril-aggregator/src/database/migration.rs Show resolved Hide resolved
mithril-aggregator/src/database/migration.rs Show resolved Hide resolved
mithril-aggregator/src/database/migration.rs Outdated Show resolved Hide resolved
drop table certificate;
alter table new_certificate rename to certificate;

CREATE INDEX epoch_index on certificate(epoch);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need an index on signed_entity_type_id?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question, we did no indexes on any of our foreign key columns whatever the table. If we changes stance on that maybe we should do that on a dedicated ticket ?

mithril-aggregator/src/database/migration.rs Show resolved Hide resolved
mithril-common/src/messages/certificate_list.rs Outdated Show resolved Hide resolved
mithril-common/src/entities/time_point.rs Show resolved Hide resolved
mithril-signer/src/runtime/runner.rs Outdated Show resolved Hide resolved
@Alenar Alenar force-pushed the ensemble/1562/multiple_beacon_types branch from b9a97ac to f3228ad Compare March 28, 2024 17:38
…te_genesis_certificate`

As later we will remove the immutable_file_number this will make the
removal commit lighter.
@Alenar Alenar force-pushed the ensemble/1562/multiple_beacon_types branch from f3228ad to a87ff74 Compare March 28, 2024 17:41
@Alenar Alenar temporarily deployed to testing-sanchonet March 28, 2024 17:47 — with GitHub Actions Inactive
@Alenar Alenar temporarily deployed to testing-sanchonet March 29, 2024 09:52 — with GitHub Actions Inactive
@Alenar Alenar merged commit c721515 into main Mar 29, 2024
41 checks passed
@Alenar Alenar deleted the ensemble/1562/multiple_beacon_types branch March 29, 2024 10:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants