Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(storage-manager): search replication log only once #1505

Merged

Commits on Oct 2, 2024

  1. refactor(storage-manager): split aligner.rs file

    The `aligner.rs` file was only growing in size, rendering navigation
    complicated. This commit splits it in the following manner:
    - `aligner_query.rs` inherits the `AlignmentQuery` and the logic that
      generates a reply to it.
    - `aligner_reply.rs` inherits the `AlignmentReply` and the logic to
      process it.
    - `core.rs` inherits the `spawn_query_replica_aligner` that spawns a
      task to query the "Aligner" of a Replica.
    
    * plugins/zenoh-plugin-storage-manager/src/replication/aligner.rs:
      deleted and split the content of the file.
    
    * plugins/zenoh-plugin-storage-manager/src/replication/core.rs:
      - Declared the new modules `aligner_query` and `aligner_reply`.
      - Updated the imports as the method `spawn_query_replica_aligner`
        was moved there.
    
    * plugins/zenoh-plugin-storage-manager/src/replication/core/aligner_query.rs:
      new file that holds the logic to generate a reply to an AlignmentQuery.
    
    * plugins/zenoh-plugin-storage-manager/src/replication/core/aligner_reply.rs:
      new file that holds the logic to process an AlignmentReply.
    
    * plugins/zenoh-plugin-storage-manager/src/replication/mod.rs: removed
      the, no longer used, module `aligner`.
    
    Signed-off-by: Julien Loudet <julien.loudet@zettascale.tech>
    J-Loudet committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    77f564a View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2024

  1. feat(storage-manager): remove_older for LogLatest

    This commit adds the method `remove_older` to the `LogLatest`
    structure. The motivation for this addition is to later optimise the
    alignment by only searching through the Replication Log once when
    replacing / adding Events.
    
    To make the rest of the code more coherent, the method
    `if_newer_remove_older` were renamed to `remove_older` for the
    `Interval` and `SubInterval` structures.
    
    * plugins/zenoh-plugin-storage-manager/src/replication/classification.rs:
      - Renamed `if_newer_remove_older` to `remove_older` for the Interval
        structure.
      - Renamed `if_newer_remove_older` to `remove_older` for the
        SubInterval structure.
      - Added a line in the documentation of both `remove_older` methods to
        indicate that their respective Fingerprint would be updated.
    
    * plugins/zenoh-plugin-storage-manager/src/replication/log.rs:
      - Updated the method `insert_event` to leverage the newly created
        `remove_older` method.
      - Added the `remove_older` method that removes from the Replication
        Log an older Event on the same key expression.
    
    * plugins/zenoh-plugin-storage-manager/src/replication/tests/classification.test.rs:
      updated the calls to `remove_older` due to the renaming.
    
    Signed-off-by: Julien Loudet <julien.loudet@zettascale.tech>
    J-Loudet committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    e6bb542 View commit details
    Browse the repository at this point in the history
  2. refactor(storage-manager): AlignmentReply::Events -> EventsMetadata

    This renaming makes the content of this variant clear: only the metadata
    is processed.
    
    * plugins/zenoh-plugin-storage-manager/src/replication/core/aligner_query.rs:
      renamed `AlignmentReply::Events` to `AlignmentReply::EventsMetadata`.
    * plugins/zenoh-plugin-storage-manager/src/replication/core/aligner_reply.rs:
      renamed `AlignmentReply::Events` to `AlignmentReply::EventsMetadata`,
      including in logging.
    
    Signed-off-by: Julien Loudet <julien.loudet@zettascale.tech>
    J-Loudet committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    3f8493b View commit details
    Browse the repository at this point in the history
  3. refactor(storage-manager): separate processing of EventMetadata

    This commit moves, for clarity purposes, the logic to process an
    `EventMetadata` after receiving an `AlignmentReply::EventsMetadata` in a
    separate method: `process_event_metadata`.
    
    This notably removes indentation levels and makes the code more
    readable.
    
    * plugins/zenoh-plugin-storage-manager/src/replication/core/aligner_reply.rs:
      move the logic to process an `EventMetadat` in a separate method
      `process_event_metadata`.
    
    Signed-off-by: Julien Loudet <julien.loudet@zettascale.tech>
    J-Loudet committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    a508ade View commit details
    Browse the repository at this point in the history
  4. refactor(storage-manager): separate processing of Retrieval

    This commit moves, for clarity purposes, the logic to process an
    `Retrieval` in a separate method: `process_event_retrieval`.
    
    This notably removes indentation levels and makes the code more
    readable.
    
    * plugins/zenoh-plugin-storage-manager/src/replication/core/aligner_reply.rs:
      move the logic to process a `Retrieval` in a separate method
      `process_event_retrieval`.
    
    Signed-off-by: Julien Loudet <julien.loudet@zettascale.tech>
    J-Loudet committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    87922b5 View commit details
    Browse the repository at this point in the history
  5. refactor(storage-manager): optimise insertion during Replication

    Before this commit, the Replication Log was searched twice when aligning
    an Event: once while calling `lookup` and a second time when calling
    `insert_event`.
    
    To reduce the number of searches to a single one, the method
    `insert_event_unchecked` was introduced. This function assumes that the
    Replication Log contains no Event with the same key expression as the
    Event it will insert.
    
    Leveraging this, the calls to `LogLatest::lookup` followed by
    `LogLatest::insert_event` were replaced with calls to
    `LogLatest::remove_older` followed by
    `LogLatest::insert_event_unchecked`.
    
    Indeed, if the call to `LogLatest::remove_older` yields either
    `NotFound` or `RemovedOlder` then the Replication Log does not contain
    any event with the key expression, and
    `LogLatest::insert_event_unchecked` can be confidently called.
    
    Another consequence of the introduction of the new method
    `insert_event_unchecked` is that the variant
    `EventInsertion::NotInsertedAsOutOfBound` was no longer used and was
    thus removed.
    
    * plugins/zenoh-plugin-storage-manager/src/replication/core/aligner_reply.rs:
      Replaced the calls to `lookup` followed by `insert_event` to calls
      to `remove_older` followed by `insert_event_unchecked` in the
      methods `process_event_metadata` and `process_event_retrieval`.
    
    * plugins/zenoh-plugin-storage-manager/src/replication/log.rs:
      - Removed the no longer used `EventInsertion::NotInsertedAsOutOfBound`
        variant.
      - Added a comment to clearly indicate that the method `insert_event`
        will remove older Event.
      - Introduced the new method `insert_event_unchecked` that inserts an
        Event in the Replication Log without checking if it is the only
        Event (i.e. it assumes it is the case).
    
    Signed-off-by: Julien Loudet <julien.loudet@zettascale.tech>
    J-Loudet committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    c5063ae View commit details
    Browse the repository at this point in the history
  6. feat(storage-manager): add assertion on Replication Log in Debug

    This commit introduces an assertion test every time an Event in inserted
    in the Replication Log ONLY IN DEBUG.
    
    The purpose is to make sure that the invariant of the Replication Log is
    upheld: it should only contain one Event per key expression.
    
    * plugins/zenoh-plugin-storage-manager/src/replication/classification.rs:
      added the method `assert_only_one_event_per_key_expr` to the
      `Interval` and `SubInterval` structures.
    
    * plugins/zenoh-plugin-storage-manager/src/replication/log.rs: added the
      method `assert_only_one_event_per_key_expr` to the `LogLatest`
      structure.
    
    Signed-off-by: Julien Loudet <julien.loudet@zettascale.tech>
    J-Loudet committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    f1a1dac View commit details
    Browse the repository at this point in the history