Commits on Oct 19, 2023

1. Fix bug where retries within RemoteStoreRefreshListener cause infos/c… …

   …heckpoint mismatch (#10655)
   
   * Fix bug where retries within RemoteStoreRefreshListener cause mismatch between ReplicationCheckpoint and uploaded SegmentInfos.
   
   Retries within RemoteStoreRefreshListener run outside of the refresh thread.  This means that concurrent refreshes
   may occur during syncSegments execution updating the on-reader SegmentInfos.  A shard's latest ReplicationCheckpoint
   is computed and set in a refresh listener, but it is not guaranteed the listener has run before the retry fetches the infos or checkpoint independently.
   This fix ensures the listener recomputes the checkpoint while fetching the SegmentInfos. This change also
   ensures that we only recompute the checkpoint when necessary because it comes with an IO cost to compute StoreFileMetadata.
   
   Signed-off-by: Marc Handalian <handalm@amazon.com>
   
   Update refresh listener to recompute checkpoint from latest infos snapshot.
   
   Signed-off-by: Marc Handalian <handalm@amazon.com>
   
   Fix broken test case by comparing segments gen
   
   Signed-off-by: Marc Handalian <handalm@amazon.com>
   
   spotless
   
   Signed-off-by: Marc Handalian <handalm@amazon.com>
   
   Fix RemoteStoreRefreshListener tests
   
   Signed-off-by: Marc Handalian <handalm@amazon.com>
   
   * add extra log
   
   Signed-off-by: Marc Handalian <handalm@amazon.com>
   
   ---------
   
   Signed-off-by: Marc Handalian <handalm@amazon.com>
   (cherry picked from commit e389a09)
   Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

   

   github-actions[bot] committed Oct 19, 2023
   Configuration menu

   • View commit details
   • Copy full SHA for 23d691d
   • Browse repository at this point

   Copy the full SHA

   23d691d View commit details

   Browse the repository at this point in the history