[7.x] [Upgrade Assistant] Server-side batch reindexing (#58598) #59532
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Added server side logic for handling batch reindex * Remove literal string interpolation from translation * Refactor return value of batch endpoint "sucesses" does not communicate accurately what has happened. "started" more closely reflects what has happened. * First iteration of batch queues * Single queue Changed the batchqueues implementation to only using a single queue - since there is only one ES that it is interacting with. Before continuing with this work, just making sure that these pre- cautions are necessary! * Clean up old batch queue implementation * Slight refactor * Revert batch queues implementation * Introduction of QueueSettings Queue settings can be set on a reindex operation and set a timemstamp value on the reindex operation for the scheduler to use down the line for ordering operations and running them in series * Updated worker logic to handle items in queue in series * Refactor /batch endpoint response to "enqueued" not "started" * Fixed jest tests * Refactor worker refresh operations for readability Created a new file op_utils where logic repsonsible for sorting and ordering reindex operation saved objects is. * Add batch API integration test Also assert that reindexing is happening in the expected order * Added a new endpoint: GET batch/queue This allows users of the API to see what the current queue state is for visibility. Using the queue endpoint int he API integration tests for batch too. * Reset the queuedAt timestamp on resume If a reindexOperation is being resumed and put in a queue we also need to reset the queuedAt timestamp to respect the new batch queue ordering. * Fix jest test Added 'undefined' as the second optional param to resumeIndexOperation call. Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> # Conflicts: # x-pack/plugins/upgrade_assistant/server/lib/reindexing/error.ts # x-pack/plugins/upgrade_assistant/server/lib/reindexing/error_symbols.ts # x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.ts # x-pack/plugins/upgrade_assistant/server/routes/reindex_indices/reindex_indices.test.ts # x-pack/plugins/upgrade_assistant/server/routes/reindex_indices/reindex_indices.ts # x-pack/test/upgrade_assistant_integration/upgrade_assistant/reindexing.js
|
@elasticmachine merge upstream |
💚 Build SucceededTo update your PR or re-run it, just comment with: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backports the following commits to 7.x: