[stable25] optimize background image migration job #35326
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- separate in two stages: to prepare, and to actually migrate - in step one, prepare a list of users to be migrated, and store it compressed as app config - gzcompress can be used, because we already require zlib - upon the next calls (step two), slice off the first 5000 users and migrate them. Re-add job if necessary to repeat. - downside is that an app config value will in the beginning use the RAM with any request, until it thins out. Examples: 2m UUIDs (75 MiB) result in ~40 MiB compressed data, while 0.2Mib for 10 000 UUIDs, 0.4MiB for 20 000 and 4.1 MiB for 200 000. Acceptable. Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
- and use count over isset for better understanding Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
Co-authored-by: Pytal <24800714+Pytal@users.noreply.github.com> Signed-off-by: blizzz <blizzz@arthur-schiwon.de>
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
PVince81
requested review from
blizzz,
artonge,
skjnldsv,
CarlSchwan and
szaimen
1 task
blizzz
approved these changes
Nov 22, 2022
blizzz
added
4. to release
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport of #34677 to stable25