feat(replays): Add materialized tags column and index #6308
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This PR has a migration; here is the generated SQL -- start migrations
-- forward migration replays : 0021_index_tags
Local op: ALTER TABLE replays_local ADD COLUMN IF NOT EXISTS _tags_hash_map Array(UInt64) MATERIALIZED arrayMap((k, v) -> cityHash64(concat(replaceRegexpAll(k, '(\\=|\\\\)', '\\\\\\1'), '=', v)), tags.key, tags.value) AFTER tags.value;
Distributed op: ALTER TABLE replays_dist ADD COLUMN IF NOT EXISTS _tags_hash_map Array(UInt64) MATERIALIZED arrayMap((k, v) -> cityHash64(concat(replaceRegexpAll(k, '(\\=|\\\\)', '\\\\\\1'), '=', v)), tags.key, tags.value) AFTER tags.value;
Local op: ALTER TABLE replays_local ADD INDEX IF NOT EXISTS bf_tags_hash_map _tags_hash_map TYPE bloom_filter GRANULARITY 1;
-- end forward migration replays : 0021_index_tags
-- backward migration replays : 0021_index_tags
Local op: ALTER TABLE replays_local DROP INDEX IF EXISTS bf_tags_hash_map;
Distributed op: ALTER TABLE replays_dist DROP COLUMN IF EXISTS _tags_hash_map;
Local op: ALTER TABLE replays_local DROP COLUMN IF EXISTS _tags_hash_map;
-- end backward migration replays : 0021_index_tags |
❌ 1 Tests Failed:
View the top 1 failed tests by shortest run time
To view individual test run time comparison to the main branch, go to the Test Analytics Dashboard |
…ithub.com/getsentry/snuba into cmanallen/replays-add-indexed-tags-column
nikhars
approved these changes
Sep 17, 2024
cmanallen
added a commit
that referenced
this pull request
Sep 19, 2024
Exposes _tags_hash_map column. Follow the prior art established by the errors table. Related issue: getsentry/sentry#76289 Migration PR: #6308
cmanallen
added a commit
to getsentry/sentry
that referenced
this pull request
Sep 20, 2024
Tags search is expensive from a bytes and rows scanned perspective. This optimization makes it so that single tag searches can be run quickly and efficiently. Depends on: getsentry/snuba#6308 and getsentry/snuba#6309 Related: #76289
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds materialized _tags_hash_map column and indexes it. Follow the prior art established by the errors table.
Addresses the following problems:
Related: getsentry/sentry#76289