Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add duplicate columns with different index to improve Pinot partial match performance #6149

Merged
merged 1 commit into from
Jun 27, 2024

Conversation

neil-xie
Copy link
Contributor

@neil-xie neil-xie commented Jun 27, 2024

What changed?
Add duplicate workflowID/runID/workflowType columns which will be applied text index to improve Pinot prefix partial match performance.

Why?
Currently we applied inverted index for these 3 columns, the partial is kind of slow there. We want to apply text index but the columns can only have one index at a time. So we added 3 duplicate columns and apply text index (in monorepo) to improve the performance.

How did you test it?

Potential risks

Release notes

Documentation Changes

Copy link
Contributor

@bowenxia bowenxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@coveralls
Copy link

coveralls commented Jun 27, 2024

Pull Request Test Coverage Report for Build 01905abe-099f-49c8-a5c2-4352a3db223f

Details

  • 4 of 4 (100.0%) changed or added relevant lines in 1 file are covered.
  • 1913 unchanged lines in 27 files lost coverage.
  • Overall coverage increased (+0.1%) to 71.579%

Files with Coverage Reduction New Missed Lines %
common/task/weighted_round_robin_task_scheduler.go 1 89.05%
service/history/task/transfer_standby_task_executor.go 2 86.64%
common/cache/lru.go 2 93.01%
common/task/fifo_task_scheduler.go 2 87.63%
common/constants.go 2 0.0%
common/persistence/statsComputer.go 3 98.21%
service/history/engine/engineimpl/register_domain_failover_callback.go 3 60.0%
service/history/execution/context.go 4 93.33%
common/persistence/sql/sql_shard_store.go 4 97.16%
service/history/engine/engineimpl/reset_queues.go 4 0.0%
Totals Coverage Status
Change from base Build 0190573d-ff12-4850-94f0-8c77deb099df: 0.1%
Covered Lines: 107168
Relevant Lines: 149719

💛 - Coveralls

@neil-xie neil-xie merged commit 270f17a into uber:master Jun 27, 2024
19 checks passed
@neil-xie neil-xie deleted the cdnc_9628 branch June 27, 2024 18:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants