statistics: add bucket ndv for index histogram #20580
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
winoros
requested review from
wshwsh12 and
eurekaka
and removed request for
a team
|
No release note, Please follow https://github.com/pingcap/community/blob/master/contributors/release-note-checker.md |
|
Please follow PR Title Format:
Or if the count of mainly changed packages are more than 3, use
|
winoros
changed the title
qw4990
reviewed
Oct 23, 2020
|
@winoros could you share some testing results about the selectivity estimation accuracy improvements? |
winoros
force-pushed
the
bucket-ndv-index-hist
branch
from
148f1b5
to
b4bdd72
Compare
|
No release note, Please follow https://github.com/pingcap/community/blob/master/contributors/release-note-checker.md |
|
No release note, Please follow https://github.com/pingcap/community/blob/master/contributors/release-note-checker.md |
qw4990
reviewed
Dec 14, 2020
qw4990
reviewed
Dec 15, 2020
qw4990
reviewed
Dec 21, 2020
… into bucket-ndv-index-hist
ti-srebot
added
status/LGT2
ti-srebot
previously approved these changes
Dec 30, 2020
10 tasks
qw4990
previously approved these changes
Jan 5, 2021
|
/merge |
ti-srebot
added
the
status/can-merge
|
Your auto merge job has been accepted, waiting for:
|
|
/run-all-tests |
|
@winoros merge failed. |
qw4990
approved these changes
Jan 13, 2021
|
/run-all-tests tidb-test=pr/1139 |
|
/run-tics-test |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: close #xxx
Problem Summary:
add bucket ndv for hist to improve its accuracy.
What is changed and how it works?
Proposal: xxx
What's Changed:
Add new stats version number to solve compability problem. if the hist is in old version number, we estimate row count using the old way.
When build index hist, record the NDV. And update the NDV when merging buckets.
When estimate row counts, if the bucket ndv is zero, fallback to cm sketch currently.(After @qw4990 decouple the cmsketch and ndv. This part will be changed to use topn to estimate.).
When updating feedback, collect ndv infomation at coprocessor's reader. And use it to update the current histogram.
When dumping feedback, since the estimation logic is changed, just dump the index's feedback instead of the original way.
How it Works:
Check List
Tests
Side effects
Release note
No release note