Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*:add the indexlookup runtime stats infomation #20145

Merged
merged 60 commits into from
Oct 23, 2020

Conversation

jyz0309
Copy link
Contributor

@jyz0309 jyz0309 commented Sep 22, 2020

Signed-off-by: jyz0309 45495947@qq.com

What problem does this PR solve?

Issue Number: close #19349, close #20323

Problem Summary:Currently, the runtime stats of IndexLookUp is hard to understand
After this pr:

mysql> explain analyze select * from t use index(a) where a > 1;
+-------------------------------+---------+---------+-----------+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------+--------------+------+
| id                            | estRows | actRows | task      | access object       | execution info                                                                                                                                                         | operator info                                  | memory       | disk |
+-------------------------------+---------+---------+-----------+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------+--------------+------+
| IndexLookUp_7                 | 1.00    | 2       | root      |                     | time:2.102174ms, loops:2, index_task:868.167µs, table_task:{num:1, concurrency:5, time:5.210409ms}                                                                     |                                                | 8.7578125 KB | N/A  |
| ├─IndexRangeScan_5(Build)     | 1.00    | 2       | cop[tikv] | table:t, index:a(a) | time:873.8µs, loops:1, cop_task: {num: 1, max:754.356µs, proc_keys: 2, rpc_num: 1, rpc_time: 698.426µs, copr_cache_hit_ratio: 0.00}, tikv_task:{time:0s, loops:1}      | range:(1,+inf], keep order:false, stats:pseudo | N/A          | N/A  |
| └─TableRowIDScan_6(Probe)     | 1.00    | 2       | cop[tikv] | table:t             | time:619.809µs, loops:2, cop_task: {num: 1, max:540.214µs, proc_keys: 2, rpc_num: 1, rpc_time: 510.336µs, copr_cache_hit_ratio: 0.00}, tikv_task:{time:0s, loops:1}    | keep order:false, stats:pseudo                 | N/A          | N/A  |
+-------------------------------+---------+---------+-----------+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------+--------------+------+
3 rows in set (0.00 sec)

What is changed and how it works?

How it Works:

  • add the indexlookup runtime stats to get the runstat infomation
  • optimze the explain analyze execution info,make the copStat and rootStat output together

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Manual test (add detailed scripts or steps below)

Side effects

  • Performance regression
    • Consumes more CPU
    • Consumes more MEM

Release note

  • add the indexlookup runtime stats infomation

Signed-off-by: jyz0309 <45495947@qq.com>
@jyz0309 jyz0309 requested a review from a team as a code owner September 22, 2020 06:54
@jyz0309 jyz0309 requested review from fzhedu and removed request for a team September 22, 2020 06:54
@ti-srebot ti-srebot added the contribution This PR is from a community contributor. label Sep 22, 2020
@sre-bot
Copy link
Contributor

sre-bot commented Sep 22, 2020

Please follow PR Title Format:

  • pkg [, pkg2, pkg3]: what's changed

Or if the count of mainly changed packages are more than 3, use

  • *: what's changed

@sre-bot
Copy link
Contributor

sre-bot commented Sep 22, 2020

@sre-bot
Copy link
Contributor

sre-bot commented Sep 22, 2020

Please follow PR Title Format:

  • pkg [, pkg2, pkg3]: what's changed

Or if the count of mainly changed packages are more than 3, use

  • *: what's changed

@sre-bot
Copy link
Contributor

sre-bot commented Sep 22, 2020

@github-actions github-actions bot added the sig/execution SIG execution label Sep 22, 2020
@jyz0309 jyz0309 changed the title init *:add the indexlookup runtime stats infomation Sep 22, 2020
Signed-off-by: jyz0309 <45495947@qq.com>
Signed-off-by: jyz0309 <45495947@qq.com>
@jyz0309
Copy link
Contributor Author

jyz0309 commented Sep 23, 2020

/run-check_dev

jyz0309 and others added 3 commits September 23, 2020 16:37
Signed-off-by: jyz0309 <45495947@qq.com>
Signed-off-by: jyz0309 <45495947@qq.com>
jyz0309 and others added 10 commits September 25, 2020 11:03
Signed-off-by: jyz0309 <45495947@qq.com>
Signed-off-by: jyz0309 <45495947@qq.com>
Signed-off-by: jyz0309 <45495947@qq.com>
Signed-off-by: jyz0309 <45495947@qq.com>
Signed-off-by: jyz0309 <45495947@qq.com>
Signed-off-by: jyz0309 <45495947@qq.com>
Signed-off-by: jyz0309 <45495947@qq.com>
Signed-off-by: jyz0309 <45495947@qq.com>
@jyz0309
Copy link
Contributor Author

jyz0309 commented Sep 26, 2020

/run-check_dev_2

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Oct 20, 2020
@jyz0309
Copy link
Contributor Author

jyz0309 commented Oct 21, 2020

@ichn-hu @qw4990 @SunRunAway PTAL

Copy link
Contributor

@qw4990 qw4990 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Oct 23, 2020
@qw4990
Copy link
Contributor

qw4990 commented Oct 23, 2020

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Oct 23, 2020
@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot
Copy link
Contributor

@jyz0309 merge failed.

@ti-srebot
Copy link
Contributor

@jyz0309 Oops! auto merge is restricted to Committers of the SIG.See the corresponding SIG page for more information. Related SIG: execution(slack).

@crazycs520
Copy link
Contributor

/merge

@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot
Copy link
Contributor

@jyz0309 merge failed.

@crazycs520
Copy link
Contributor

/run-all-tests

@crazycs520 crazycs520 merged commit 58f2a48 into pingcap:master Oct 23, 2020
@ti-srebot
Copy link
Contributor

@breeswish Oops! auto merge is restricted to Committers of the SIG.See the corresponding SIG page for more information. Related SIG: execution(slack).

@crazycs520
Copy link
Contributor

/run-cherry-picker

ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Oct 28, 2020
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot
Copy link
Contributor

cherry pick to release-4.0 in PR #20695

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/executor sig/execution SIG execution status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

incorrect explain analyze result of IndexLookUp on the probe side refine IndexLookUp executor runtime stats
6 participants