Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

executor:add runtime information for StreamAggExec #20861

Closed
wants to merge 12 commits into from

Conversation

jyz0309
Copy link
Contributor

@jyz0309 jyz0309 commented Nov 5, 2020

What problem does this PR solve?

Problem Summary:
Currently, the runtime infomation for StreamAggExec is Unclear and detailed。

What is changed and how it works?

Add the Allocate group runtime information for StreamAggExec

What's Changed:
The explain analyze execution info result when using the StreamAgg.
How it Works:
Add the StreamAggRuntimeStat.
Divide StreamAgg into two parts,Allocate and Split, and time them separately. Among them, Allocate refers to calculating the previous group of data, and Split means the time that used to split a chunk into multiple groups which the row in the same group have the same groupKey
Before this pr:

mysql> explain analyze SELECT /*+ STREAM_AGG() */ count(*) FROM t1 WHERE a > 10;
+------------------------------+----------+---------+-----------+---------------+------------------------------------------------------------------------+---------------------------------+-----------+------+
| id                           | estRows  | actRows | task      | access object | execution info                                                         | operator info                   | memory    | disk |
+------------------------------+----------+---------+-----------+---------------+------------------------------------------------------------------------+---------------------------------+-----------+------+
| StreamAgg_16                 | 1.00     | 1       | root      |               | time:412.884µs, loops:2                                                | funcs:count(Column#5)->Column#4 | 372 Bytes | N/A  |
| └─TableReader_17             | 1.00     | 1       | root      |               | time:407.927µs, loops:2, rpc num: 1, rpc time:349.812µs, proc keys:0   | data:StreamAgg_9                | 208 Bytes | N/A  |
|   └─StreamAgg_9              | 1.00     | 1       | cop[tikv] |               | time:55ms, loops:98                                                    | funcs:count(1)->Column#5        | N/A       | N/A  |
|     └─Selection_15           | 33332.33 | 99989   | cop[tikv] |               | time:55ms, loops:98                                                    | gt(test.t1.a, 10)               | N/A       | N/A  |
|       └─TableFullScan_14     | 99997.00 | 99997   | cop[tikv] | table:t1      | time:54ms, loops:98                                                    | keep order:false, stats:pseudo  | N/A       | N/A  |
+------------------------------+----------+---------+-----------+---------------+------------------------------------------------------------------------+---------------------------------+-----------+------+
5 rows in set (0.00 sec)

After this pr:

mysql> explain analyze SELECT /*+ STREAM_AGG() */ count(*) FROM t1 WHERE a > 10;
+-----------------------------+---------+---------+-----------+------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------+-----------+------+
| id                          | estRows | actRows | task      | access object          | execution info                                                                                                                          | operator info                                   | memory    | disk |
+-----------------------------+---------+---------+-----------+------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------+-----------+------+
| StreamAgg_15                | 1.00    | 1       | root      |                        | time:268.403µs, loops:2, allocate_time:241.607µs,split_time:6.209µs                                                                     | funcs:count(Column#7)->Column#6                 | 372 Bytes | N/A  |
| └─IndexReader_16            | 1.00    | 1       | root      |                        | time:245.611µs, loops:2, cop_task: {num: 1, max:141.252µs, proc_keys: 0, rpc_num: 1, rpc_time: 85.469µs, copr_cache_hit_ratio: 0.00}    | index:StreamAgg_9                               | 162 Bytes | N/A  |
|   └─StreamAgg_9             | 1.00    | 1       | cop[tikv] |                        | time:0ns, loops:0, tikv_task:{time:0s, loops:0}                                                                                         | funcs:count(1)->Column#7                        | N/A       | N/A  |
|     └─IndexRangeScan_14     | 2.08    | 0       | cop[tikv] | table:t1, index:t1a(a) | time:0ns, loops:0, tikv_task:{time:0s, loops:0}                                                                                         | range:(10,+inf], keep order:false, stats:pseudo | N/A       | N/A  |
+-----------------------------+---------+---------+-----------+------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------+-----------+------+
4 rows in set (0.00 sec)

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Manual test (add detailed scripts or steps below)

Side effects

  • Performance regression
    • Consumes more CPU
    • Consumes more MEM

Release note

  • executor:add runtime information for StreamAggExec

Signed-off-by: jyz0309 <45495947@qq.com>
Signed-off-by: jyz0309 <45495947@qq.com>
@jyz0309 jyz0309 requested a review from a team as a code owner November 5, 2020 06:45
@jyz0309 jyz0309 requested review from wshwsh12 and removed request for a team November 5, 2020 06:45
@github-actions github-actions bot added the sig/execution SIG execution label Nov 5, 2020
Signed-off-by: jyz0309 <45495947@qq.com>
@jyz0309
Copy link
Contributor Author

jyz0309 commented Nov 10, 2020

/run-check_dev_2

@jyz0309 jyz0309 self-assigned this Nov 10, 2020
Signed-off-by: jyz0309 <45495947@qq.com>
Signed-off-by: jyz0309 <45495947@qq.com>
@jyz0309
Copy link
Contributor Author

jyz0309 commented Nov 12, 2020

@wshwsh12 PTAL

Signed-off-by: jyz0309 <45495947@qq.com>
Signed-off-by: jyz0309 <45495947@qq.com>
@ti-srebot
Copy link
Contributor

@wshwsh12, PTAL.

@ti-srebot
Copy link
Contributor

@wshwsh12, @crazycs520, PTAL.

@ti-srebot
Copy link
Contributor

@wshwsh12, @crazycs520, PTAL.

@jyz0309 jyz0309 closed this Jan 1, 2021
@jyz0309 jyz0309 deleted the streamaggExec branch January 27, 2021 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/execution SIG execution
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants