parser,ast: parse `admin show slow` statement #7688

tiancaiamao · 2018-09-13T09:20:49Z

What problem does this PR solve?

Add the following new syntax:

admin show slow top [internal | all] N
admin show slow recent N

I expect to use it like this:

mysql> admin show log top 3;
+-------------------------------------+----------------------------+----------+------------------------------------------------------------------------------------------------------+------+---------+--------------------+----------------+------+-----------------+-----------+----------+
| SQL                                 | START                      | DURATION | DETAILS                                                                                              | SUCC | CONN_ID | TXNTS              | USER           | DB   | TABLE_IDS       | INDEX_IDS | INTERNAL |
+-------------------------------------+----------------------------+----------+------------------------------------------------------------------------------------------------------+------+---------+--------------------+----------------+------+-----------------+-----------+----------+
| select sleep(2)                     | 2018-09-13 14:18:39.723710 | 00:00:02 |                                                                                                      |    1 |       1 | 402868016168042497 | root@127.0.0.1 | test |                 |           |        1 |
| select count(*) from t limit 330000 | 2018-09-13 14:20:38.970281 | 00:00:08 | process_time:1m0.665s wait_time:28.216s request_count:14 total_keys:20000014 processed_keys:20000000 |    1 |       1 | 402868047428714497 | root@127.0.0.1 | test | table_ids:[32], |           |        1 |
| select count(*) from t              | 2018-09-13 14:17:45.066304 | 00:00:09 | process_time:1m5.771s wait_time:31.32s request_count:14 total_keys:20000014 processed_keys:20000000  |    1 |       1 | 402868001842397185 | root@127.0.0.1 | test | table_ids:[32], |           |        1 |
+-------------------------------------+----------------------------+----------+------------------------------------------------------------------------------------------------------+------+---------+--------------------+----------------+------+-----------------+-----------+----------+
3 rows in set (0.00 sec)

mysql> admin show log recent 3;
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------+----------+------------------------------------------------------------------------------------------------------+------+---------+--------------------+----------------+------+-----------------+----------------+----------+
| SQL                                                                                                                                                                               | START                      | DURATION | DETAILS                                                                                              | SUCC | CONN_ID | TXNTS              | USER           | DB   | TABLE_IDS       | INDEX_IDS      | INTERNAL |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------+----------+------------------------------------------------------------------------------------------------------+------+---------+--------------------+----------------+------+-----------------+----------------+----------+
| select count(*) from t limit 110000                                                                                                                                               | 2018-09-13 14:20:55.715194 | 00:00:09 | process_time:1m6.188s wait_time:30.196s request_count:14 total_keys:20000014 processed_keys:20000000 |    1 |       1 | 402868051819626497 | root@127.0.0.1 | test | table_ids:[32], |                |        1 |
| SELECT version, table_id, modify_count, count from mysql.stats_meta where version > 400040410672791554 order by version                                                           | 2018-09-13 14:20:57.148856 | 00:00:03 | wait_time:3.884s request_count:2 total_keys:266 processed_keys:144                                   |    1 |       0 | 402868052199735298 |                |      | table_ids:[19], | index_ids:[1], |        0 |
| select variable_name, variable_value from mysql.global_variables where variable_name in ('tidb_auto_analyze_ratio', 'tidb_auto_analyze_start_time', 'tidb_auto_analyze_end_time') | 2018-09-13 14:20:57.148658 | 00:00:03 | wait_time:3.884s request_count:1 total_keys:3                                                        |    1 |       0 | 402868052199735297 |                |      | table_ids:[13], | index_ids:[1], |        0 |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------+----------+------------------------------------------------------------------------------------------------------+------+---------+--------------------+----------------+------+-----------------+----------------+----------+
3 rows in set (0.00 sec)

What is changed and how it works?

Update parser.y

Check List

Unit test

Related changes

Need to update the documentation

PTAL @winkyao @coocood

Add the following new syntax: * admin show log top [user | internal | all] N * admin show log recent N

coocood · 2018-09-13T09:29:49Z

parser/parser.y

@@ -3606,6 +3611,10 @@ FunctionCallNonKeyword:
 			Args: []ast.ExprNode{ast.NewValueExpr($3), $5},
 		}
 	}
+|	"LOG" '(' ExpressionListOpt ')'


Why add this?

After this PR, "LOG" change to a token from an identifier, it's a not keyword token.
There is no rule for builtin function "select log(...)" , so I add this rule.

coocood · 2018-09-13T09:33:10Z

parser/parser.y

@@ -5221,6 +5230,53 @@ AdminStmt:
 			JobIDs: $6.([]int64),
 		}
 	}
+|	"ADMIN" "SHOW" "LOG" AdminShowLog


"LOG" is too general, "SLOW_QUERY" is better.

@winkyao What's your opinion ?

I vote for the slow query.

Support for using "LOG" .

How about ADMIN SHOW SLOW? I agree that log is too generic, but with slow it could be implied.

coocood · 2018-09-13T09:36:29Z

parser/parser.y

+	}
+
+AdminShowLog:
+	"RECENT" NUM


We can remove the RECENT and make the NUM optional

There will be two containers, one heap for TopN , one queue for Recent.
Suggested by @shenli

No "TOP" means recent.

TOP doesn't mean recent. It means order by x limit n.

coocood · 2018-09-13T09:38:29Z

parser/parser.y

+			Count: getUint64FromNUM($2),
+		}
+	}
+|	"TOP" "USER" NUM


"USER" can be removed, default only show "USER"

Good point!

coocood · 2018-09-13T09:39:33Z

parser/parser.y

+			Count: getUint64FromNUM($3),
+		}
+	}
+|	"TOP" "INTERNAL" NUM


This syntax can be removed, use ALL is enough.

Or Keep this and remove "ALL"

How to represent the result "user" + "internal" ?
We have three cases:

"user"

"internal"

"user" + "internal"

one keyword is not enough

shenli · 2018-09-13T09:48:53Z

Admin show slow query .... is more readable.

shenli · 2018-09-13T09:51:19Z

The value in TABLE_IDS and INDEX_IDS do not have to contain table_ids: and index_ids: as prefixes.
s/TXNTS/Transaction_TS/

mysql> admin show log recent 3;
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------+----------+------------------------------------------------------------------------------------------------------+------+---------+--------------------+----------------+------+-----------------+----------------+----------+
| SQL                                                                                                                                                                               | START                      | DURATION | DETAILS                                                                                              | SUCC | CONN_ID | TXNTS              | USER           | DB   | TABLE_IDS       | INDEX_IDS      | INTERNAL |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------+----------+------------------------------------------------------------------------------------------------------+------+---------+--------------------+----------------+------+-----------------+----------------+----------+
| select count(*) from t limit 110000                                                                                                                                               | 2018-09-13 14:20:55.715194 | 00:00:09 | process_time:1m6.188s wait_time:30.196s request_count:14 total_keys:20000014 processed_keys:20000000 |    1 |       1 | 402868051819626497 | root@127.0.0.1 | test | table_ids:[32], |                |        1 |
| SELECT version, table_id, modify_count, count from mysql.stats_meta where version > 400040410672791554 order by version                                                           | 2018-09-13 14:20:57.148856 | 00:00:03 | wait_time:3.884s request_count:2 total_keys:266 processed_keys:144                                   |    1 |       0 | 402868052199735298 |                |      | table_ids:[19], | index_ids:[1], |        0 |
| select variable_name, variable_value from mysql.global_variables where variable_name in ('tidb_auto_analyze_ratio', 'tidb_auto_analyze_start_time', 'tidb_auto_analyze_end_time') | 2018-09-13 14:20:57.148658 | 00:00:03 | wait_time:3.884s request_count:1 total_keys:3                                                        |    1 |       0 | 402868052199735297 |                |      | table_ids:[13], | index_ids:[1], |        0 |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------+----------+------------------------------------------------------------------------------------------------------+------+---------+--------------------+----------------+------+-----------------+----------------+----------+
3 rows in set (0.00 sec)

tiancaiamao · 2018-09-21T03:21:17Z

PTAL @coocood @shenli

tiancaiamao · 2018-09-25T06:41:12Z

PTAL @coocood @shenli

winkyao · 2018-09-25T14:52:36Z

ast/misc.go

+type ShowSlow struct {
+	Tp    ShowSlowType
+	Count uint64
+	// May be "internal" or "all", default is ""


what about only showing general type?

What do you mean by "general" type? is that "user + internal"?
"user" == "" (the default)
"user + internal" == "all"

Please add some comment to explain what "user" is, and the default.

I've defined a ShowSlowKind type, instead of using string. @winkyao @crazycs520

coocood · 2018-09-26T02:34:04Z

LGTM

crazycs520 · 2018-09-26T02:46:58Z

parser/parser.y

+	{
+		$$ = &ast.ShowSlow{
+			Tp: ast.ShowSlowTop,
+			Kind: "internal",


How about define a const : KindInternal, KindAll, KindDefault ?

crazycs520 · 2018-09-26T02:48:07Z

parser/parser.y

+	{
+		$$ = &ast.ShowSlow{
+			Tp: ast.ShowSlowTop,
+			Kind: "all",


crazycs520

LGTM

crazycs520 · 2018-09-26T05:13:50Z

/run-all-tests

tiancaiamao · 2018-09-26T05:34:07Z

/run-all-tests

zhexuany

LGTM

parser,ast: parse admin show log statement

3bb2107

Add the following new syntax: * admin show log top [user | internal | all] N * admin show log recent N

tiancaiamao added the component/parser label Sep 13, 2018

coocood reviewed Sep 13, 2018

View reviewed changes

tiancaiamao mentioned this pull request Sep 13, 2018

domain: add recent slow queries queue and ShowSlowQuery() function #7692

Merged

address comment

da18d4b

winkyao reviewed Sep 25, 2018

View reviewed changes

tiancaiamao changed the title ~~parser,ast: parse admin show log statement~~ parser,ast: parse admin show slow statement Sep 26, 2018

tiancaiamao added the priority/release-blocker This issue blocks a release. Please solve it ASAP. label Sep 26, 2018

crazycs520 reviewed Sep 26, 2018

View reviewed changes

address comment

0b59e45

crazycs520 reviewed Sep 26, 2018

View reviewed changes

crazycs520 added the status/LGT2 Indicates that a PR has LGTM 2. label Sep 26, 2018

Merge branch 'master' into parse-admin-show-log

edfa66c

zhexuany reviewed Sep 26, 2018

View reviewed changes

crazycs520 approved these changes Sep 26, 2018

View reviewed changes

tiancaiamao merged commit b098b47 into pingcap:master Sep 26, 2018

tiancaiamao deleted the parse-admin-show-log branch September 26, 2018 06:04

tiancaiamao mentioned this pull request Sep 26, 2018

planner,executor: support admin show slow command #7785

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

parser,ast: parse `admin show slow` statement #7688

parser,ast: parse `admin show slow` statement #7688

tiancaiamao commented Sep 13, 2018 •

edited

Loading

coocood Sep 13, 2018

tiancaiamao Sep 13, 2018

coocood Sep 13, 2018

tiancaiamao Sep 13, 2018

shenli Sep 13, 2018

ciscoxll Sep 13, 2018

morgo Sep 13, 2018

coocood Sep 13, 2018

tiancaiamao Sep 13, 2018

coocood Sep 13, 2018

shenli Sep 13, 2018

coocood Sep 13, 2018

tiancaiamao Sep 13, 2018

coocood Sep 13, 2018

coocood Sep 13, 2018

tiancaiamao Sep 13, 2018

shenli commented Sep 13, 2018

shenli commented Sep 13, 2018 •

edited

Loading

tiancaiamao commented Sep 21, 2018

tiancaiamao commented Sep 25, 2018

winkyao Sep 25, 2018

tiancaiamao Sep 26, 2018

winkyao Sep 26, 2018

tiancaiamao Sep 26, 2018

coocood commented Sep 26, 2018

crazycs520 Sep 26, 2018

crazycs520 Sep 26, 2018

crazycs520 left a comment

crazycs520 commented Sep 26, 2018

tiancaiamao commented Sep 26, 2018

zhexuany left a comment

parser,ast: parse admin show slow statement #7688

parser,ast: parse admin show slow statement #7688

Conversation

tiancaiamao commented Sep 13, 2018 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shenli commented Sep 13, 2018

shenli commented Sep 13, 2018 • edited Loading

tiancaiamao commented Sep 21, 2018

tiancaiamao commented Sep 25, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

coocood commented Sep 26, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

crazycs520 left a comment

Choose a reason for hiding this comment

crazycs520 commented Sep 26, 2018

tiancaiamao commented Sep 26, 2018

zhexuany left a comment

Choose a reason for hiding this comment

parser,ast: parse `admin show slow` statement #7688

parser,ast: parse `admin show slow` statement #7688

tiancaiamao commented Sep 13, 2018 •

edited

Loading

shenli commented Sep 13, 2018 •

edited

Loading