planner: support lock/unlock stats of partition tables

[Rustin170506]

What problem does this PR solve?

Issue Number: ref #46351

Problem Summary:

What is changed and how it works?

• added support for lock/unlock stats of partition tables
• added test for lock/unlock stats of partition tables

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

[Rustin170506]

🔢 self-check

[Rustin170506]

/retest

[Rustin170506]

/retest

[Rustin170506]

1. Start the TiDB cluster: tiup playground v7.3.0 --tiflash 0 --db.binpath /Users/hi-rustin/GolandProjects/tidb/bin/tidb-server
2. Create table and analyze it:

create table t(a int, b varchar(10), index idx_b (b)) partition by range(a) (partition p0 values less than (10), partition p1 values less than (20));
analyze table test.t;
show warnings;

+---------+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Level   | Code | Message                                                                                                                                                              |
+---------+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Warning | 1105 | disable dynamic pruning due to t has no global stats                                                                                                                 |
| Note    | 1105 | Analyze use auto adjusted sample rate 1.000000 for table test.t's partition p0, reason to use this rate is "TiDB assumes that the table is empty, use sample-rate=1" |
| Warning | 1105 | disable dynamic pruning due to t has no global stats                                                                                                                 |
| Note    | 1105 | Analyze use auto adjusted sample rate 1.000000 for table test.t's partition p1, reason to use this rate is "TiDB assumes that the table is empty, use sample-rate=1" |
+---------+------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+

3. Lock one partition: lock stats t partition p0;
4. Check the lock result: show stats_locked;

mysql> show stats_locked;
+---------+------------+----------------+--------+
| Db_name | Table_name | Partition_name | Status |
+---------+------------+----------------+--------+
| test    | t          | p0             | locked |
+---------+------------+----------------+--------+
1 row in set (0.00 sec

5. Insert some data: insert into t(a, b) values(1,'a'); insert into t(a, b) values(2,'b');
6. Analyze table again: analyze table test.t;

mysql> analyze table test.t;
Query OK, 0 rows affected, 3 warnings (0.11 sec)

mysql> show warnings;
+---------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Level   | Code | Message                                                                                                                                                                                                                              |
+---------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Note    | 1105 | Analyze use auto adjusted sample rate 1.000000 for table test.t's partition p0, reason to use this rate is "Row count in stats_meta is much smaller compared with the row count got by PD, use min(1, 15000/2) as the sample-rate=1" |
| Note    | 1105 | Analyze use auto adjusted sample rate 1.000000 for table test.t's partition p1, reason to use this rate is "TiDB assumes that the table is empty, use sample-rate=1"                                                                 |
| Warning | 1105 | skip analyze locked table: test.t partition (p0)                                                                                                                                                                                     |
+---------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
3 rows in set (0.00 sec)

7. Unlock partition: unlock stats t partition p0;
8. Analyze partition again: analyze table test.t partition p0;

mysql> analyze table test.t partition p0;
Query OK, 0 rows affected, 1 warning (0.10 sec)

mysql> show warnings;
+-------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
| Level | Code | Message                                                                                                                                                |
+-------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
| Note  | 1105 | Analyze use auto adjusted sample rate 1.000000 for table test.t's partition p0, reason to use this rate is "use min(1, 110000/2) as the sample-rate=1" |
+-------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

[Rustin170506]

If a partition is locked before generating valid statistics, what will happen as a result?

1. Start the TiDB cluster: tiup playground v7.3.0 --tiflash 0 --db.binpath /Users/hi-rustin/GolandProjects/tidb/bin/tidb-server
2. Create table and lock it:

create table t(a int, b varchar(10), index idx_b (b)) partition by range(a) (partition p0 values less than (10), partition p1 values less than (20)); lock stats t partition p0;

3. Check the lock result: show stats_locked;

mysql> show stats_locked;
+---------+------------+----------------+--------+
| Db_name | Table_name | Partition_name | Status |
+---------+------------+----------------+--------+
| test    | t          | p0             | locked |
+---------+------------+----------------+--------+
1 row in set (0.00 sec

4. Insert some data: insert into t(a, b) values(1,'a'); insert into t(a, b) values(2,'b');
5. Analyze table again: analyze table test.t;

mysql> analyze table test.t;
Query OK, 0 rows affected, 3 warnings (0.11 sec)

mysql> show warnings;
+---------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Level   | Code | Message                                                                                                                                                                                                                              |
+---------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Warning | 1105 | disable dynamic pruning due to t has no global stats                                                                                                                                                                                 |
| Note    | 1105 | Analyze use auto adjusted sample rate 1.000000 for table test.t's partition p0, reason to use this rate is "Row count in stats_meta is much smaller compared with the row count got by PD, use min(1, 15000/2) as the sample-rate=1" |
| Warning | 1105 | disable dynamic pruning due to t has no global stats                                                                                                                                                                                 |
| Note    | 1105 | Analyze use auto adjusted sample rate 1.000000 for table test.t's partition p1, reason to use this rate is "TiDB assumes that the table is empty, use sample-rate=1"                                                                 |
| Warning | 1105 | skip analyze locked table: test.t partition (p0)                                                                                                                                                                                     |
+---------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
5 rows in set (0.00 sec)
`
6. After one minute, try it again: `analyze table test.t;`
```sql
mysql> analyze table test.t;
Query OK, 0 rows affected, 3 warnings (0.09 sec)

mysql> show warnings;
+---------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Level   | Code | Message                                                                                                                                                                                                                              |
+---------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Note    | 1105 | Analyze use auto adjusted sample rate 1.000000 for table test.t's partition p0, reason to use this rate is "Row count in stats_meta is much smaller compared with the row count got by PD, use min(1, 15000/2) as the sample-rate=1" |
| Note    | 1105 | Analyze use auto adjusted sample rate 1.000000 for table test.t's partition p1, reason to use this rate is "TiDB assumes that the table is empty, use sample-rate=1"                                                                 |
| Warning | 1105 | skip analyze locked table: test.t partition (p0)                                                                                                                                                                                     |
+---------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
3 rows in set (0.01 sec)

7. Get meta stats: select * from stats_meta; and you can see p1 has been updated.

mysql> select * from stats_meta;
+--------------------+----------+--------------+-------+--------------------+
| version            | table_id | modify_count | count | snapshot           |
+--------------------+----------+--------------+-------+--------------------+
| 444245517030129674 |      101 |            0 |     0 |                  0 |
| 444245637485297674 |      102 |            0 |     0 | 444245637472190466 |
| 444245637498404890 |      100 |            0 |     0 |                  0 |
+--------------------+----------+--------------+-------+--------------------+
3 rows in set (0.01 sec)

8. Unlock partition: unlock stats t partition p0;
9. Analyze again: analyze table test.t;

mysql> analyze table test.t;
Query OK, 0 rows affected, 2 warnings (0.14 sec)

mysql> show warnings;
+-------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Level | Code | Message                                                                                                                                                                                                                              |
+-------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Note  | 1105 | Analyze use auto adjusted sample rate 1.000000 for table test.t's partition p0, reason to use this rate is "Row count in stats_meta is much smaller compared with the row count got by PD, use min(1, 15000/2) as the sample-rate=1" |
| Note  | 1105 | Analyze use auto adjusted sample rate 1.000000 for table test.t's partition p1, reason to use this rate is "TiDB assumes that the table is empty, use sample-rate=1"                                                                 |
+-------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2 rows in set (0.00 sec)

10. Get meta stats again: select * from stats_meta; and p0 has been updated.

mysql> select * from stats_meta;
+--------------------+----------+--------------+-------+--------------------+
| version            | table_id | modify_count | count | snapshot           |
+--------------------+----------+--------------+-------+--------------------+
| 444245677226328080 |      101 |            0 |     2 | 444245677213220865 |
| 444245677213220879 |      102 |            0 |     0 | 444245677213220865 |
| 444245677239435306 |      100 |            0 |     2 |                  0 |
+--------------------+----------+--------------+-------+--------------------+
3 rows in set (0.00 sec)

[qw4990]

Better to remove the panic, it may cause the process to exit if we add some new task but forget to update here in the future.

[Rustin170506]

Do you have any suggestions on how to handle this case?

[Rustin170506]

Maybe I can return an error here?

[easonn7]

/approve
There is no sysvar change.

[time-and-fate]

In a locked table, locking unlocked partitions and unlocking locked partitions are both meaningless in our implementation.
We disallow the former case but allow the latter case. Is that expected?

[time-and-fate]

I think it'll be better if we add table names and partition names. The same for the unlock stats case and lock/unlock normal table case.

[time-and-fate]

Also OK for me to improve this in another PR 😆

[Rustin170506]

I will improve it in my next PR. Because I want to change how we pass these args.

[Rustin170506]

In a locked table, locking unlocked partitions and

Was that a typo? I think there are no unlocked partitions for a locked table. Or do you mean newly added partitions?

[Rustin170506]

unlocking locked partitions are both meaningless in our implementation.
We disallow the former case but allow the latter case. Is that expected?

Fixed in @hi-rustin
executor: do not allow to unlock partitions of the locked table

Thanks for your review! 💚 💙 💜 💛 ❤️

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: easonn7, qw4990, time-and-fate

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [qw4990,time-and-fate]
• parser/OWNERS [easonn7]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2023-09-14 07:16:13.308453128 +0000 UTC m=+156139.276041179: ☑️ agreed by qw4990.
• 2023-09-15 12:06:43.015294512 +0000 UTC m=+259968.982882562: ☑️ agreed by time-and-fate.