Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update tidb-configuration and system variables for 6.1 #8689

Merged
merged 9 commits into from
May 27, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions benchmark/benchmark-tidb-using-sysbench.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,12 @@ It is recommended to use Sysbench 1.0 or later, which can be [downloaded here](h

### TiDB configuration

Higher log level means fewer logs to be printed and thus positively influences TiDB performance. Enable `prepared plan cache` in the TiDB configuration to lower the cost of optimizing execution plan. Specifically, you can add the following command in the TiUP configuration file:
Higher log level means fewer logs to be printed and thus positively influences TiDB performance. Specifically, you can add the following command in the TiUP configuration file:

```yaml
server_configs:
tidb:
log.level: "error"
prepared-plan-cache.enabled: true
```

### TiKV configuration
Expand Down
1 change: 0 additions & 1 deletion best-practices/three-nodes-hybrid-deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,6 @@ tikv:
rocksdb.rate-bytes-per-sec: “200M”

tidb:
performance.committer-concurrency: 4
performance.max-procs: 8
```

Expand Down
7 changes: 1 addition & 6 deletions command-line-flags-for-tidb-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -188,9 +188,4 @@ When you start the TiDB cluster, you can use command-line options or environment
## `--repair-list`

+ The names of the tables to be repaired in the repair mode.
+ Default: `""`

## `--require-secure-transport`

+ Determines whether to require the client to use the secure mode for data transport.
+ Default: `false`
+ Default: `""`
45 changes: 10 additions & 35 deletions configure-memory-usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,58 +6,34 @@ aliases: ['/docs/dev/configure-memory-usage/','/docs/dev/how-to/configure/memory

# TiDB Memory Control

Currently, TiDB can track the memory quota of a single SQL query and take actions to prevent OOM (out of memory) or troubleshoot OOM when the memory usage exceeds a specific threshold value. In the TiDB configuration file, you can configure the options as below to control TiDB behaviors when the memory quota exceeds the threshold value:
Currently, TiDB can track the memory quota of a single SQL query and take actions to prevent OOM (out of memory) or troubleshoot OOM when the memory usage exceeds a specific threshold value. The system variable [`tidb_mem_quota_query`](/system-variables.md#tidb_mem_quota_query) specifies the action to take when a query reaches the memory limit:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Currently, TiDB can track the memory quota of a single SQL query and take actions to prevent OOM (out of memory) or troubleshoot OOM when the memory usage exceeds a specific threshold value. The system variable [`tidb_mem_quota_query`](/system-variables.md#tidb_mem_quota_query) specifies the action to take when a query reaches the memory limit:
Currently, TiDB can track the memory quota of a single SQL query and take actions to prevent OOM (out of memory) or troubleshoot OOM when the memory usage exceeds a specific threshold value. The system variable [`tidb_mem_oom_action`](/system-variables.md#tidb_mem_oom_action) specifies the action to take when a query reaches the memory limit:

@morgo Is this correct?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes you are right. it should be tidb_mem_oom_action here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in #8761


```
# Valid options: ["log", "cancel"]
oom-action = "cancel"
```

- If the configuration item above uses "log", when the memory quota of a single SQL query exceeds the threshold value which is controlled by the `tidb_mem_quota_query` variable, TiDB prints an entry of log. Then the SQL query continues to be executed. If OOM occurs, you can find the corresponding SQL query in the log.
- If the configuration item above uses "cancel", when the memory quota of a single SQL query exceeds the threshold value, TiDB stops executing the SQL query immediately and returns an error to the client. The error information clearly shows the memory usage of each physical execution operator that consumes much memory in the SQL execution process.
- A value of `LOG` means that queries will continue to execute when the [`tidb_mem_quota_query`](/system-variables.md#tidb_mem_quota_query) limit is reached, but TiDB will print an entry to the log.
- A value of `CANCEL` means TiDB stops executing the SQL query immediately after the [`tidb_mem_quota_query`](/system-variables.md#tidb_mem_quota_query) limit is reached, and returns an error to the client. The error information clearly shows the memory usage of each physical execution operator that consumes memory in the SQL execution process.

## Configure the memory quota of a query

In the configuration file, you can set the default Memory Quota for each Query. The following example sets it to 32GB:

```
mem-quota-query = 34359738368
```

In addition, you can control the memory quota of a query using the following session variables. Generally, you only need to configure `tidb_mem_quota_query`. Other variables are used for advanced configuration which most users do not need to care about.

| Variable Name | Description | Unit | Default Value |
| -------------------------------- | ------------------------------------------------- | ---- | ------------- |
| tidb_mem_quota_query | Control the memory quota of a query | Byte | 1 << 30 (1 GB) |
| tidb_mem_quota_hashjoin | Control the memory quota of "HashJoinExec" | Byte | 32 << 30 |
| tidb_mem_quota_mergejoin | Control the memory quota of "MergeJoinExec" | Byte | 32 << 30 |
| tidb_mem_quota_sort | Control the memory quota of "SortExec" | Byte | 32 << 30 |
| tidb_mem_quota_topn | Control the memory quota of "TopNExec" | Byte | 32 << 30 |
| tidb_mem_quota_indexlookupreader | Control the memory quota of "IndexLookUpExecutor" | Byte | 32 << 30 |
| tidb_mem_quota_indexlookupjoin | Control the memory quota of "IndexLookUpJoin" | Byte | 32 << 30 |
| tidb_mem_quota_nestedloopapply | Control the memory quota of "NestedLoopApplyExec" | Byte | 32 << 30 |

Some usage examples:
The system variable [`tidb_mem_quota_query`](/system-variables.md#tidb_mem_quota_query) sets the limit for a query in bytes. Some usage examples:

{{< copyable "sql" >}}

```sql
-- Set the threshold value of memory quota for a single SQL query to 8GB:
set @@tidb_mem_quota_query = 8 << 30;
SET tidb_mem_quota_query = 8 << 30;
```

{{< copyable "sql" >}}

```sql
-- Set the threshold value of memory quota for a single SQL query to 8MB:
set @@tidb_mem_quota_query = 8 << 20;
SET tidb_mem_quota_query = 8 << 20;
```

{{< copyable "sql" >}}

```sql
-- Set the threshold value of memory quota for a single SQL query to 8KB:
set @@tidb_mem_quota_query = 8 << 10;
SET tidb_mem_quota_query = 8 << 10;
```

## Configure the memory usage threshold of a tidb-server instance
Expand Down Expand Up @@ -93,7 +69,6 @@ The following example constructs a memory-intensive SQL statement that triggers
{{< copyable "" >}}

```toml
mem-quota-query = 34359738368 // Increases the memory limit of each query to construct SQL statements that take up larger memory.
[performance]
memory-usage-alarm-ratio = 0.8
```
Expand Down Expand Up @@ -132,7 +107,7 @@ The following example constructs a memory-intensive SQL statement that triggers

TiDB supports disk spill for execution operators. When the memory usage of a SQL execution exceeds the memory quota, tidb-server can spill the intermediate data of execution operators to the disk to relieve memory pressure. Operators supporting disk spill include Sort, MergeJoin, HashJoin, and HashAgg.

- The disk spill behavior is jointly controlled by the [`mem-quota-query`](/tidb-configuration-file.md#mem-quota-query), [`oom-use-tmp-storage`](/tidb-configuration-file.md#oom-use-tmp-storage), [`tmp-storage-path`](/tidb-configuration-file.md#tmp-storage-path), and [`tmp-storage-quota`](/tidb-configuration-file.md#tmp-storage-quota) parameters.
- The disk spill behavior is jointly controlled by [`tidb_mem_quota_query`](/system-variables.md#tidb_mem_quota_query), [`oom-use-tmp-storage`](/tidb-configuration-file.md#oom-use-tmp-storage), [`tmp-storage-path`](/tidb-configuration-file.md#tmp-storage-path), and [`tmp-storage-quota`](/tidb-configuration-file.md#tmp-storage-quota) parameters.
- When the disk spill is triggered, TiDB outputs a log containing the keywords `memory exceeds quota, spill to disk now` or `memory exceeds quota, set aggregate mode to spill-mode`.
- Disk spill for the Sort, MergeJoin, and HashJoin operator is introduced in v4.0.0; disk spill for the HashAgg operator is introduced in v5.2.0.
- When the SQL executions containing Sort, MergeJoin, or HashJoin cause OOM, TiDB triggers disk spill by default. When SQL executions containing HashAgg cause OOM, TiDB does not trigger disk spill by default. You can configure the system variable `tidb_executor_concurrency = 1` to trigger disk spill for HashAgg.
Expand All @@ -148,7 +123,7 @@ The following example uses a memory-consuming SQL statement to demonstrate the d
{{< copyable "sql" >}}

```sql
set tidb_mem_quota_query = 1 << 30;
SET tidb_mem_quota_query = 1 << 30;
```

2. Create a single table `CREATE TABLE t(a int);` and insert 256 rows of different data.
Expand All @@ -172,7 +147,7 @@ The following example uses a memory-consuming SQL statement to demonstrate the d
{{< copyable "sql" >}}

```sql
set tidb_executor_concurrency = 1;
SET tidb_executor_concurrency = 1;
```

5. Execute the same SQL statement. You can find that this time, the statement is successfully executed and no error message is returned. From the following detailed execution plan, you can see that HashAgg has used 600 MB of hard disk space.
Expand Down
2 changes: 1 addition & 1 deletion dashboard/dashboard-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ If your deployment tool is TiUP, take the following steps to solve this problem.

### An `invalid connection` error is shown on the **Slow Queries** page

The possible reason is that you have enabled the `prepared-plan-cache` feature of TiDB. As an experimental feature, when enabled, `prepared-plan-cache` might not function properly in specific TiDB versions, which could cause this problem in TiDB Dashboard (and other applications). You can disable `prepared-plan-cache` by updating [TiDB Configuration file](/tidb-configuration-file.md#prepared-plan-cache) to solve this problem.
The possible reason is that you have enabled the Prepared Plan Cache feature of TiDB. As an experimental feature, when enabled, Prepared Plan Cache might not function properly in specific TiDB versions, which could cause this problem in TiDB Dashboard (and other applications). You can disable Prepared Plan Cache by setting the system variable `tidb_enable_prepared_plan_cache = OFF`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As an experimental feature, when enabled, Prepared Plan Cache might not function properly in specific TiDB versions

I think we should update this part to declare that prepared plan cache has been GAed since v5.3.0, it could be something like 'before v5.3, Prepared Plan Cache is an experimental feature...' @qw4990 PTAL

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't recommend it to be honest. It becomes hard to read if the docs tells too much history. The MySQL manual has a policy of removing/rewriting version specific messages as of the next release. So it would document in the v5.3 docs that the feature has GA'ed, but the 5.4 docs would make no mention of this event.


### A `required component NgMonitoring is not started` error is shown

Expand Down
1 change: 0 additions & 1 deletion dynamic-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -318,7 +318,6 @@ The following TiDB configuration items can be modified online:

| Configuration item | SQL variable | Description |
| :--- | :--- |
| `mem-quota-query` | `tidb_mem_quota_query` | The memory usage limit of a query |
| `log.enable-slow-log` | `tidb_enable_slow_log` | Whether to enable slow log |
| `log.slow-threshold` | `tidb_slow_log_threshold` | The threshold of slow log |
| `log.expensive-threshold` | `tidb_expensive_query_time_threshold` | The threshold of a expensive query |
2 changes: 1 addition & 1 deletion enable-disk-spill-encrypt.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ summary: Learn how to enable encryption for disk spill in TiDB.

# Enable Encryption for Disk Spill

When the `oom-use-tmp-storage` configuration item is set to `true`, if the memory usage of a single SQL statement exceeds the limit of `mem-quota-query` setting, some operators can save the intermediate results during execution as a temporary file to the disk and delete the file after the query is completed.
When the `oom-use-tmp-storage` configuration item is set to `true`, if the memory usage of a single SQL statement exceeds the limit of the system variable [`tidb_mem_quota_query`](/system-variables.md#tidb_mem_quota_query), some operators can save the intermediate results during execution as a temporary file to the disk and delete the file after the query is completed.

You can enable encryption for disk spill to prevent attackers from accessing data by reading these temporary files.

Expand Down
2 changes: 1 addition & 1 deletion enable-tls-between-clients-and-servers.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ To use connections secured with TLS, you first need to configure the TiDB server

Similar to MySQL, TiDB allows TLS and non-TLS connections on the same TCP port. For a TiDB server with TLS enabled, you can choose to securely connect to the TiDB server through an encrypted connection, or to use an unencrypted connection. You can use the following ways to require the use of secure connections:

+ Configure the launch parameter `--require-secure-transport` to require secure connections to the TiDB server for all users.
+ Configure the system variable `require_secure_transport` to require secure connections to the TiDB server for all users.
+ Specify `REQUIRE SSL` when you create a user (`create user`), or modify an existing user (`alter user`), which is to specify that specified users must use the encrypted connection to access TiDB. The following is an example of creating a user:

{{< copyable "sql" >}}
Expand Down
2 changes: 1 addition & 1 deletion error-codes.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ TiDB is compatible with the error codes in MySQL, and in most cases returns the

The memory used by the request exceeds the threshold limit for the TiDB memory usage.

Increase the memory limit for a single SQL statement by configuring `mem-quota-query`.
Increase the memory limit for a single SQL statement by configuring the system variable [`tidb_mem_quota_query`](/system-variables.md#tidb_mem_quota_query).

* Error Number: 8002

Expand Down
2 changes: 2 additions & 0 deletions faq/sql-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,8 @@ Trigger strategy: `auto analyze` is automatically triggered when the number of r

When the modified number or the current total row number is larger than `tidb_auto_analyze_ratio`, the `analyze` statement is automatically triggered. The default value of `tidb_auto_analyze_ratio` is 0.5, indicating that this feature is enabled by default. To ensure safety, its minimum value is 0.3 when the feature is enabled, and it must be smaller than `pseudo-estimate-ratio` whose default value is 0.8, otherwise pseudo statistics will be used for a period of time. It is recommended to set `tidb_auto_analyze_ratio` to 0.5.

Auto analyze can be disabled with the system variable `tidb_enable_auto_analyze`.

## Can I use hints to override the optimizer behavior?

TiDB supports multiple ways to override the default query optimizer behavior, including [hints](/optimizer-hints.md) and [SQL Plan Management](/sql-plan-management.md). The basic usage is similar to MySQL, with several TiDB specific extensions:
Expand Down
2 changes: 1 addition & 1 deletion grafana-tidb-dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ To understand the key metrics displayed on the TiDB dashboard, check the followi
- Transaction Statement Num: the number of SQL statements in a transaction
- Transaction Retry Num: the number of times that a transaction retries
- Session Retry Error OPS: the number of errors encountered during the transaction retry per second. This metric includes two error types: retry failure and exceeding the maximum number of retries
- Commit Token Wait Duration: the wait duration in the flow control queue during the transaction commit. If the wait duration is long, it means that the transaction to commit is too large and the flow is controlled. If the system still has resources available, you can speed up the commit process by increasing the `committer-concurrency` value in the TiDB configuration file
- Commit Token Wait Duration: the wait duration in the flow control queue during the transaction commit. If the wait duration is long, it means that the transaction to commit is too large and the flow is controlled. If the system still has resources available, you can speed up the commit process by increasing the system variable `tidb_committer_concurrency`.
- KV Transaction OPS: the number of transactions executed per second within each TiDB instance
- A user transaction might trigger multiple transaction executions in TiDB, including reading internal metadata, atomic retries of the user transaction, and so on
- TiDB's internally scheduled tasks also operate on the database through transactions, which are also included in this panel
Expand Down
2 changes: 1 addition & 1 deletion identify-expensive-queries.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ aliases: ['/docs/dev/identify-expensive-queries/','/docs/dev/how-to/maintain/ide

# Identify Expensive Queries

TiDB allows you to identify expensive queries during SQL execution, so you can diagnose and improve the performance of SQL execution. Specifically, TiDB prints the information about statements whose execution time exceeds [`tidb_expensive_query_time_threshold`](/system-variables.md#tidb_expensive_query_time_threshold) (60 seconds by default) or memory usage exceeds [`mem-quota-query`](/tidb-configuration-file.md#mem-quota-query) (1 GB by default) to the [tidb-server log file](/tidb-configuration-file.md#logfile) ("tidb.log" by default).
TiDB allows you to identify expensive queries during SQL execution, so you can diagnose and improve the performance of SQL execution. Specifically, TiDB prints the information about statements whose execution time exceeds [`tidb_expensive_query_time_threshold`](/system-variables.md#tidb_expensive_query_time_threshold) (60 seconds by default) or memory usage exceeds [`tidb_mem_quota_query`](/system-variables.md#tidb_mem_quota_query) (1 GB by default) to the [tidb-server log file](/tidb-configuration-file.md#logfile) ("tidb.log" by default).

> **Note:**
>
Expand Down
4 changes: 2 additions & 2 deletions optimizer-hints.md
Original file line number Diff line number Diff line change
Expand Up @@ -319,7 +319,7 @@ In the following hint, `MEMORY_QUOTA(1024 MB)` means that the memory usage is li
select /*+ MEMORY_QUOTA(1024 MB) */ * from t;
```

In addition to this hint, the `tidb_mem_quota_query` system variable can also limit the memory usage of a statement.
In addition to this hint, the [`tidb_mem_quota_query`](/system-variables.md#tidb_mem_quota_query) system variable can also limit the memory usage of a statement.

### READ_CONSISTENT_REPLICA()

Expand All @@ -337,7 +337,7 @@ In addition to this hint, setting the `tidb_replica_read` environment variable t

The `IGNORE_PLAN_CACHE()` hint reminds the optimizer not to use the Plan Cache when handling the current `prepare` statement.

This hint is used to temporarily disable the Plan Cache for a certain type of queries when [prepare-plan-cache](/tidb-configuration-file.md#prepared-plan-cache) is enabled.
This hint is used to temporarily disable the Plan Cache for a certain type of queries when [prepare-plan-cache](/sql-prepared-plan-cache.md) is enabled.

In the following example, the Plan Cache is forcibly disabled when executing the `prepare` statement.

Expand Down
2 changes: 1 addition & 1 deletion releases/release-4.0.10.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ TiDB version: 4.0.10

+ Dumpling

- Avoid TiDB out of memory (OOM) by setting the default behavior of `tidb_mem_quota_query` [#233](https://github.com/pingcap/dumpling/pull/233)
- Avoid TiDB out of memory (OOM) by setting the default behavior of [`tidb_mem_quota_query`](/system-variables.md#tidb_mem_quota_query) [#233](https://github.com/pingcap/dumpling/pull/233)

+ Backup & Restore (BR)

Expand Down
Loading