Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/release-6.3' into pr/11523
Browse files Browse the repository at this point in the history
  • Loading branch information
shichun-0415 committed Dec 5, 2022
2 parents 1760c5b + 433db88 commit 2a6ee66
Show file tree
Hide file tree
Showing 4 changed files with 21 additions and 1 deletion.
22 changes: 21 additions & 1 deletion configure-memory-usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,4 +172,24 @@ The following example uses a memory-consuming SQL statement to demonstrate the d
| └─TableFullScan_16 | 256.00 | 256 | cop[tikv] | table:t1 | tikv_task:{time:71.4µs, loops:256} | keep order:false, stats:pseudo | N/A | N/A |
+---------------------------------+-------------+----------+-----------+---------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------+-----------+----------+
9 rows in set (1 min 37.428 sec)
```
```

## Others

### Mitigate OOM issues by configuring `GOMEMLIMIT`

GO 1.19 introduces an environment variable [`GOMEMLIMIT`](https://pkg.go.dev/runtime@go1.19#hdr-Environment_Variables) to set the memory limit that triggers GC.

For v6.1.3 <= TiDB < v6.5.0, you can mitigate a typical category of OOM issues by manually setting `GOMEMLIMIT`. The typical category of OOM issues is: before OOM occurs, the estimated memory in use on Grafana occupies only half of the entire memory (TiDB-Runtime > Memory Usage > estimate-inuse), as shown in the following figure:

![normal OOM case example](/media/configure-memory-usage-oom-example.png)

To verify the performance of `GOMEMLIMIT`, a test is performed to compare the specific memory usage with and without `GOMEMLIMIT` configuration.

- In TiDB v6.1.2, the TiDB server encounters OOM (system memory: about 48 GiB) after the simulated workload runs for several minutes:

![v6.1.2 workload oom](/media/configure-memory-usage-612-oom.png)

- In TiDB v6.1.3, `GOMEMLIMIT` is set to 40000 MiB. It is found that the simulated workload runs stably for a long time, OOM does not occur in the TiDB server, and the maximum memory usage of the process is stable at around 40.8 GiB:

![v6.1.3 workload no oom with GOMEMLIMIT](/media/configure-memory-usage-613-no-oom.png)
Binary file added media/configure-memory-usage-612-oom.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added media/configure-memory-usage-613-no-oom.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added media/configure-memory-usage-oom-example.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 2a6ee66

Please sign in to comment.