Skip to content

Commit

Permalink
Improve the documentation based on official feedback
Browse files Browse the repository at this point in the history
  • Loading branch information
wangbin579 committed Nov 3, 2024
1 parent 742a6bb commit 9dc4dc4
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 15 deletions.
6 changes: 3 additions & 3 deletions cats.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ Given this context, several considerations emerge:
1. **Impact on Performance Testing:** The extensive error logs and the resulting disruptions could potentially skew the performance evaluation, leading to inaccurate assessments of the system's capabilities.
2. **Effectiveness of the CATS Algorithm:** The performance improvement of the CATS algorithm may need re-evaluation. If the extensive output of error logs significantly impacts performance, its actual effectiveness may not be as high as initially believed.

Remove all logs from the **Deadlock_notifier::notify** function, recompile MySQL, and perform SysBench read-write tests under Pareto distribution. Details are provided in the following figure:
Set `innodb_print_all_deadlocks=OFF` or remove all logging from the `Deadlock_notifier::notify` function, recompile MySQL, and run SysBench read-write tests with a Pareto distribution. Details are provided in the following figure:

<img src="images/image-20240829101534550.png" alt="image-20240829101534550" style="zoom:150%;" />

Expand All @@ -121,7 +121,7 @@ Let's conduct performance comparison tests on the improved MySQL 8.0.32, with de

Figure 9. Impact of CATS on throughput at various concurrency levels for improved MySQL 8.0.32 after eliminating interference.

From the figure, it is evident that removing the interference results in only a slight performance difference. This small variation makes it understandable why the severity of FIFO scheduling problems may be difficult to notice. The perceived bias from CATS authors and MySQL officials likely stems from the extensive log output interference caused by deadlocks.
From the figure, it is evident that removing the interference results in only a slight performance difference. This small variation makes it understandable why the severity of FIFO scheduling problems may be difficult to notice. The perceived bias from the CATS author and MySQL officials is likely due to interference from extensive deadlock log output.

Using the same 32 warehouses as in the CATS algorithm paper, TPC-C tests were conducted at various concurrency levels. MySQL was based on the improved MySQL 8.0.27, and BenchmarkSQL was modified to support 100 concurrent transactions per warehouse.

Expand Down Expand Up @@ -150,7 +150,7 @@ Figure 12. Impact of CATS on BenchmarkSQL throughput after eliminating interfere

From the figure, it is evident that there is little difference between the two algorithms in low-conflict scenarios. In other words, the CATS algorithm does not offer significant benefits in situations with fewer conflicts.

Overall, while CATS shows some improvement in Pareto testing, it is less pronounced than expected. The interference from deadlock log outputs during performance testing impacted the results. The CATS algorithm significantly reduces transaction deadlocks, leading to fewer log outputs and less performance degradation compared to the FIFO algorithm. When deadlock logs are suppressed, the difference between these algorithms is minimal, clarifying the confusion surrounding the CATS algorithm's performance [4].
Overall, while CATS shows some improvement in Pareto testing, it is less pronounced than expected. The CATS algorithm significantly reduces transaction deadlocks, potentially resulting in less performance degradation than the FIFO algorithm. When deadlock logs are suppressed, the difference between these algorithms is minimal, clarifying the confusion surrounding the CATS algorithm's performance [4].

Database performance testing is inherently complex and error-prone [5]. It cannot be judged by data alone and requires thorough investigation to ensure logical consistency.

Expand Down
22 changes: 10 additions & 12 deletions scalability.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# How can the scalability of MySQL be improved?

## Current state of MySQL 5.7
## Current state of MySQL 5.7

MySQL 5.7 is not ideal in terms of scalability. The following figure illustrates the relationship between TPC-C throughput and concurrency in MySQL 5.7.39 under a specific configuration. This includes setting the transaction isolation level to Read Committed and adjusting the *innodb_spin_wait_delay* parameter to mitigate throughput degradation.

Expand Down Expand Up @@ -30,7 +30,7 @@ The first major improvement is redo log optimization [3].
commit 6be2fa0bdbbadc52cc8478b52b69db02b0eaff40
Author: Paweł Olchawa <pawel.olchawa@oracle.com>
Date: Wed Feb 14 09:33:42 2018 +0100

WL#10310 Redo log optimization: dedicated threads and concurrent log buffer.

0. Log buffer became a ring buffer, data inside is no longer shifted.
Expand Down Expand Up @@ -76,7 +76,7 @@ A test comparing TPC-C throughput with different levels of concurrency before an
Figure 3. Impact of redo log optimization under different concurrency levels.
The results in the figure show a significant improvement in throughput at low concurrency levels.
The results in the figure show a significant improvement in throughput at a concurrency level of 100.
### Optimizing Lock-Sys Through Latch Sharding
Expand All @@ -86,7 +86,7 @@ The second major improvement is lock-sys optimization [5].
commit 1d259b87a63defa814e19a7534380cb43ee23c48
Author: Jakub Łopuszański <jakub.lopuszanski@oracle.com>
Date: Wed Feb 5 14:12:22 2020 +0100
WL#10314 - InnoDB: Lock-sys optimization: sharded lock_sys mutex
The Lock-sys orchestrates access to tables and rows. Each table, and each row,
Expand All @@ -95,19 +95,18 @@ Date: Wed Feb 5 14:12:22 2020 +0100
problems if the two operations conflict with each other, Lock-sys remembers
lists of already GRANTED lock requests and checks new requests for conflicts in
which case they have to start WAITING for their turn.
Lock-sys stores both GRANTED and WAITING lock requests in lists known as queues.
To allow concurrent operations on these queues, we need a mechanism to latch
these queues in safe and quick fashion.
In the past a single latch protected access to all of these queues.
This scaled poorly, and the managment of queues become a bottleneck.
In this WL, we introduce a more granular approach to latching.
Reviewed-by: Pawel Olchawa <pawel.olchawa@oracle.com>
Reviewed-by: Debarun Banerjee <debarun.banerjee@oracle.com>
RB:23836
```

Based on the program before and after optimizing with lock-sys, using BenchmarkSQL to compare TPC-C throughput with concurrency, the specific results are as shown in the following figure:
Expand All @@ -126,9 +125,9 @@ The third major improvement is latch sharding for trx-sys.
commit bc95476c0156070fd5cedcfd354fa68ce3c95bdb
Author: Paweł Olchawa <pawel.olchawa@oracle.com>
Date: Tue May 25 18:12:20 2021 +0200

BUG#32832196 SINGLE RW_TRX_SET LEADS TO CONTENTION ON TRX_SYS MUTEX

1. Introduced shards, each with rw_trx_set and dedicated mutex.
2. Extracted modifications to rw_trx_set outside its original critical sections
(removal had to be extracted outside trx_erase_lists).
Expand All @@ -137,7 +136,7 @@ Date: Tue May 25 18:12:20 2021 +0200
fields to avoid risk of torn reads on egzotic platforms.
5. Added assertions which ensure that thread operating on transaction has rights
to do so (to show there is no possible race condition).

RB: 26314
Reviewed-by: Jakub Łopuszański jakub.lopuszanski@oracle.com
```
Expand Down Expand Up @@ -205,4 +204,3 @@ Overall, it is entirely feasible for MySQL to maintain performance without colla
[4] Xiangyao Yu. An evaluation of concurrency control with one thousand cores. PhD thesis, Massachusetts Institute of Technology, 2015.
[5] https://dev.mysql.com/doc/refman/8.0/en/.

0 comments on commit 9dc4dc4

Please sign in to comment.