-
Notifications
You must be signed in to change notification settings - Fork 8.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
global_table 2分10s延迟删除太慢了,在高并发下,这张表数据太多了,有啥优化方案吗 #6615
Comments
|
我设想了以下几种方案:
By using sharding, assuming there are 3 TC nodes, each TC node's corresponding global table name is set to global_table01 and so on. Then, remove the distributed-lock-table parameter so that distributed locking is not enabled. This way, parallel delayed deletion can be achieved by taking TC nodes and their corresponding global tables as dimensions. By combining the capabilities of Raft, DB, Redis, etc., internal or indirect communication can be achieved. Raft: After setting up a Raft cluster, the leader periodically scans transactions in the committing and rollbacking states. Transactions that have reached the 2-minute 10-second requirement are divided among the nodes. For example, if there are 3 TC nodes and a total of 3000 transactions need delayed deletion, each TC node will handle 1000 transactions assigned by the leader. This increases concurrency. |
结论:1.先解决单节点的问题,使用多线程的方式 2. 先不往复杂分布式的情况考虑 |
Ⅰ. Issue Description
global_table 2分10s延迟删除太慢了,在高并发下,这张表数据太多了,有啥优化方案吗?
除了调低这个130s的参数,还有其他方案吗?
如果等它低频的时候删除完毕,我很担心在高频的时候随着global_table的堆积,影响tps,浪费数据库磁盘空间
参考图:
Ⅱ. Describe what happened
If there is an exception, please attach the exception trace:
Ⅲ. Describe what you expected to happen
Ⅳ. How to reproduce it (as minimally and precisely as possible)
Minimal yet complete reproducer code (or URL to code):
Ⅴ. Anything else we need to know?
Ⅵ. Environment:
java -version
):uname -a
):The text was updated successfully, but these errors were encountered: