Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Checkpoint lag gets larger as the number captured tables increases #3900

Closed
overvenus opened this issue Dec 16, 2021 · 0 comments · Fixed by #3899
Closed

Checkpoint lag gets larger as the number captured tables increases #3900

overvenus opened this issue Dec 16, 2021 · 0 comments · Fixed by #3899
Labels
area/ticdc Issues or PRs related to TiCDC. component/sink Sink component. severity/minor type/bug The issue is confirmed as a bug.

Comments

@overvenus
Copy link
Member

What did you do?

1k tables 2k tables
image image
image image

What did you expect to see?

Checkpoint lag should be within 10s when capturing 2k tables.

What did you see instead?

Checkpoint lag is much larger than 10s when capturing 2k tables.

Versions of the cluster

Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

TiCDC version (execute cdc version):

https://github.com/pingcap/ticdc/tree/d381504a9fc3e5e6615e17627c424ec5bd17b0b4
@overvenus overvenus added type/bug The issue is confirmed as a bug. component/sink Sink component. severity/minor area/ticdc Issues or PRs related to TiCDC. labels Dec 16, 2021
@overvenus overvenus mentioned this issue Dec 16, 2021
Merged
This was referenced Dec 17, 2021
Closed
Closed
Closed
Closed
Closed
@overvenus overvenus mentioned this issue Jan 13, 2022
Merged
14 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/ticdc Issues or PRs related to TiCDC. component/sink Sink component. severity/minor type/bug The issue is confirmed as a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

sink(ticdc): optimize buffer sink flush from O(N^2) to O(N) overvenus/ticdc
1 participant
@overvenus