Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

schema snapshot takes a long time to initialize when synchronize 150k tables in single changefeed #11207

Closed
CharlesCheung96 opened this issue May 30, 2024 · 1 comment · Fixed by #11217
Closed

schema snapshot takes a long time to initialize when synchronize 150k tables in single changefeed #11207

CharlesCheung96 opened this issue May 30, 2024 · 1 comment · Fixed by #11217
Assignees
@asddongmen
Labels
affects-7.5 affects-8.1 area/ticdc Issues or PRs related to TiCDC. severity/moderate type/enhancement The issue or PR belongs to an enhancement.

Comments

@CharlesCheung96
Copy link
Contributor

CharlesCheung96 commented May 30, 2024 •
edited
Loading

What did you do?

Update a changefeed which synchronize 150k tables.

  1. In master
    image

  2. Revert snapshot (ticdc): reduce list tables time consumption #11095
    image

The new version took about 4 times as long to initialize. So it may be necessary to add a parameter to keep the LTS version in its original behavior, or to do further optimization.

Versions of the cluster

TiCDC version (execute cdc version):

7fbae794a5cf5f1e6608927abbd67d579dbc0c0f
(paste TiCDC version here)
@CharlesCheung96 CharlesCheung96 added type/bug The issue is confirmed as a bug. area/ticdc Issues or PRs related to TiCDC. labels May 30, 2024
@fubinzh
Copy link

fubinzh commented May 30, 2024

/severity major

@asddongmen asddongmen self-assigned this May 31, 2024
@asddongmen asddongmen mentioned this issue May 31, 2024
Merged
@ti-chi-bot ti-chi-bot bot closed this as completed in 4860903 Jun 17, 2024
@ti-chi-bot ti-chi-bot mentioned this issue Jul 10, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue Jul 18, 2024
@ti-chi-bot
entry (ticdc): speed up schema snapshot initialization (#11217) (#11399)
344ad4d 
close #11207
@flowbehappy flowbehappy added type/enhancement The issue or PR belongs to an enhancement. and removed type/bug The issue is confirmed as a bug. labels Jul 30, 2024
@ti-chi-bot ti-chi-bot mentioned this issue Jul 30, 2024
Merged
ti-chi-bot bot pushed a commit that referenced this issue Aug 5, 2024
@ti-chi-bot
entry (ticdc): speed up schema snapshot initialization (#11217) (#11437)
a3aa5c0 
close #11207
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@asddongmen asddongmen

Labels
affects-7.5 affects-8.1 area/ticdc Issues or PRs related to TiCDC. severity/moderate type/enhancement The issue or PR belongs to an enhancement.
Projects
Question and Bug Reports
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

entry (ticdc): speed up schema snapshot initialization asddongmen/tiflow
4 participants
@flowbehappy @fubinzh @asddongmen @CharlesCheung96