simple (ticdc): support send all tables bootstrap message at changefeed start #11239

asddongmen · 2024-06-04T08:14:44Z

What problem does this PR solve?

Issue Number: close #11315

What is changed and how it works?

A boolean parameter send-all-bootstrap-at-start is added, which is set to false by default.
If this parameter is set to true, the changefeed, upon starting, will send the schema of all synchronized tables downstream as Bootstrap events. This will be done synchronously in the ddlSink. After the Bootstrap events are synchronized, row changes will begin to be synchronized.

Check List

Tests

Unit test
Manual test (add detailed scripts or steps below)

Questions

Will it cause performance regression or break compatibility?

Do you need to update user documentation, design documentation or monitoring documentation?

Release note

None

Signed-off-by: dongmen <414110582@qq.com>

ti-chi-bot · 2024-06-04T08:14:46Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

Signed-off-by: dongmen <414110582@qq.com>

…p_at_beginning

Signed-off-by: dongmen <414110582@qq.com>

asddongmen · 2024-06-13T09:36:44Z

/test all

Signed-off-by: dongmen <414110582@qq.com>

3AceShowHand · 2024-06-14T10:20:56Z

/retest

Signed-off-by: dongmen <414110582@qq.com>

codecov · 2024-06-17T02:35:05Z

Codecov Report

Attention: Patch coverage is 68.00000% with 24 lines in your changes missing coverage. Please review.

Project coverage is 57.7412%. Comparing base (999d393) to head (8ea0f6d).
Report is 6 commits behind head on master.

Additional details and impacted files

Components	Coverage Δ
cdc	`61.4548% <68.0000%> (+0.0225%)`	⬆️
dm	`51.4356% <ø> (+0.2325%)`	⬆️
engine	`63.5518% <ø> (+0.1638%)`	⬆️

Flag	Coverage Δ
unit	`57.7412% <68.0000%> (+0.1098%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

@@               Coverage Diff                @@
##             master     #11239        +/-   ##
================================================
+ Coverage   57.6313%   57.7412%   +0.1098%     
================================================
  Files           850        850                
  Lines        126256     126634       +378     
================================================
+ Hits          72763      73120       +357     
- Misses        48088      48092         +4     
- Partials       5405       5422        +17

asddongmen · 2024-06-17T06:52:53Z

/retest

cdc/owner/ddl_sink.go

cdc/owner/ddl_manager.go

Signed-off-by: dongmen <414110582@qq.com>

hongyunyan

rest is LGTM

cdc/owner/ddl_manager.go

cdc/owner/ddl_manager_test.go

cdc/owner/ddl_sink.go

ti-chi-bot · 2024-06-18T02:01:42Z

[LGTM Timeline notifier]

Timeline:

2024-06-17 08:05:47.002720788 +0000 UTC m=+15673.488209632: ☑️ agreed by 3AceShowHand.
2024-06-18 02:01:41.785244765 +0000 UTC m=+80228.270733597: ☑️ agreed by hongyunyan.

Signed-off-by: dongmen <414110582@qq.com>

asddongmen · 2024-06-18T02:56:45Z

/retest

asddongmen · 2024-06-18T04:10:35Z

/retest

ti-chi-bot · 2024-06-20T20:28:53Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: 3AceShowHand, benmeadowcroft, hongyunyan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [3AceShowHand,hongyunyan]
~~cdc/api/v2/OWNERS~~ [benmeadowcroft]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2024-07-10T08:26:48Z

In response to a cherrypick label: new pull request created to branch release-7.5: #11400.

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

…ed start (#11239) (#11400) close #11315

ti-chi-bot · 2024-07-30T08:35:32Z

In response to a cherrypick label: new pull request created to branch release-8.1: #11439.

…ed start (#11239) (#11439) close #11315

add config send-all-bootstrap-at-start

cd8c86a

Signed-off-by: dongmen <414110582@qq.com>

asddongmen added 3 commits June 6, 2024 09:40

check bootstrap sending in ddlManager'c tick

b0859df

Signed-off-by: dongmen <414110582@qq.com>

Merge remote-tracking branch 'upstream/master' into send_all_bootstra…

d36294e

…p_at_beginning

fix unit test error

c10f6b8

Signed-off-by: dongmen <414110582@qq.com>

asddongmen added 2 commits June 14, 2024 17:25

add unit test case

8058293

Signed-off-by: dongmen <414110582@qq.com>

add config to integretion test

bd8e999

Signed-off-by: dongmen <414110582@qq.com>

ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/needs-linked-issue release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Jun 14, 2024

asddongmen requested a review from 3AceShowHand June 14, 2024 09:31

asddongmen marked this pull request as ready for review June 14, 2024 10:04

ti-chi-bot bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 14, 2024

asddongmen added 2 commits June 14, 2024 18:29

fix make check

43c7a55

Signed-off-by: dongmen <414110582@qq.com>

fix ut

732a5fa

Signed-off-by: dongmen <414110582@qq.com>

asddongmen requested a review from sdojjy June 17, 2024 02:15

asddongmen self-assigned this Jun 17, 2024

asddongmen added the require-LGT2 This PR needs to get two LGTMs before merging. label Jun 17, 2024

hongyunyan reviewed Jun 17, 2024

View reviewed changes

cdc/owner/ddl_sink.go Outdated Show resolved Hide resolved

cdc/owner/ddl_manager.go Outdated Show resolved Hide resolved

3AceShowHand approved these changes Jun 17, 2024

View reviewed changes

ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Jun 17, 2024

resolve comments

4fb9bcd

Signed-off-by: dongmen <414110582@qq.com>

hongyunyan approved these changes Jun 18, 2024

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jun 18, 2024

resolve comments 2

8ea0f6d

Signed-off-by: dongmen <414110582@qq.com>

asddongmen removed the require-LGT2 This PR needs to get two LGTMs before merging. label Jun 18, 2024

benmeadowcroft approved these changes Jun 20, 2024

View reviewed changes

ti-chi-bot bot added the approved label Jun 20, 2024

ti-chi-bot bot merged commit 58636ed into pingcap:master Jun 20, 2024
27 of 28 checks passed

ti-chi-bot added the needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. label Jul 10, 2024

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Jul 10, 2024

This is an automated cherry-pick of pingcap#11239

a9d0544

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>

ti-chi-bot mentioned this pull request Jul 10, 2024

simple (ticdc): support send all tables bootstrap message at changefeed start (#11239) #11400

Merged

ti-chi-bot bot pushed a commit that referenced this pull request Jul 24, 2024

simple (ticdc): support send all tables bootstrap message at changefe…

b00c3ee

…ed start (#11239) (#11400) close #11315

ti-chi-bot added the needs-cherry-pick-release-8.1 Should cherry pick this PR to release-8.1 branch. label Jul 30, 2024

ti-chi-bot mentioned this pull request Jul 30, 2024

simple (ticdc): support send all tables bootstrap message at changefeed start (#11239) #11439

Merged

ti-chi-bot bot pushed a commit that referenced this pull request Aug 2, 2024

simple (ticdc): support send all tables bootstrap message at changefe…

8a8bed9

…ed start (#11239) (#11439) close #11315

3AceShowHand mentioned this pull request Sep 11, 2024

changefeed(ticdc): send bootstrap message asynchronously to prevent block other changefeeds #11573

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

simple (ticdc): support send all tables bootstrap message at changefeed start #11239

simple (ticdc): support send all tables bootstrap message at changefeed start #11239

asddongmen commented Jun 4, 2024 •

edited

Loading

ti-chi-bot bot commented Jun 4, 2024

asddongmen commented Jun 13, 2024

3AceShowHand commented Jun 14, 2024

codecov bot commented Jun 17, 2024 •

edited

Loading

asddongmen commented Jun 17, 2024

hongyunyan left a comment

ti-chi-bot bot commented Jun 18, 2024

asddongmen commented Jun 18, 2024

asddongmen commented Jun 18, 2024

ti-chi-bot bot commented Jun 20, 2024

ti-chi-bot commented Jul 10, 2024

ti-chi-bot commented Jul 30, 2024

simple (ticdc): support send all tables bootstrap message at changefeed start #11239

simple (ticdc): support send all tables bootstrap message at changefeed start #11239

Conversation

asddongmen commented Jun 4, 2024 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

Tests

Questions

Will it cause performance regression or break compatibility?

Do you need to update user documentation, design documentation or monitoring documentation?

Release note

ti-chi-bot bot commented Jun 4, 2024

asddongmen commented Jun 13, 2024

3AceShowHand commented Jun 14, 2024

codecov bot commented Jun 17, 2024 • edited Loading

Codecov Report

asddongmen commented Jun 17, 2024

hongyunyan left a comment

Choose a reason for hiding this comment

ti-chi-bot bot commented Jun 18, 2024

[LGTM Timeline notifier]

asddongmen commented Jun 18, 2024

asddongmen commented Jun 18, 2024

ti-chi-bot bot commented Jun 20, 2024

ti-chi-bot commented Jul 10, 2024

ti-chi-bot commented Jul 30, 2024

asddongmen commented Jun 4, 2024 •

edited

Loading

codecov bot commented Jun 17, 2024 •

edited

Loading