Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

owner(ticdc): update changefeed configuration changefeed-error-stuck-duration correctly #11042

Merged
merged 7 commits into from
May 13, 2024

Conversation

wk989898
Copy link
Collaborator

@wk989898 wk989898 commented May 7, 2024

What problem does this PR solve?

Issue Number: close #10998

What is changed and how it works?

When changefeed state has changed, reacquiring changefeed-error-stuck-duration. feedStateManager checks for new value about changefeedErrorStuckDuration every feedStateManager.Tick call so that it can sense the updated value.

Check List

Tests

  • Unit test
  • Manual test (add detailed scripts or steps below)
    1. start a CDC server
    2. run script
    bin/cdc cli changefeed create -c test --sink-uri "kafka://127.0.0.1:9092/test?topic=test&protocol=open-protocol&max-message-bytes=41943040&compressionType=ZLIB" --changefeed-id="test"
    bin/cdc cli changefeed pause --changefeed-id="test"
    bin/cdc cli changefeed update --changefeed-id="test" --config=changefeed.toml
    bin/cdc cli changefeed resume --changefeed-id="test"
    changefeed.toml
     changefeed-error-stuck-duration = "1h30m"
    1. check CDC log
    Screenshot 2024-05-11 at 17 05 18

Questions

Will it cause performance regression or break compatibility?

No

Do you need to update user documentation, design documentation or monitoring documentation?

No

Release note

None

@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. contribution This PR is from a community contributor. labels May 7, 2024
Copy link
Contributor

ti-chi-bot bot commented May 7, 2024

Hi @wk989898. Thanks for your PR.

I'm waiting for a pingcap member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ti-chi-bot ti-chi-bot bot added needs-ok-to-test Indicates a PR created by contributors and need ORG member send '/ok-to-test' to start testing. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels May 7, 2024
@CharlesCheung96
Copy link
Contributor

/test all

@@ -168,6 +168,15 @@ func (m *feedStateManager) Tick(resolvedTs model.Ts,
// `handleAdminJob` returns true means that some admin jobs are pending
// skip to the next tick until all the admin jobs is handled
adminJobPending = true
changefeedErrorStuckDuration := *m.state.GetChangefeedInfo().Config.ChangefeedErrorStuckDuration
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use util.GetOrZero to avoid possible nil pointer dereference

Copy link

codecov bot commented May 8, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 57.3879%. Comparing base (894b36f) to head (ce895c0).
Report is 10 commits behind head on master.

Additional details and impacted files
Components Coverage Δ
cdc 61.6153% <100.0000%> (-0.0494%) ⬇️
dm 51.2472% <ø> (+0.0095%) ⬆️
engine 63.4585% <ø> (+0.0706%) ⬆️
Flag Coverage Δ
unit 57.3879% <100.0000%> (-0.0088%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@@               Coverage Diff                @@
##             master     #11042        +/-   ##
================================================
- Coverage   57.3966%   57.3879%   -0.0088%     
================================================
  Files           851        851                
  Lines        125469     125617       +148     
================================================
+ Hits          72015      72089        +74     
- Misses        48034      48115        +81     
+ Partials       5420       5413         -7     

@ti-chi-bot ti-chi-bot bot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels May 9, 2024
@asddongmen asddongmen added needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. needs-cherry-pick-release-8.1 Should cherry pick this PR to release-8.1 branch. labels May 9, 2024
@asddongmen
Copy link
Contributor

/ok-to-test

@ti-chi-bot ti-chi-bot bot added ok-to-test Indicates a PR is ready to be tested. and removed needs-ok-to-test Indicates a PR created by contributors and need ORG member send '/ok-to-test' to start testing. labels May 9, 2024
@asddongmen
Copy link
Contributor

Please also add an unit test case to cover this change.

@ti-chi-bot ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels May 9, 2024
@wk989898
Copy link
Collaborator Author

wk989898 commented May 9, 2024

/test all

@wk989898 wk989898 requested a review from asddongmen May 10, 2024 11:23
@ti-chi-bot ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label May 12, 2024
@ti-chi-bot ti-chi-bot bot added the approved label May 12, 2024
@wk989898
Copy link
Collaborator Author

/retest

Copy link
Contributor

ti-chi-bot bot commented May 13, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: asddongmen, CharlesCheung96

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [CharlesCheung96,asddongmen]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels May 13, 2024
Copy link
Contributor

ti-chi-bot bot commented May 13, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-05-12 23:47:32.668209341 +0000 UTC m=+1437806.425344910: ☑️ agreed by asddongmen.
  • 2024-05-13 08:52:38.678043967 +0000 UTC m=+1470512.435179540: ☑️ agreed by CharlesCheung96.

@asddongmen
Copy link
Contributor

/retest

@ti-chi-bot ti-chi-bot bot merged commit 985f8af into pingcap:master May 13, 2024
28 checks passed
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-7.5: #11083.

@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created to branch release-8.1: #11085.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved contribution This PR is from a community contributor. lgtm needs-cherry-pick-release-7.5 Should cherry pick this PR to release-7.5 branch. needs-cherry-pick-release-8.1 Should cherry pick this PR to release-8.1 branch. ok-to-test Indicates a PR is ready to be tested. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

update of changefeed configiration changefeed-error-stuck-duration not working
4 participants