ddl_puller (ticdc): handle dorp pk/uk ddl correctly

[asddongmen]

What problem does this PR solve?

Issue Number: close #10890

What is changed and how it works?

When handling a DDL job in ddlPuller, check if it will transition the table from eligible to ineligible. If so, return an error and stop the changefeed. The error message should provide instructions for the user on how to manage the situation.

Check List

Tests

• Unit test
• Manual test (add detailed scripts or steps below)

1. Create a table t1.

CREATE TABLE t1 (
id INT PRIMARY KEY /*T![clustered_index] NONCLUSTERED */,
name VARCHAR(255),
email VARCHAR(255) UNIQUE
);

3. Drop its primary key.

ALTER TABLE t1 DROP PRIMARY KEY; 

4. List this changefeed.

-> # ./cdc cli changefeed list 
[
  {
    "id": "test",
    "namespace": "default",
    "summary": {
      "state": "warning",
      "tso": 449299724779651075,
      "checkpoint": "2024-04-24 15:07:11.563",
      "error": {
        "time": "2024-04-24T16:52:17.129593307+08:00",
        "addr": "127.0.0.1:8300",
        "code": "CDC:ErrProcessorUnknown",
        "message": "An eligible table become ineligible after DDL: [alter table t1 drop primary key] it is a dangerous operation. If you want to replicate this ddl, pelase pause the changefeed and update the `force-replicate=true` in the changefeed configuration, then resume the changefeed. Otherwise, will cause data loss."
      }
    }
  }
]

5. Update the changefeed's config as the instructions in the error message and resume the changefeed. Check the CDC log to confirm if the DDL was replicated.

[2024/04/24 16:55:09.679 +08:00] [INFO] [ddl_manager.go:558] ["execute a ddl event successfully"] [ddl="ALTER TABLE `t1` DROP PRIMARY KEY"] [namespace=default] [changefeed=test] ....

Questions

Will it cause performance regression or break compatibility?

Do you need to update user documentation, design documentation or monitoring documentation?

Release note

Fix a bug that might cause the replication of the 'drop primary key' or 'drop unique key' DDL to be lost.

[ti-chi-bot]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[asddongmen]

/test verify

[asddongmen]

/test all

[asddongmen]

/review

[asddongmen]

/test verify

[asddongmen]

/test verify

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: CharlesCheung96, sdojjy

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [CharlesCheung96,sdojjy]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2024-04-25 10:12:08.432744559 +0000 UTC m=+255085.172647469: ☑️ agreed by CharlesCheung96.
• 2024-04-25 10:46:44.144215404 +0000 UTC m=+257160.884118308: ☑️ agreed by sdojjy.

[asddongmen]

/retest

[ti-chi-bot]

In response to a cherrypick label: new pull request created to branch release-6.5: #10981.

[ti-chi-bot]

In response to a cherrypick label: new pull request created to branch release-7.1: #10982.

[ti-chi-bot]

In response to a cherrypick label: new pull request created to branch release-7.5: #10983.

[ti-chi-bot]

In response to a cherrypick label: new pull request created to branch release-8.1: #10984.

[wuhuizuo]

/review summary the changes.

[ti-chi-bot]

@wuhuizuo:

Pull Request Review

Title

"ddl_puller (ticdc): handle drop primary key/unique key DDL correctly"

Changes Made

This pull request focuses on handling a DDL job in the ddlPuller by checking if it will transition the table from eligible to ineligible. If so, an error is returned and the changefeed is stopped. The error message provides instructions for the user on how to manage the situation.

The changes include:

1. Check if the DDL event belongs to an ineligible table. If so, ignore it.
2. Add a new function checkIneligibleTableDDL to check if the table is ineligible before and after the DDL.
3. Adjust tests to match the new changes.

The main changes are in the following files:

• cdc/owner/ddl_manager.go
• cdc/puller/ddl_puller.go
• cdc/puller/ddl_puller_test.go

Tests

The changes include unit tests and manual tests. The manual test is detailed and checks various scenarios including creating a table, dropping its primary key, listing the changefeed, and updating the changefeed's config.

Release Note

Fix a bug that might cause the replication of the 'drop primary key' or 'drop unique key' DDL to be lost.

In response to this:

/review summary the changes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[wuhuizuo]

/review default

[ti-chi-bot]

@wuhuizuo:

Review Report

General

The pull request is intended to solve the issue of correctly handling the DDL operations when a primary key or a unique key is dropped from a table in the TiFlow system. The PR checks if the table transitions from eligible to ineligible after a DDL operation and provides an error message with instructions if it does so.

Coding Style

The code follows the coding style of the project and is well-formatted. Variable and function names are descriptive and self-explanatory.

Logic

The logic of the changes seems correct. The function checkIneligibleTableDDL checks whether a table is ineligible before and after a DDL operation. If the table becomes ineligible after the operation, an error is returned. The unit tests cover all the cases and therefore the logic is sound.

Tests

The PR includes both unit tests and manual tests, which is very good. The unit tests cover the different scenarios and seem to be comprehensive. It's great that the manual tests are also included in the PR as they guide users to test the feature manually.

Potential Issues

1. There is a minor typo in the error message: "pelase pause the changefeed" should be "please pause the changefeed".
2. There are some spelling mistakes in the comments, like "dorp" instead of "drop".

Suggested Changes

1. Correct the spelling mistakes in the error message and comments.
2. It seems that the PR does not handle the case where a DDL operation makes an ineligible table eligible. Although this might be a rare case, it would be good to handle it for completeness.

```markdown
# Review Report

## General

The pull request is intended to solve the issue of correctly handling the DDL operations when a primary key or a unique key is dropped from a table in the TiFlow system. The PR checks if the table transitions from eligible to ineligible after a DDL operation and provides an error message with instructions if it does so.

## Coding Style

The code follows the coding style of the project and is well-formatted. Variable and function names are descriptive and self-explanatory.

## Logic

The logic of the changes seems correct. The function `checkIneligibleTableDDL` checks whether a table is ineligible before and after a DDL operation. If the table becomes ineligible after the operation, an error is returned. The unit tests cover all the cases and therefore the logic is
......
> Response is trunked for length limits.

<details>

In response to [this](https://github.com/pingcap/tiflow/pull/10965#issuecomment-2078822403):

>/review default


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>