Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-35884][mysql-pipeline-connector] MySQL pipeline support snapshot chunk key-column #3490

Merged
merged 9 commits into from
Aug 12, 2024

Conversation

beryllw
Copy link
Contributor

@beryllw beryllw commented Jul 23, 2024

Flink-connector-mysql-cdc MySqlSourceOptions supports specifying the parameter scan.incremental.snapshot.chunk.key-column to divide chunks, pipeline connector should also support.

@beryllw beryllw changed the title [FLINK-35884] MySQL pipeline support snapshot chunk key-column [FLINK-35884][mysql-pipeline-connector] MySQL pipeline support snapshot chunk key-column Jul 23, 2024
@leonardBang leonardBang requested a review from ruanhang1993 July 29, 2024 03:58
@beryllw
Copy link
Contributor Author

beryllw commented Jul 31, 2024

@ruanhang1993 PTAL

@beryllw beryllw requested a review from ruanhang1993 August 1, 2024 09:47
@beryllw
Copy link
Contributor Author

beryllw commented Aug 5, 2024

@ruanhang1993 PTAL, thanks.

Copy link
Contributor

@ruanhang1993 ruanhang1993 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@beryllw
Copy link
Contributor Author

beryllw commented Aug 5, 2024

compile_and_test
may be related to #3508

Error:  Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 122.202 s <<< FAILURE! - in org.apache.flink.cdc.connectors.starrocks.sink.StarRocksMetadataApplierITCase
Error:  org.apache.flink.cdc.connectors.starrocks.sink.StarRocksMetadataApplierITCase  Time elapsed: 122.202 s  <<< ERROR!
java.util.concurrent.CompletionException: org.testcontainers.containers.ContainerLaunchException: Container startup failed for image starrocks/allin1-ubuntu:3.2.6

@beryllw
Copy link
Contributor Author

beryllw commented Aug 6, 2024

@leonardBang PTAL, thanks.

Copy link
Contributor

@leonardBang leonardBang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @beryllw for the contribution and @ruanhang1993 for the review, LGTM, wait the CI green

@beryllw beryllw requested a review from leonardBang August 12, 2024 01:33
@beryllw
Copy link
Contributor Author

beryllw commented Aug 12, 2024

2024-08-12T03:56:34.3999892Z [WARNING] Tests run: 20, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 624.39 s - in org.apache.flink.cdc.connectors.oracle.table.OracleConnectorITCase
2024-08-12T03:56:34.4428837Z [INFO] 
2024-08-12T03:56:34.4429524Z [INFO] Results:
2024-08-12T03:56:34.4430148Z [INFO] 
2024-08-12T03:56:34.4432410Z [ERROR] Failures: 
2024-08-12T03:56:34.4436613Z [ERROR]   NewlyAddedTableITCase.testJobManagerFailoverForNewlyAddedTable:229->testNewlyAddedTableOneByOne:585->testNewlyAddedTableOneByOne:725->OracleSourceTestBase.assertEqualsInAnyOrder:111->OracleSourceTestBase.assertEqualsInOrder:119 arrays first differed at element [0]; expected:<...16874195632735147, C[HINA], BEIJING, BEIJING W...> but was:<...16874195632735147, C[hina], BEIJING, BEIJING W...>
2024-08-12T03:56:34.4440159Z [INFO] 
2024-08-12T03:56:34.4440793Z [ERROR] Tests run: 56, Failures: 1, Errors: 0, Skipped: 0

@leonardBang It seems that the failure of CI has nothing to do with this pr. Could we retrigger CI?Thanks.

@ruanhang1993 ruanhang1993 merged commit 15ab858 into apache:master Aug 12, 2024
20 checks passed
qiaozongmi pushed a commit to qiaozongmi/flink-cdc that referenced this pull request Sep 23, 2024
…ports to set chunk key-column (apache#3490)


Co-authored-by: wangjunbo <wangjunbo@qiyi.com>
Co-authored-by: Hang Ruan <ruanhang1993@hotmail.com>
ChaomingZhangCN pushed a commit to ChaomingZhangCN/flink-cdc that referenced this pull request Jan 13, 2025
…ports to set chunk key-column (apache#3490)


Co-authored-by: wangjunbo <wangjunbo@qiyi.com>
Co-authored-by: Hang Ruan <ruanhang1993@hotmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants