Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-35127] remove HybridSource to avoid CI failure. #3237

Merged
merged 1 commit into from
Apr 19, 2024

Conversation

lvyanquan
Copy link
Contributor

At present, CI testing often causes OOM due to ValuesSource, which may be caused by the use of HybridSource. Considering that this will block other prs, remove it first to restore CI.

@lvyanquan
Copy link
Contributor Author

@PatrickRen @yuxiqian PTAL.

@lvyanquan
Copy link
Contributor Author

Looks like HybridSource has a requirement for parallelism, but some of our tests use the core number of the machine itself https://github.com/apache/flink/blob/a312a3bdd258e0ff7d6f94e979b32e2bc762b82f/flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/source/hybrid/HybridSourceITCase.java#L55

@lvyanquan
Copy link
Contributor Author

lvyanquan commented Apr 19, 2024

The failure in current CI contains a error message of

(4185ef2c5087300d32f25ea842d3ec59_cbc357ccb763df2852fee8c4fc7d55f2_0_0) switched from RUNNING to FAILED with failure cause:
java.lang.IllegalStateException: The "before" field of UPDATE/DELETE message is null, please check the Postgres table has been set REPLICA IDENTITY to FULL level. You can update the setting by running the command in Postgres 'ALTER TABLE inventory.products REPLICA IDENTITY FULL'. Please see more in Debezium documentation: https://debezium.io/documentation/reference/1.9/connectors/postgresql.html#postgresql-replica-identity
	at org.apache.flink.cdc.connectors.postgres.table.PostgresValueValidator.validate(PostgresValueValidator.java:44) ~[classes/:?]

Which is unrelated with this pr.

And CC @loserwang1024.

Copy link
Contributor

@PatrickRen PatrickRen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lvyanquan Thanks for the patch! LGTM as a temporary solution. Still we need to investigate the root cause of this one.

@PatrickRen PatrickRen merged commit d4ed7db into apache:master Apr 19, 2024
13 checks passed
wuzhenhua01 pushed a commit to wuzhenhua01/flink-cdc-connectors that referenced this pull request Aug 4, 2024
ChaomingZhangCN pushed a commit to ChaomingZhangCN/flink-cdc that referenced this pull request Jan 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants