TiCDC OOM（Kafka Sink ）

[Bowen-Tang]

What did you do?

The operations on the application side is not yet identified. The default transaction size is 100MB, and the downstream is Kafka. Meanwhile, the changefeed is configured to synchronize single table.

The attachment contains golang heap and table schema.
cdcoom.zip

What did you expect to see?

No response

What did you see instead?

No matter the VM is configured with 32GB or 64 GB memory, CDC drains the memory entirely.

Versions of the cluster

V5.3.1

[zhaoxinyu]

This issue is under investigation.

[CharlesCheung96]

@Bowen-Tang Can you provide the monitoring data and detailed version information of cdc?

[Bowen-Tang]

version: 5.3.1
@CharlesCheung96 Sorry，only this monitoring data：

[CharlesCheung96]

@Bowen-Tang According to Puller's monitoring, the workload of this CDC node is relatively abnormal, which is far beyond the throughput capacity of the downstream under normal circumstances. Therefore, we highly suspect that inconsistent upstream and downstream throughput will cause data accumulation in memory, which will eventually lead to OOM.

This pr alleviates the OOM problem caused by upstream and downstream throughput to a certain extent, so we temporarily closed the issue. If similar problems occur again, feel free to reopen it.