drainer: fix kafka message limit problem #1039
Conversation
|
No release note, Please follow https://github.com/pingcap/community/blob/master/contributors/release-note-checker.md |
|
/rebuild |
|
/run-all-tests |
1 similar comment
|
/run-all-tests |
|
/run-all-tests |
|
/run-all-tests |
drainer/config.go
Outdated
| maxMsgSize = maxKafkaMsgSize | ||
|
|
||
| if cfg.SyncerCfg.To.KafkaMaxMessageSize > 0 { | ||
| maxMsgSize = cfg.SyncerCfg.To.KafkaMaxMessageSize |
There was a problem hiding this comment.
What if KafkaMaxMessageSize > MaxInt32 。 I think we need to do this check here
|
No release note, Please follow https://github.com/pingcap/community/blob/master/contributors/release-note-checker.md |
|
/rebuild |
|
/run-all-tests |
|
[REVIEW NOTIFICATION] This pull request has been approved by:
To complete the pull request process, please ask the reviewers in the list to review by filling The full list of commands accepted by this bot can be found here. Reviewer can indicate their review by submitting an approval review. |
|
/rebuild |
|
/run-check |
|
/run-all-tests |
|
In response to a cherrypick label: new pull request created: #1077. |
|
In response to a cherrypick label: new pull request created: #1078. |
glorv
added
needs-cherry-pick-release-5.1
needs-cherry-pick-release-5.2
|
In response to a cherrypick label: new pull request created: #1079. |
|
In response to a cherrypick label: new pull request created: #1080. |
|
close pingcap/tidb#28659 |
What problem does this PR solve?
close #1156
When we use kafka as downstream, the transaction size exceed 1G can't to transport to downstream.
What is changed and how it works?
max.Int32.max-message-sizein drainer's config.Check List
Tests
Code changes
Related changes
Release note
max.Int32. Add configurationmax-message-sizein drainer's config.