From 7c09581b55c02f65d0a9fc7a6788a284c24fd777 Mon Sep 17 00:00:00 2001 From: Erik van Oosten Date: Thu, 16 Nov 2023 19:43:23 +0100 Subject: [PATCH 1/2] Extend tuning docs --- docs/consumer-tuning.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/consumer-tuning.md b/docs/consumer-tuning.md index 65460e14b..bfecd3d8d 100644 --- a/docs/consumer-tuning.md +++ b/docs/consumer-tuning.md @@ -41,8 +41,12 @@ val highThroughputSettings = ConsumerSettings(bootstrapServers).tuneForHighThrou val lowLatencySettings = ConsumerSettings(bootstrapServers).tuneForLowLatency ``` -Kafka’s performance is not very sensitive to record size. However, when records become very small (< 100 bytes) or very -large (> 100Kb), increasing or decreasing `max.poll.records` and `partitionPreFetchBufferLimit` can be considered. +## Small and large records + +Kafka’s performance is not very sensitive to record size. However, when records become very small (< 100 bytes) it +might be beneficial to increase `max.poll.records` and `partitionPreFetchBufferLimit`. Similarly, when records are +very large (> 100Kb), `max.poll.records` can be decreased and pre-fetching can be limited by decreasing +`partitionPreFetchBufferLimit` or even disabled by using `ConsumerSettngs.withoutPartitionPreFetching`. ## High number of partitions From e06377525ff97942ff329466ba466e569fa72467 Mon Sep 17 00:00:00 2001 From: Erik van Oosten Date: Thu, 16 Nov 2023 19:45:30 +0100 Subject: [PATCH 2/2] Extend tuning docs (2) --- docs/consumer-tuning.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/consumer-tuning.md b/docs/consumer-tuning.md index bfecd3d8d..3e34c521d 100644 --- a/docs/consumer-tuning.md +++ b/docs/consumer-tuning.md @@ -45,7 +45,7 @@ val lowLatencySettings = ConsumerSettings(bootstrapServers).tuneForLowLatency Kafka’s performance is not very sensitive to record size. However, when records become very small (< 100 bytes) it might be beneficial to increase `max.poll.records` and `partitionPreFetchBufferLimit`. Similarly, when records are -very large (> 100Kb), `max.poll.records` can be decreased and pre-fetching can be limited by decreasing +very large (> 100Kb), `max.poll.records` can be decreased. Also, pre-fetching can be limited by decreasing `partitionPreFetchBufferLimit` or even disabled by using `ConsumerSettngs.withoutPartitionPreFetching`. ## High number of partitions