Skip to content

Commit

Permalink
Update consumer docs
Browse files Browse the repository at this point in the history
- Mention the importance of using Int.MaxValue or similar when using partitionedStream with flatMapPar, otherwise it will lead to a hung consumer that gets evicted.
  • Loading branch information
petern-sc committed May 18, 2024
1 parent e122b22 commit 0e5b88f
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion docs/consuming-kafka-topics-using-zio-streams.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ import zio._
import zio.kafka.consumer._
import zio.kafka.serde._

val data: Task[Chunk[CommittableRecord[String, String]]] =
val data: Task[Chunk[CommittableRecord[String, String]]] =
Consumer.plainStream(Subscription.topics("topic"), Serde.string, Serde.string).take(50).runCollect
.provideSomeLayer(consumer)
```
Expand Down Expand Up @@ -62,3 +62,5 @@ Consumer.partitionedStream(Subscription.topics("topic150"), Serde.string, Serde.
.mapZIO(_.commit)
.runDrain
```

When using partitionedStream with `flatMapPar(n)`, it is recommended to set n to `Int.MaxValue`. N must be equal or greater than the number of partitions your consumer subscribes to otherwise there'll be unhandled partitions and Kafka will eventually evict your consumer.

0 comments on commit 0e5b88f

Please sign in to comment.