Consumer: make partition assignment handler public API #949
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
seglo
reviewed
Oct 23, 2019
| /** | ||
| * See [[org.apache.kafka.clients.consumer.KafkaConsumer#seekToEnd(java.util.Collection[TopicPartition])]] | ||
| */ | ||
| def seekToEnd(tps: java.util.Collection[TopicPartition]): Unit = consumer.seekToEnd(tps) | ||
| } |
There was a problem hiding this comment.
I suggest we also add KafkaConsumer.offsetsForTimes while we're updating the RestrictedConsumer. It's a blocking call. It's useful for apps running in multi-dc Kafka environments.
| /** | ||
| * The API is new and may change in further releases. | ||
| * | ||
| * Allows to execute user code when Kafka rebalances partitions between consumers, or an Alpakka Kafka consumer is stopped. |
There was a problem hiding this comment.
Suggested change
| * Allows to execute user code when Kafka rebalances partitions between consumers, or an Alpakka Kafka consumer is stopped. | |
| * Allows the user to execute user code when Kafka rebalances partitions between consumers, or an Alpakka Kafka consumer is stopped. |
| * A warning will be logged if a callback takes longer than the configured `partition-handler-warning`. | ||
| * | ||
| * There is no point in calling `CommittableOffset`'s commit methods as their committing won't be executed as long as any of | ||
| * the callbacks in this class are called. |
There was a problem hiding this comment.
Could mention that RestrictedConsumer.commitSync is available though.
| * the callbacks in this class are called. | ||
| * | ||
| * This complements the methods of Kafka's [[org.apache.kafka.clients.consumer.ConsumerRebalanceListener ConsumerRebalanceListener]] with | ||
| * an `onStop` callback. |
There was a problem hiding this comment.
To match the scaladsl comments.
Suggested change
| * an `onStop` callback. | |
| * an `onStop` callback which is called before `Consumer.close`. |
| */ | ||
| @ApiMayChange | ||
| trait PartitionAssignmentHandler { | ||
|
|
||
| /** | ||
| * See [[org.apache.kafka.clients.consumer.ConsumerRebalanceListener#onPartitionsRevoked]] | ||
| * | ||
| * @param revokedTps The list of partitions that were assigned to the consumer on the last rebalance | ||
| * @param revokedTps The list of partitions that were revoked from the consumer | ||
| * @param consumer A restricted version of the internally used [[org.apache.kafka.clients.consumer.Consumer Consumer]] |
There was a problem hiding this comment.
It's not really a KafkaConsumer though in the is a sense. Should we link to RestrictedConsumer instead, or in addition, to this?
seglo
approved these changes
Oct 30, 2019
| @@ -0,0 +1,8 @@ | |||
| # Internal API | |||
There was a problem hiding this comment.
I like the PR-based path convention. I'll do that in other PRs too (#930)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
Allow users to pass a
PartitionAssignmentHandlerwhich allows for hooking into Kafka rebalances.Potential use cases may include
References
Fixes #539, #843, #911
#761
Changes
PartitionAssignmentHandlerjavadsl.PartitionAssignmentHandlerseekToBeginningandseekToEnd(usful for User-defined seeking for subscriptions #911)TODO