Cooperative Incremental sticky rebalance

[SreeniIO]

Fixes: #363

[SreeniIO]

This function can be avoid if there is a way to read the config partition.assignment.strategy value

[benesch]

Thanks. I figured out how to do this automatically.

[benesch]

@SreeniIO I pushed a new implementation that auto-detects whether cooperative rebalancing is in use. Could you give this a spin and let me know if it works for your use case?

[SreeniIO]

@SreeniIO I pushed a new implementation that auto-detects whether cooperative rebalancing is in use. Could you give this a spin and let me know if it works for your use case?

Yes, it works fine.

For Cooperative rebalancing, since not all partitions are revoked during a rebalance, can you add the TopicPartitionList being revoked as part of the Rebalance::Revoke enum?

pub enum Rebalance<'a> {
    /// A new partition assignment is received.
    Assign(&'a TopicPartitionList),
    /// All partitions are revoked.
    Revoke, // can this be changed to Revoke(&'a TopicPartitionList),
    /// Unexpected error from Kafka.
    Error(String),
}

[oronsh]

any news about this one? :)

[benesch]

For Cooperative rebalancing, since not all partitions are revoked during a rebalance, can you add the TopicPartitionList being revoked as part of the Rebalance::Revoke enum?

pub enum Rebalance<'a> {
    /// A new partition assignment is received.
    Assign(&'a TopicPartitionList),
    /// All partitions are revoked.
    Revoke, // can this be changed to Revoke(&'a TopicPartitionList),
    /// Unexpected error from Kafka.
    Error(String),
}

Ack. I filed #398 for this.