Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No support for FetchRequest v10 which is required for ZStd on 2.2.1 brokers #2880

Closed
edenhill opened this issue May 11, 2020 · 0 comments
Closed

Comments

@edenhill
Copy link
Contributor

confluentinc/confluent-kafka-python#858

--

This is from the AK source:

 * - {@link Errors#UNSUPPORTED_COMPRESSION_TYPE} If a fetched topic is using a compression type which is
 *     not supported by the fetch request version

Which implies that compression.type=zstd is configured on the topic (broker-side config), but the client is using a FetchRequest version that is too old (<10).
The problem lies in librdkafka, it only implements FetchRequest versions 1, 2, 4 and 11, and broker 2.2.1 supports up to version 10, so librdkafka selects version 4 which does not support ZStd.

We'll fix this for the upcoming v1.5.0 release (mid june).
As a workaround I think you can set the broker's topic configuration compression.type=producer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant