KAFKA-4514: Add Codec for ZStandard Compression#2267

Merged

hachikuji merged 22 commits intoapache:trunkfrom dongjinleekr:feature/KAFKA-4514

Oct 10, 2018

+610-86

Commits on Oct 9, 2018

KAFKA-4514: Add Codec for ZStandard Compression

dongjinleekr
authored and
Jason Gustafson
committed
Minor updates

dongjinleekr
authored and
Jason Gustafson
committed
Update ZStd version: 1.3.3 -> 1.3.4 (zstd-jni 1.3.4-10)

dongjinleekr
authored and
Jason Gustafson
committed
Update ZStd version: 1.3.4 -> 1.3.5 (zstd-jni 1.3.5-2)

dongjinleekr
authored and
Jason Gustafson
committed
Add license description of zstd and zstd-jni

dongjinleekr
authored and
Jason Gustafson
committed
Disallow fetch request with API version < 10 for ZStandard compressed partition

dongjinleekr
authored and
Jason Gustafson
committed
Disallow produce request with API version < 7 for ZStandard compressed partition

dongjinleekr
authored and
Jason Gustafson
committed
Disallow MemoryRecordsBuilder instantiation with ZSTD compression codec if magic is not v2

dongjinleekr
authored and
Jason Gustafson
committed
Disallow downconversion of ZSTD compressed records

dongjinleekr
authored and
Jason Gustafson
committed
Make the constructor of ProduceRequest.Builder public: 1. parity with FetchRequest.Builder. 2. Required for testing produce request with any version.

dongjinleekr
authored and
Jason Gustafson
committed
Add integration tests: FetchRequestTest, ProduceRequestTest

dongjinleekr
authored and
Jason Gustafson
committed
Update zstd-jni version: 1.3.5-2 -> 1.3.5-4

dongjinleekr
authored and
Jason Gustafson
committed
Fix: remove unneeded newline addition at CompressionType.java

dongjinleekr
authored and
Jason Gustafson
committed
Revert "Disallow downconversion of ZSTD compressed records"

dongjinleekr
authored and
Jason Gustafson
committed
Reimplement down-conversion logic

dongjinleekr
authored and
Jason Gustafson
committed
Update for the previous commit (Thanks to Jason Gustafson's guide)

dongjinleekr
authored and
Jason Gustafson
committed
Add UNSUPPORTED_COMPRESSION_TYPE to the possible error codes list of FetchResponse

dongjinleekr
authored and
Jason Gustafson
committed
Refactor tests: MemoryRecordsTest, MemoryRecordsBuilderTest

dongjinleekr
authored and
Jason Gustafson
committed
Remove ProduceRequest version check in KafkaApis#handleProduceRequest method

dongjinleekr
authored and
Jason Gustafson
committed
Add validation for the case magic < 2 with Zstandard Compression

dongjinleekr
authored and
Jason Gustafson
committed
Consistency: all log or exception messages are using 'ZStandard', not 'Zstd'

dongjinleekr
authored and
Jason Gustafson
committed
Add validation for ProduceRequest: update ProduceRequest.validateRecords

dongjinleekr
authored and
Jason Gustafson
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

KAFKA-4514: Add Codec for ZStandard Compression#2267

Commits on Oct 9, 2018

KAFKA-4514: Add Codec for ZStandard Compression

Minor updates

Update ZStd version: 1.3.3 -> 1.3.4 (zstd-jni 1.3.4-10)

Update ZStd version: 1.3.4 -> 1.3.5 (zstd-jni 1.3.5-2)

Add license description of zstd and zstd-jni

Disallow fetch request with API version < 10 for ZStandard compressed partition

Disallow produce request with API version < 7 for ZStandard compressed partition

Disallow MemoryRecordsBuilder instantiation with ZSTD compression codec if magic is not v2

Disallow downconversion of ZSTD compressed records

Make the constructor of ProduceRequest.Builder public: 1. parity with FetchRequest.Builder. 2. Required for testing produce request with any version.

Add integration tests: FetchRequestTest, ProduceRequestTest

Update zstd-jni version: 1.3.5-2 -> 1.3.5-4

Fix: remove unneeded newline addition at CompressionType.java

Revert "Disallow downconversion of ZSTD compressed records"

Reimplement down-conversion logic

Update for the previous commit (Thanks to Jason Gustafson's guide)

Add UNSUPPORTED_COMPRESSION_TYPE to the possible error codes list of FetchResponse

Refactor tests: MemoryRecordsTest, MemoryRecordsBuilderTest

Remove ProduceRequest version check in KafkaApis#handleProduceRequest method

Add validation for the case magic < 2 with Zstandard Compression

Consistency: all log or exception messages are using 'ZStandard', not 'Zstd'

Add validation for ProduceRequest: update ProduceRequest.validateRecords