Skip to content

v1.4.0
Compare
Choose a tag to compare
@edenhill edenhill released this 02 Apr 20:05
· 969 commits to master since this release
v1.4.0
e4a8c0f

librdkafka v1.4.0

v1.4.0 is a feature release:

Transactional Producer API

librdkafka now has complete Exactly-Once-Semantics (EOS) functionality, supporting the idempotent producer (since v1.0.0), a transaction-aware consumer (since v1.2.0) and full producer transaction support (in this release).
This enables developers to create Exactly-Once applications with Apache Kafka.

See the Transactions in Apache Kafka page for an introduction and check the librdkafka transactions example for a complete transactional application example.

Security fixes

Two security issues have been identified in the SASL SCRAM protocol handler:

  • The client nonce, which is expected to be a random string, was a static string.
  • If sasl.username and sasl.password contained characters that needed escaping, a buffer overflow and heap corruption would occur. This was protected, but too late, by an assertion.

Both of these issues are fixed in this release.

Enhancements

  • Add FNV-1a partitioner (by @Manicben, #2724).
    The new fnv1a_random partitioner is compatible with Sarama's NewHashPartitioner partition, easing transition from Sarama to librdkafka-based clients such as confluent-kafka-go.
  • Added rd_kafka_error_t / RdKafka::Error complex error type which provides error attributes such as indicating if an error is retriable.
  • The builtin mock broker now supports balanced consumer groups.
  • Support finding headers in nonstandard directories in CMake (@benesch)
  • Improved static library bundles which can now contain most dependencies.
  • Documentation, licenses, etc, is now installed by make install
  • Bump OpenSSL to v1.0.2u (when auto-building dependencies)

Fixes

General:

  • Correct statistics names in docs (@TimWSpence, #2754)
  • Wake up broker thread based on next request retry.
    Prior to this fix the next wakeup could be delayed up to 1 second regardless of next retry.
  • Treat SSL peer resets as usual Disconnects, making log.connection.close work
  • Reset buffer corrid on connection close to honour ApiVers and Sasl request priorities (@xzxxzx401, #2666)
  • Cleanup conf object if failing to creat producer or consumer (@fboranek)
  • Fix build of rdkafka_example project for windows, when using building it using Visual Studio 2017/2019 (by @Eliyahu-Machluf)
  • Minor fix to rdkafka_example usage: add lz4 and zstd compression codec to usage (by @Eliyahu-Machluf)
  • Let broker nodename updates propagate as ERR__TRANSPORT rather than ERR__NODE_UPDATE to avoid an extra error code for the application to handle.
  • Fix erroneous refcount assert in enq_once_del_source (e.g., on admin operation timeout)
  • Producers could get stuck in INIT state after a disconnect until a to-be-retried request timed out or the connection was needed for other purposes (metadata discovery, etc), this is now fixed.

Producer:

  • flush() now works with RD_KAFKA_EVENT_DR
  • Fix race condition when finding EOS-supporting broker

Consumer:

  • Consumers could get stuck after rebalance if assignment was empty
  • Enforce session.timeout.ms in the consumer itself (#2631)
  • max.poll.interval.ms is now only enforced when using subscribe()
  • Fix consumer_lag calculation for transactional topics
  • Show fetch/no-fetch reason in topic debugging
  • Properly propagate commit errors per partition
  • Don't send heartbeats after max.poll.interval.ms is exceeded.
  • Honour array size in rd_kafka_event_message_array() to avoid overflow (#2773)

Checksums

Release asset checksums:

  • v1.4.0.zip SHA256 eaf954e3b8a2ed98360b2c76f55048ee911964de8aefd8a9e1133418ec9f48dd
  • v1.4.0.tar.gz SHA256 ae27ea3f3d0d32d29004e7f709efbba2666c5383a107cc45b3a1949486b2eb84
Assets 2
Loading