producev without varargs

[andrewthad]

This is not an issue so much as a request for an additional function in the API. Currently, the most general and flexible way to create a single message with headers is rd_kafka_producev. This is, for example, the only way to create a message with headers. However, this function uses varargs, which makes it difficult to bind to in most high-level languages. The varargs are used to build a rd_kafka_msg_t, which is then fed into rd_kafka_msg_partitioner. It would make it easier to interface with librdkafka from high-level languages if the user could construct a rd_kafka_msg_t directly and then feed it into some produce* variant. I'm not terribly well versed in C library conventions, but I assume that this was not done because adding any field to rd_kafka_msg_t would be a breaking change in the API. I'm not sure if there is a way around that issue or not.

[edenhill]

We could make a produce variant that takes an array of RD_KAFKA_V_..s.
E.g.:

typedef union {
  rd_kafka_topic_t *rkt;
  const char *const_str;
  char *str;
  int i;
  int32 i32;
  int64 i64;
 ...
} rd_kafka_v_t ;

rd_kafka_error_t *rd_kafka_produceva(rd_kafka_t *rk, rd_kafka_v_t *vs, size_t v_cnt);

What do you think?

[andrewthad]

Since C unions are not tagged, I think that you would need to tag them explicitly with something like:

typedef union { ... } rd_kafka_v_t; // As in your example
enum rd_field {topic, flags, opaque, ...};
typedef union {
   rd_field field;
   rd_kafka_v_t val;
} rd_kafka_vp_t;
rd_kafka_error_t *rd_kafka_produceva(rd_kafka_t *rk, rd_kafka_vp_t *vs, size_t v_cnt);

But aside from that quibble, the union-style approach would work fine for my case.

[edenhill]

Please review this PR: #2902