Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lumberjack V2 output #27

Open
robin13 opened this issue May 25, 2018 · 5 comments
Open

Lumberjack V2 output #27

robin13 opened this issue May 25, 2018 · 5 comments

Comments

@robin13
Copy link

robin13 commented May 25, 2018

  • Version: 3.1.7

This plugin currently communicates only using Lumberjack V1 output, but this is not compatible with the Lumberjack V2 protocol. This can be problematic if a custom application (not logstash-input-beats which speaks both V1 and V2) has been designed to work with the Lumberjack V2 protocol only.
It would be great to have an option in the logstash-output-lumberjack plugin to choose the protocol version being used for communication.

@webmat
Copy link

webmat commented May 25, 2018

Note to future implementer, Lumberjack protocol V1 is explained here.

According to @ph, V2 differences are:

Just to make sure we don't lose reference to this, the protocol defined here contains accurate information about v1.
In short, V2 is replaced with V1 and the payload is in the form of key/value pair instead of using a JSON format.
The ACK in V1 is a bit different:
Do not support ACK0 keep alive.
Do no support partial ACK
Only ACK on full batch

@webmat
Copy link

webmat commented May 25, 2018

@ph @jordansissel Is there a more formally defined spec for Lumberjack V2? Or is the Go implementation in Beats the only reference?

@ntim
Copy link

ntim commented Jun 14, 2018

Lumberjack v2 also works better in combination with custom load balancers.

Furthermore, when configuring logstash-to-logstash communication as shown in the official docs, every minute or so, the sending logstash complains about a malformed frame.

@ntim
Copy link

ntim commented Jul 5, 2018

@ph @jsvd @jordansissel any updates?

@ntim
Copy link

ntim commented Jul 5, 2018

@webmat documentation of the protocol will not be done: https://github.com/elastic/libbeat/issues/279

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants