Add an option to support plain TCP

[ph]

The current master of ruby-lumberjack include an option to disable SSL for the socket connection.
We have to expose this feature to the input, I suggest we use the same naming that we use in the elasticsearch output (https://github.com/logstash-plugins/logstash-output-elasticsearch/blob/master/lib/logstash/outputs/elasticsearch.rb#L161), keep in mind that it will be turn on by default and people need to explicitly turn it off.

 # Enable SSL
config :ssl, :validate => :boolean, :default => true

Ref: elastic/ruby-lumberjack#14 elastic/ruby-lumberjack#12

[ralphm]

@ph: I was looking at using the Lumberjack protocol over spiped. Do you agree the output plugin should have a similar configuration for symmetry?

[ph]

@ralphm +1 for symmetry, and it's just a matter of exposing the underlying configuration options.
If you want to go ahead I will gladly review your PR, I think the feature is still in master and not part of a released gems. I've created logstash-plugins/logstash-output-lumberjack#14 to track it.

[ph]

@suyograo Do we still want to add this to the lumberjack input since LSF wont support plain text connection and we have created logstash-input-beats which come with ssl turn off by default?

[m4ce]

+1

[torrancew]

@ph as a user, my gut would say to keep behavior here as-is, and focus on making sure we have a clean migration path for users moving from LSF to FB, in preparation for full-on removal of this input at some future date. Any thoughts?

[suyograo]

Yeah, i would keep the current behavior as is and put our efforts on input-beats. Filebeats is already being deployed as a replacement for LSF, so there is really no need for this feature. I'd close this issue if you agree

[torrancew]

@suyograo I agree. Though it's worth noting, there was a report in IRC today of difficulties using LSF with 2.x. While the user is planning on adopting FB, they are also trying to modernize their LS install. Is there a recommended upgrade path in terms of versions, etc, for a user to move from LSF to FB incrementally?

[ralphm]

I am confused, maybe by the many acronyms. I currently use this to receive events shipped by a Logstash instance in another data center like so:

Kafka -> Logstash -------------> Logstash -> Kafka

The receiving instance does some additional filtering and selects the destination Kafka topic based on data in the event.

How do beats come into play here? Does it use the same protocol?

[torrancew]

@ralphm My understanding is that that use case is satisfied by moving the output/input combination from lumberjack/lumberjack to beats/beats.

Again, my understanding here is that beats is an advancement of the custom protocol that this plugin piloted, but I welcome any corrections from the team.

[ph]

@torrancew You are correct, Its an evolution of the protocol of LSF and also an updated light agent.

[ph]

Since we are moving our effort to inputs beats I will be closing this issue.