syslog input won't bind to IPv6 UDP #42

lingfish · 2017-12-03T02:42:19Z

Version: logstash 6.0.0, logstash-input-syslog-3.2.2
Operating System: Debian 9.2
Java version: openjdk version "1.8.0_151"
Config File (if you have sensitive info, please remove it):

input {
    syslog {
        port => 9999
    }
}
filter { }
output {
    stdout { codec => rubydebug }
}

Steps to Reproduce: run as /usr/share/logstash/bin/logstash -f /etc/logstash/testing/syslog.conf --path.data /tmp/blah --path.settings /etc/logstash -l /tmp/log --verbose

Running it this was results in some strange bindings:

COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
java    7383 root   74u  IPv6 151679      0t0  TCP *:9999 (LISTEN)
java    7383 root   78u  IPv4 146388      0t0  UDP *:9999

As you can see, no UDP listeners on IPv6. I've tried various combinations of specifying host, and the Java options like -Djava.net.preferIPv6Stack=true.

Logs are clean with the default wildcard address:

[2017-12-03T13:12:17,742][INFO ][logstash.inputs.syslog   ] Starting syslog tcp listener {:address=>"0.0.0.0:9999"}
[2017-12-03T13:12:17,744][INFO ][logstash.inputs.syslog   ] Starting syslog udp listener {:address=>"0.0.0.0:9999"}

Yet the same bindings as above occur.

Specifically, when I force any form of IPv6 address, I then see:

[2017-12-03T12:00:03,099][INFO ][logstash.inputs.syslog   ] Starting syslog tcp listener {:address=>":::9999"}
[2017-12-03T12:00:03,099][INFO ][logstash.inputs.syslog   ] Starting syslog udp listener {:address=>":::9999"}
[2017-12-03T12:00:03,100][INFO ][logstash.agent           ] Pipelines running {:count=>1, :pipelines=>["main"]}
[2017-12-03T12:00:03,105][WARN ][logstash.inputs.syslog   ] syslog listener died {:protocol=>:udp, :address=>":::9999", :exception=>java.nio.channels.UnsupportedAddressTypeException, :backtrace=>["sun.nio.ch.DatagramChannelImpl.bind(DatagramChannelImpl.java:684)", "org.jruby.ext.socket.RubyUDPSocket.bind(RubyUDPSocket.java:179)", "org.jruby.ext.socket.RubyUDPSocket$INVOKER$i$2$0$bind.call(RubyUDPSocket$INVOKER$i$2$0$bind.gen)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:358)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:195)", "org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:323)", "org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)", "org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:83)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:179)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:165)", "org.jruby.RubyClass.finvoke(RubyClass.java:531)", "org.jruby.RubyBasicObject.send19(RubyBasicObject.java:1670)", "org.jruby.RubyKernel.send19(RubyKernel.java:1986)", "org.jruby.RubyKernel$INVOKER$s$send19.call(RubyKernel$INVOKER$s$send19.gen)", "org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:208)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:358)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:195)", "org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:323)", "org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)", "org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:89)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:214)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:200)", "org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:208)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:358)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:195)", "org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:323)", "org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)", "org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:132)", "org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:148)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:73)", "org.jruby.runtime.Block.call(Block.java:124)", "org.jruby.RubyProc.call(RubyProc.java:289)", "org.jruby.RubyProc.call(RubyProc.java:246)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:104)", "java.lang.Thread.run(Thread.java:748)"]}

This loops. I think the interesting part there is :exception=>java.nio.channels.UnsupportedAddressTypeException

The text was updated successfully, but these errors were encountered:

stefan-as · 2018-02-08T13:48:59Z

Seems that someone likes the idea to be more specific than expected and dropped IPv6 support:

logstash-input-syslog/lib/logstash/inputs/syslog.rb

Line 148 in c75275d

@udp = UDPSocket.new(Socket::AF_INET)

No need to specify address family here?

abraxxa · 2018-02-26T12:23:27Z

We would need IPv6 support to the UDP receiver too.

davama · 2018-03-20T14:53:39Z

Is this a more wide spread issue??

logstash-plugins/logstash-input-udp#31
path-network/logstash-codec-sflow#10 << i posted that one
I also have issues with netflow-codec but there is no ticket for that, dont think cisco fully supports netflow with ipv6

headius · 2018-03-26T20:10:46Z

This appears to be the same issue as jruby/jruby#5112. CRuby also raises an exception if you do not explicitly specify AF_INET6 in the UDPSocket constructor, but we raise the wrong exception (we allow a Java exception to bubble out). I'm looking into a remedy.

headius · 2018-03-26T20:15:13Z

Let me put that differently: the error that's going into logs is related to jruby/jruby#5112. The actual bug/feature here for logstash is outside my expertise :-)

headius · 2018-03-27T01:39:45Z

I have "fixed" jruby/jruby#5112 and the resulting error now matches CRuby (at least type...message is different).

davama · 2018-04-24T18:01:20Z

Is this fixed with logstash 6.2.4?

for me it did solve it but im not using this particular input...
https://discuss.elastic.co/t/logstash-input-fails-listening-ipv6/122114

i had simply commented here because more than one plugin were being affected so thought it was more wide spread.

thanks!
dave

jamesog · 2019-01-06T14:24:22Z

I'm running 6.4.2 and it's still broken.

Although I've just discovered I can use the generic udp input which does do IPv6 correctly. :-)

boeningc · 2019-01-12T22:31:31Z

6.5.4 seems to still be broken as well.

Sandeep-VD · 2019-06-18T19:45:30Z

Hi,

Checked in 6.8. Still seems like issue exists. Can someone help please. We are planning to use 6.5.4, and hitting the issue. Basically syslog input starts up on IPv6 tcp port and IPv4 UDP port. It does not start on IPv6 UDP port and IPv4 TCP port.
Input looks like this:
input {
tcp {
port => 9514
type => syslog
codec => plain {
charset => "ISO-8859-1"
}
}
udp {
port => 9514
type => syslog
codec => plain {
charset => "ISO-8859-1"
}
}
}

netstat shows :
[logstash]$ sudo netstat -anp | grep 9514
tcp6 0 0 :::9514 :::* LISTEN 28419/java
udp 0 0 0.0.0.0:9514 0.0.0.0:* 28419/java

How can we get over the issue.?

antondollmaier · 2019-07-06T11:50:22Z

Issue present in 7.2.0 as well.

Specifying host => '::' explicitly fails:

[INFO ] 2019-07-06 11:50:05.884 [Ruby-0-Thread-13: /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:309] syslog - Starting syslog udp listener {:address=>":::5000"}
[WARN ] 2019-07-06 11:50:05.885 [Ruby-0-Thread-13: /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:309] syslog - syslog listener died {:protocol=>:udp, :address=>":::5000", :exception=>#<SocketError: bind: unsupported address "::" for protocol family INET>, :backtrace=>["org/jruby/ext/socket/RubyUDPSocket.java:197:in `bind'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-syslog-3.4.1/lib/logstash/inputs/syslog.rb:149:in `udp_listener'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-syslog-3.4.1/lib/logstash/inputs/syslog.rb:130:in `server'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-syslog-3.4.1/lib/logstash/inputs/syslog.rb:110:in `block in run'"]}

hangxie · 2019-09-09T16:59:51Z

#55 shows a quick and dirty fix, I don't expect it can be accepted as there should be an elegant way to determine IP address family, also I guess people need support to listen on both IPv4 and IPv6 though it's not case for me.

* Disabling IPv6 until the logstash udp-based syslog listener can use it: logstash-plugins/logstash-input-syslog#42 * Fixing missing logstash.service systemd unit file elastic/logstash#11309 fixes: #87

axxelG · 2021-11-17T14:52:05Z

Im a bit confused that this issue is still open. Wasn't it fixed with #56 ?

maheshritgithub · 2024-10-03T07:36:43Z

It is not working in the latest version of Logstash 8. Is it still open?

davama mentioned this issue Apr 20, 2018

Add Ipv6 Listeners robcowart/elastiflow#51

Closed

sadsfae mentioned this issue Nov 15, 2019

Disable IPv6 until it works with logstash/syslog UDP sadsfae/ansible-elk#87

Closed

sadsfae mentioned this issue Nov 15, 2019

Disable IPv6 until Logstash/UDP syslog can use it. sadsfae/ansible-elk#88

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

syslog input won't bind to IPv6 UDP #42

syslog input won't bind to IPv6 UDP #42

lingfish commented Dec 3, 2017 •

edited

Loading

stefan-as commented Feb 8, 2018

abraxxa commented Feb 26, 2018

davama commented Mar 20, 2018

headius commented Mar 26, 2018

headius commented Mar 26, 2018

headius commented Mar 27, 2018

davama commented Apr 24, 2018

jamesog commented Jan 6, 2019 •

edited

Loading

boeningc commented Jan 12, 2019

Sandeep-VD commented Jun 18, 2019

antondollmaier commented Jul 6, 2019

hangxie commented Sep 9, 2019

axxelG commented Nov 17, 2021

maheshritgithub commented Oct 3, 2024 •

edited

Loading

syslog input won't bind to IPv6 UDP #42

syslog input won't bind to IPv6 UDP #42

Comments

lingfish commented Dec 3, 2017 • edited Loading

stefan-as commented Feb 8, 2018

abraxxa commented Feb 26, 2018

davama commented Mar 20, 2018

headius commented Mar 26, 2018

headius commented Mar 26, 2018

headius commented Mar 27, 2018

davama commented Apr 24, 2018

jamesog commented Jan 6, 2019 • edited Loading

boeningc commented Jan 12, 2019

Sandeep-VD commented Jun 18, 2019

antondollmaier commented Jul 6, 2019

hangxie commented Sep 9, 2019

axxelG commented Nov 17, 2021

maheshritgithub commented Oct 3, 2024 • edited Loading

lingfish commented Dec 3, 2017 •

edited

Loading

jamesog commented Jan 6, 2019 •

edited

Loading

maheshritgithub commented Oct 3, 2024 •

edited

Loading