Rancher/fluentd missing mandatory "host" field

[chuegel]

We´re using Rancher with fluentd to export logs to Graylog.
Unfortunately this does not work: messages are discarded with following error:

2019-09-24T13:11:34.474Z WARN  [GelfCodec] GELF message <d54bf390-decc-11e9-94ba-00505688767a> (received from <xxx.xxxx.xxx.xxx:41938>) is missing mandatory "host" field.
2019-09-24T13:11:34.474Z ERROR [DecodingProcessor] Unable to decode raw message RawMessage{id=d54bf390-decc-11e9-94ba-00505688767a, journalOffset=5496823099, codec=gelf, payloadSize=84, timestamp=2019-09-24T13:11:34.473Z, remoteAddress=/xxx.xxx.xxx.xxx:41938} on input <5d89e01991c0380d006cfb3f>.
2019-09-24T13:11:34.474Z ERROR [DecodingProcessor] Error processing message RawMessage{id=d54bf390-decc-11e9-94ba-00505688767a, journalOffset=5496823099, codec=gelf, payloadSize=84, timestamp=2019-09-24T13:11:34.473Z, remoteAddress=/xxx.xxx.xxx.xxx:41938}
java.lang.IllegalArgumentException: GELF message <d54bf390-decc-11e9-94ba-00505688767a> (received from <10.2.28.40:41938>) is missing mandatory "short_message" or "message" field.
	at org.graylog2.inputs.codecs.GelfCodec.validateGELFMessage(GelfCodec.java:262) ~[graylog.jar:?]
	at org.graylog2.inputs.codecs.GelfCodec.decode(GelfCodec.java:134) ~[graylog.jar:?]
	at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:150) ~[graylog.jar:?]
	at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:91) [graylog.jar:?]
	at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:86) [graylog.jar:?]
	at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:45) [graylog.jar:?]
	at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
	at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222]

Expected Behavior

echo -n -e '{ "version": "1.1", "host": "example.org", "short_message": "A short message", "level": 5, "_some_info": "foo" }'"\0" | nc -w1 graylog.xxxx.com 12201

works as expected

Current Behavior

See above

Steps to Reproduce (for bugs)

1. install rancher
2. install graylog
3. create GELF TCP input (without TLS)
4. fill out the fluentd options in rancher
5. Hit Test

Context

Your Environment

• Graylog Version: 3.1
• Elasticsearch Version: 6.8.3
• MongoDB Version: 3.6.14
• Operating System: Centos 7
• Browser version: FF 69.0.1

[skyscooby]

@huegelc According to the GELF spec 'host' and short_message are required fields that must be sent in a GELF message. https://docs.graylog.org/en/3.1/pages/gelf.html#gelf-payload-specification

I suggest closing your issues in the Graylog2 as nothing is wrong on the server side.

You will need to add more details on how to reproduce to your rancher issue. Specifically how you accomplished #4 'fill out the fluentd options in rancher'..

This issue is most likely an issue in the fluend output configuration.

[chuegel]

Hi @skyscooby thanks for your reply. Yes, I also think this is more a rancher/fluentd issue as injecting a message via cli works properly. I´m going to close the issue here.