Fix processor failure in Filebeat when using regex, contain, or equals #2209
Conversation
When using any of those conditions with the `message` field in Filebeat a warning would occur and no processor would be applied. The warning message was:
WARN unexpected type *string in contains condition as it accepts only strings.
This occurred because Filebeat was passing the message field as a *string (string pointer). The processor code only expected to receive string values.
This PR contains three changes:
- Enhance the processor code to accept *string and string.
- Make filebeat pass the message field as a string rather than *string.
- Modify a test case to work against the message field rather than the source field.
Fixes elastic#2178
andrewkroh
added
bug
Filebeat
| @@ -125,8 +125,10 @@ func extractInt(unk interface{}) (uint64, error) { | |||
| func extractString(unk interface{}) (string, error) { | |||
| switch s := unk.(type) { | |||
| case string: | |||
| return string(s), nil | |||
| return s, nil | |||
| case *string: | |||
There was a problem hiding this comment.
@andrewkroh I think we need to tackle this problem in the event, and make sure that the event we are exporting doesn't contain any pointers. We shouldn't allow pointers in the condition of the processor.
There was a problem hiding this comment.
I fixed the code that was producing the event containing the string pointer too. I added this code to just make it work in case someone makes this mistake in a community Beat.
We shouldn't allow pointers in the condition of the processor.
How do we enforce this? Logging a warning isn't really strong enforcement. Ideally we could prevent a *string values from ever being written to the event, but this would require an actual event type with an API that's not a MapStr.
There was a problem hiding this comment.
I was expecting that ConvertToGenericEvent should fix that after marshalling & unmarshalling a pointer to string, becomes string: https://github.com/elastic/beats/blob/master/libbeat/common/event.go#L70. But I assume it doesn't do that.
Ideally, it would be nice if we can restrict somehow the type of the fields in the event, but for now an warning should be enough.
andrewkroh
removed
the
needs_backport
When using any of those conditions with the
messagefield in Filebeat a warning would occur and no processor would be applied. The warning message was:This occurred because Filebeat was passing the message field as a *string (string pointer). The processor code only expected to receive string values.
This PR contains three changes:
Fixes #2178