Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix processor failure in Filebeat when using regex, contain, or equals #2209

Merged
merged 1 commit into from
Aug 10, 2016
Merged

Fix processor failure in Filebeat when using regex, contain, or equals #2209

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ https://github.com/elastic/beats/compare/v5.0.0-alpha5...master[Check the HEAD d
*Topbeat*

*Filebeat*
- Fix processor failure in Filebeat when using regex, contain, or equals with the message field. {issue}2178[2178]

*Winlogbeat*

Expand Down
4 changes: 2 additions & 2 deletions filebeat/input/event.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ func (f *Event) ToMapStr() common.MapStr {

if f.JSONConfig != nil && len(f.JSONFields) > 0 {
mergeJSONFields(f, event)
} else {
event["message"] = f.Text
} else if f.Text != nil {
event["message"] = *f.Text
}

return event
Expand Down
2 changes: 1 addition & 1 deletion filebeat/tests/system/test_processors.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ def test_condition(self):
path=os.path.abspath(self.working_dir) + "/test*.log",
processors=[{
"drop_event": {
"when": "not.contains.source: test",
"when": "not.contains.message: test",
},
}]
)
Expand Down
6 changes: 4 additions & 2 deletions libbeat/processors/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,8 +125,10 @@ func extractInt(unk interface{}) (uint64, error) {
func extractString(unk interface{}) (string, error) {
switch s := unk.(type) {
case string:
return string(s), nil
return s, nil
case *string:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andrewkroh I think we need to tackle this problem in the event, and make sure that the event we are exporting doesn't contain any pointers. We shouldn't allow pointers in the condition of the processor.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed the code that was producing the event containing the string pointer too. I added this code to just make it work in case someone makes this mistake in a community Beat.

We shouldn't allow pointers in the condition of the processor.

How do we enforce this? Logging a warning isn't really strong enforcement. Ideally we could prevent a *string values from ever being written to the event, but this would require an actual event type with an API that's not a MapStr.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was expecting that ConvertToGenericEvent should fix that after marshalling & unmarshalling a pointer to string, becomes string: https://github.com/elastic/beats/blob/master/libbeat/common/event.go#L70. But I assume it doesn't do that.
Ideally, it would be nice if we can restrict somehow the type of the fields in the event, but for now an warning should be enough.

return *s, nil
default:
return "", fmt.Errorf("unkown type %T passed to extractString", unk)
return "", fmt.Errorf("unknown type %T passed to extractString", unk)
}
}
23 changes: 23 additions & 0 deletions libbeat/processors/config_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
package processors

import (
"testing"

"github.com/stretchr/testify/assert"
)

func TestExtractString(t *testing.T) {
input := "test"

v, err := extractString(input)
if err != nil {
t.Fatal(err)
}
assert.Equal(t, input, v)

v, err = extractString(&input)
if err != nil {
t.Fatal(err)
}
assert.Equal(t, input, v)
}