Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change add_cloud_metadata to not overwrite cloud field #11612

Merged
merged 9 commits into from
Apr 9, 2019
Prev Previous commit
Next Next commit
Remove code to check dots in field keys
kaiyan-sheng committed Apr 9, 2019
commit 6a3e60bd984e12458cdc205b0cb273c786c4151a
12 changes: 3 additions & 9 deletions libbeat/processors/add_cloud_metadata/add_cloud_metadata.go
Original file line number Diff line number Diff line change
@@ -25,7 +25,6 @@ import (
"io/ioutil"
"net"
"net/http"
"strings"
"sync"
"time"

@@ -368,14 +367,9 @@ func (p *addCloudMetadata) Run(event *beat.Event) (*beat.Event, error) {
// cloud fields. For example aws module writes cloud.instance.* to events already, with overwrite=false,
// add_cloud_metadata should not overwrite these fields with new values.
if !p.initData.overwrite {
for key := range event.Fields {
keySplits := strings.Split(key, ".")
if keySplits[0] == "cloud" {
cloudValue, _ := event.GetValue(key)
if cloudValue != nil {
return event, nil
}
}
cloudValue, _ := event.GetValue("cloud")
if cloudValue != nil {
return event, nil
}
}

22 changes: 18 additions & 4 deletions libbeat/processors/add_cloud_metadata/provider_aws_ec2_test.go
Original file line number Diff line number Diff line change
@@ -114,18 +114,32 @@ func TestRetrieveAWSMetadata(t *testing.T) {
},
{
common.MapStr{
"cloud.provider": "ec2",
"provider": "ec2",
},
common.MapStr{
"cloud.provider": "ec2",
"provider": "ec2",
"cloud": common.MapStr{
"provider": "ec2",
"instance": common.MapStr{
"id": "i-11111111",
},
"machine": common.MapStr{
"type": "t2.medium",
},
"region": "us-east-1",
"availability_zone": "us-east-1c",
},
},
},
{
common.MapStr{
"provider": "ec2",
"cloud.provider": "ec2",
},
// NOTE: In this case, add_cloud_metadata will overwrite cloud fields because
// it won't detect cloud.provider as a cloud field. This is not the behavior we
// expect and will find a better solution later in issue 11697.
common.MapStr{
"provider": "ec2",
"cloud.provider": "ec2",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really expected? Not even sure what ES will do with such a document.

"cloud": common.MapStr{
"provider": "ec2",
"instance": common.MapStr{