Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Empty tag value causes error on InfluxDB output #2390

Closed
lpic10 opened this issue Feb 10, 2017 · 6 comments
Closed

Empty tag value causes error on InfluxDB output #2390

lpic10 opened this issue Feb 10, 2017 · 6 comments
Labels
bug unexpected problem or unintended behavior
Milestone

Comments

@lpic10
Copy link
Contributor

lpic10 commented Feb 10, 2017

Bug report

When a kubernetes label is empty, a empty tag is created and causes InfluxDB to complain:

unable to parse 'docker_container_cpu,cpu=cpu0,io.kubernetes.pod.namespace=xxxxx,io.kubernetes.container.name=POD,cluster=xxxxx,engine_host=xxxxx,io.kubernetes.container.terminationMessagePath=,container_version=v1.2.0,io.kubernetes.container.hash=4866dc9e,io.kubernetes.pod.name=xxxxx,io.kubernetes.pod.terminationGracePeriod=30,io.kubernetes.pod.uid=b3a1ef4e-ef6e-11e6-b4b8-005056835c46,io.kubernetes.container.restartCount=0,host=xxxxx,container_name=k8s_POD.4866dc9e_xxxxx_xxxxx_b3a1ef4e-ef6e-11e6-b4b8-005056835c46_6d9803e6,container_image=xxxxx usage_total=2122656i,container_id="76f7bc0453ffb0215bf9c776e9890e5e454c1ebaabd315074c646a1d0bb4956b" 1486717284000000000': missing tag value

This issue happens when using docker input and/or prometheus input reading metrics from kubelet.

I'm not sure how this needs to be fixed, possible solutions are:

  1. Fix in the prometheus and docker input plugins checking if label value is empty, but there may be more plugins doing the same
  2. Fix in the influxdb output plugin, but maybe it is a bad idea to allow the same in other outputs
  3. Fix this in the metric/tag object, not allowing empty tags to happen

Additional info:

Telegraf built from master

@sparrc
Copy link
Contributor

sparrc commented Feb 10, 2017

please fix it in (1)

@sparrc sparrc added this to the Future Milestone milestone Feb 10, 2017
@sparrc sparrc added the bug unexpected problem or unintended behavior label Feb 10, 2017
@phemmer
Copy link
Contributor

phemmer commented Feb 11, 2017

@sparrc I thought empty tags were already supposed to be skipped? #1927 (comment)

@sparrc
Copy link
Contributor

sparrc commented Feb 11, 2017

it appears I must have been wrong about that

@sparrc
Copy link
Contributor

sparrc commented Feb 11, 2017

looks like it's a TODO....classic 🤦‍♂️ https://github.com/influxdata/telegraf/blob/master/metric/metric.go#L47

@lpic10 sorry for the earlier misdirection, can you fix this by fixing the TODO in metric.go?

@lpic10
Copy link
Contributor Author

lpic10 commented Feb 12, 2017

@sparrc no problem, do you think this is ok?

for k, v := range tags {
    vsize := len(escape(v, "tagval"))
    if vsize > 0 {
        taglen += 2 + len(escape(k, "tagkey")) + vsize
    } else {
        delete(tags, k)
    }
}

@forestjohnsonpeoplenet
Copy link

I hit this bug too in telegraf 1.2.1 with a different input plugin (consul). It's actually a regression. Telegraf 1.0.0 would skip the empty tags.

The empty tag in line protocol is invalid in both influxdb 1.0.0 and in influxdb latest. So I'm glad to see this getting fixed.

@sparrc sparrc modified the milestones: 1.3.0, Future Milestone Feb 16, 2017
@sparrc sparrc changed the title Empty Kubernetes label causes error on InfluxDB output Empty tag value causes error on InfluxDB output Feb 16, 2017
@sparrc sparrc closed this as completed in 5da40d5 Feb 16, 2017
maxunt pushed a commit that referenced this issue Jun 26, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug unexpected problem or unintended behavior
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants