Race Condition in Output Plugins

[tarrychk]

I deployed a telegraf v0.1.9 instance with multiple output plugins configured. Unfortunately it crashes every few hours. I compiled telegraf with go race detector enabled and it threw following backtrace:

==================
WARNING: DATA RACE
Write by goroutine 50:
  runtime.mapassign1()
      /tmp/workdir/go/src/runtime/hashmap.go:411 +0x0
  github.com/influxdb/telegraf/outputs/kafka.(*Kafka).Write()
      /go/src/github.com/influxdb/telegraf/outputs/kafka/kafka.go:68 +0xb76
  github.com/influxdb/telegraf.(*Agent).flush.func1()
      /go/src/github.com/influxdb/telegraf/agent.go:298 +0x10e

Previous write by goroutine 49:
  runtime.mapassign1()
      /tmp/workdir/go/src/runtime/hashmap.go:411 +0x0
  github.com/influxdb/influxdb/client.(*Client).Write()
      /go/src/github.com/influxdb/telegraf/Godeps/_workspace/src/github.com/influxdb/influxdb/client/influxdb.go:191 +0x1088
  github.com/influxdb/telegraf/outputs/influxdb.(*InfluxDB).Write()
      /go/src/github.com/influxdb/telegraf/outputs/influxdb/influxdb.go:128 +0x2f5
  github.com/influxdb/telegraf.(*Agent).flush.func1()
      /go/src/github.com/influxdb/telegraf/agent.go:298 +0x10e

Goroutine 50 (running) created at:
  github.com/influxdb/telegraf.(*Agent).flush()
      /go/src/github.com/influxdb/telegraf/agent.go:302 +0x15a
  github.com/influxdb/telegraf.(*Agent).crankParallel()
      /go/src/github.com/influxdb/telegraf/agent.go:218 +0x706
  github.com/influxdb/telegraf.(*Agent).Run()
      /go/src/github.com/influxdb/telegraf/agent.go:390 +0x218
  main.main()
      /go/src/github.com/influxdb/telegraf/cmd/telegraf/telegraf.go:160 +0x1d4b

Goroutine 49 (running) created at:
  github.com/influxdb/telegraf.(*Agent).flush()
      /go/src/github.com/influxdb/telegraf/agent.go:302 +0x15a
  github.com/influxdb/telegraf.(*Agent).crankParallel()
      /go/src/github.com/influxdb/telegraf/agent.go:218 +0x706
  github.com/influxdb/telegraf.(*Agent).Run()
      /go/src/github.com/influxdb/telegraf/agent.go:390 +0x218
  main.main()
      /go/src/github.com/influxdb/telegraf/cmd/telegraf/telegraf.go:160 +0x1d4b

Influxdb and kafka output plugin both modify the BatchPoints datastructure passed to Write func by defining Tags. Those tags are only required locally, so I think copying the Points datastructure before modifying it is in required.

Cya Ed

[sparrc]

Thanks for the report @tarrychk, looking into it