Allow service inputs to set precision

[headkace]

Have tried "ms" and "s". telegraf receives and writes to influxDB, but always turns up in ns. ... The bare-bones, almost stock telegraf config :

[global_tags]
[agent]
  interval = "60s"
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "60s"
  flush_jitter = "0s"
  precision = "ms"
  debug = false
  quiet = false
  hostname = ""
  omit_hostname = true
[[outputs.influxdb]]
  precision = "s"
  urls = ["http://localhost:8086"] # required
  database = "server_stats" # required
  retention_policy = ""
  write_consistency = "any"
  timeout = "20s"
[[inputs.udp_listener]]
  service_address = ":8092"
  allowed_pending_messages = 10000
  data_format = "influx"

[sparrc]

@headkace please provide more information, such as your version, what kind of data you're sending, how you have confirmed that it's arriving in ns, etc.

[headkace]

influxdb version 0.13.0. telegraf version 1.0.0-b3.

Data was simple one-measure, one-tag line data without timestamp sent over netcat to the UDP port and looked something like this :

cpu_all_idle,host=abc2 value=92.56

I confirmed it's arrival with the influx web interface where field "time" displayed in nanoseconds.

[sparrc]

@headkace if you look at the full agent config (with comments), you will see that the precision setting is not supported for service inputs:

[agent]
  ## Default data collection interval for all inputs
  interval = "10s"
  ## Rounds collection interval to 'interval'
  ## ie, if interval="10s" then always collect on :00, :10, :20, etc.
  round_interval = true

  ## Telegraf will send metrics to outputs in batches of at
  ## most metric_batch_size metrics.
  metric_batch_size = 1000
  ## For failed writes, telegraf will cache metric_buffer_limit metrics for each
  ## output, and will flush this buffer on a successful write. Oldest metrics
  ## are dropped first when this buffer fills.
  metric_buffer_limit = 10000

  ## Collection jitter is used to jitter the collection by a random amount.
  ## Each plugin will sleep for a random time within jitter before collecting.
  ## This can be used to avoid many plugins querying things like sysfs at the
  ## same time, which can have a measurable effect on the system.
  collection_jitter = "0s"

  ## Default flushing interval for all outputs. You shouldn't set this below
  ## interval. Maximum flush_interval will be flush_interval + flush_jitter
  flush_interval = "10s"
  ## Jitter the flush interval by a random amount. This is primarily to avoid
  ## large write spikes for users running a large number of telegraf instances.
  ## ie, a jitter of 5s and interval 10s means flushes will happen every 10-15s
  flush_jitter = "0s"

  ## By default, precision will be set to the same timestamp order as the
  ## collection interval, with the maximum being 1s.
  ## Precision will NOT be used for service inputs, such as logparser and statsd.
  ## Valid values are "ns", "us" (or "µs"), "ms", "s".
  precision = ""

so this isn't actually a bug, but could be a feature request. The reason for the precision setting not applying to service inputs is that users may want to write to their service input(s) at a certain timestamp which gets passed through the system transparently, while also maintaining the "s"-level precision on their other metrics. This becomes especially important for the logparser input, which can frequently have multiple metrics at the same timestamp.

The workaround is to write to your UDP input with a timestamp at the precision that you want. InfluxDB line-protocol does have this limitation of defaulting to nanosecond precision, so I'm not sure there is a great way to do this, and the database itself requires that users write in the precision that they want.

[headkace]

Allowing precision to be set at inputs level sounds like a good solution. That is what I'd first tried. Then looking at the sample config file again I gathered that that was not an option and tried to set it on outputs.influxdb. ... This could raise the issue of having 2 (or 3) different precision settings : input setting, output setting, and actual timestamp in the data. Having a clear, consistent way to resolve those (perhaps with further settings options) would be a very good thing.

[babayega]

Is there some progress on this or does someone has a solution for this.
I'm using socket_listener with udp and want to write to influxdb with "ms" precision.

Please let me know of any updates.

[danielnelson]

The only workaround for now is to send the data with the desired timestamp to socket_listener.