Support glob patterns in diskio devices option

[ghost]

By default, diskio ends up capturing for every partition. This probably isnt what most people want, but passing in regular expressions does not work.

This works (but includes sda AND sda[123]:

[[inputs.diskio]]
    interval = "1s"
  skip_serial_number = true
  [inputs.diskio.tagpass]
  name = [ "sd*", "md*" ]

This does not work, but should:

[[inputs.diskio]]
    interval = "1s"
  skip_serial_number = true
  [inputs.diskio.tagpass]
  name = [ "sd[a-z]+", "md[0-9]+" ]

[danielnelson]

Does this work? name = [ "sd[a-z]", "md[0-9]" ]

Still regular expressions could be nice over glob patterns in many cases, but we would have to introduce them in a backwards compatible way.

[ghost]

@danielnelson sadly, that seems to include sda1 too. In any case, we have machines with >26 SCSI devices (sdaa, etc.).

[danielnelson]

Are you sure? It seems to work for me, but I only have sda, sda1, sda2. Telegraf is using https://github.com/gobwas/glob for all filters, there is a list of supported patterns there.

For sdaa, assuming this works at all, you could always try name = ["sd[a-z]", "sd[a-z][a-z]"].

On the topic of supporting regular expression filters, we could have an option on agent to select the pattern type, but I'm concerned that it will be confusing when people try to copy the config files of others. Another option is regex variants of every filter. I'll have to think about if the added complexity is worth it.

[ghost]

With our version of telegraf (built off 91f48e7), this config:

[[inputs.diskio]]
  interval = "1s"
  skip_serial_number = true
  [inputs.diskio.tagpass]
  name = ["sd[a-z]", "sd[a-z][a-z]", "dm*"]

Results in this output:

[root@hostname a]# /usr/local/telegraf/bin/telegraf -config /etc/telegraf/telegraf.conf -debug -test | grep diskio | grep sda
> diskio,bu=linux,cls=server,dc=carf,env=production,host=hostname,name=sda,sr=splunk_indexer,trd=false io_time=39634483i,read_bytes=497947633664i,read_time=69324302i,reads=17210115i,write_bytes=9883124554752i,write_time=2333540705i,writes=63523754i 1493323075000000000
> diskio,bu=linux,cls=server,dc=carf,env=production,host=hostname,name=sda1,sr=splunk_indexer,trd=false io_time=2629i,read_bytes=144232448i,read_time=2633i,reads=5633i,write_bytes=0i,write_time=0i,writes=0i 1493323075000000000
> diskio,bu=linux,cls=server,dc=carf,env=production,host=hostname,name=sda2,sr=splunk_indexer,trd=false io_time=6574i,read_bytes=55970816i,read_time=15195i,reads=3056i,write_bytes=29834240i,write_time=203i,writes=532i 1493323075000000000
> diskio,bu=linux,cls=server,dc=carf,env=production,host=hostname,name=sda3,sr=splunk_indexer,trd=false io_time=39623571i,read_bytes=497627704320i,read_time=69300227i,reads=17197942i,write_bytes=9883094720512i,write_time=2333540502i,writes=63523222i 1493323075000000000

It may be that it works in a later version. I'll try that later. But would it be possible to change it to attempt to compile a regular expression based on the config, and if that fails fall back to glob?

[danielnelson]

The problem is that some patterns are both valid regex and glob, but they have different meanings.

[ghost]

If you required the regex to be in the form /^regex$/ (i.e with "/" at the start and end), I think that would be enough to work as no glob will end with / and match any real device.

[danielnelson]

It looks like the glob syntax above only works in master (1.3) at this point.

I think this could be another reasonable way to allow regex. It might not be 100% compatible but it is pretty close.

Since tagpass is a global filter and so any changes would have wide repercussions, and considering there is a pattern that works fairly well with the latest glob patterns, I think we should just add glob support to the diskio devices option, similar to what was discussed in #1073.

[ghost]

Thanks. We plan to upgrade as soon as you cut a 1.3 release (I understand a rc is due imminently) so the glob syntax is a nice work around (I pushed that new config). If you do support regex's in future, that would be ideal for us; if only because it feels like the right way to achieve what we want to achieve!

[danielnelson]

Yes 1.3rc1 hopefully today assuming I can figure out how to change the package builds to use go1.8.1. I'm going to rename this ticket to reflect what I think we should change on the diskio plugin in the short term.