Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add event remapping functionality #17

Open
mike-sol opened this issue Jun 6, 2019 · 0 comments
Open

Add event remapping functionality #17

mike-sol opened this issue Jun 6, 2019 · 0 comments

Comments

@mike-sol
Copy link

mike-sol commented Jun 6, 2019

The plugin works fine, but unfortunately the way it structures the events that it creates is less than ideal.

In my particular case, I tried to use the Netapp Harvester (which gets metrics from a Netapp storage system and outputs plaintext Graphite) with this input plugin so as to not have to spin up an actual Graphite server.

The result was a bunch of events with field names like:
netapp.capacity.Cluster-HQ.Cluster-HQ.node.Cluster-HQ-01.aggr.cluster_hq_01_aggr1.snapshot_reserve_avail

Elasticsearch quickly began to throw errors when the number of fields in the (brand new, empty) index hit 1000. It occurs to me that any reasonable Graphite deployment using a metrics collector would similarly hit those kinds of numbers.

What I would like to do is to have some options to re-map the name.

Option 1 could be as simple as just having a field named 'metric' containing the metric name, and a separate field called value with the value. May be some issue with conflicts for non-numeric values, might have to go with a text or similarly named field for those instead of value. If nothing else, this would make the plugin perfectly useful for me.

Option 2 would be more complex, you'd split on dots, and then with the resulting data create a hierarchy, so you'd map from:

netapp.capacity.Cluster-HQ.Cluster-HQ.node.Cluster-HQ-01.aggr.cluster_hq_01_aggr1.snapshot_reserve_avail = 100

to

metric.1 = netapp
metric.2 = capacity
metric.3 = Cluster-HQ
....
value = 100

This option would enable the canny Elasticsearcher to effectively be able to query the hierarchy the same way as a Graphiter would with wildcards.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant