Live feature #142
Live feature #142
Conversation
Instead of pulling all metrics with http, this feature allows Seyren to listen carbon relay (see pickle protocol http://graphite.readthedocs.org/en/latest/feeding-carbon.html). I introduce live package where CarbonPickleListener, the TCP server (quick implementation), is located. It reads inputstream socket, decode pickle protocol (with the help of jython, see maven dependency), and add a MetricsTask to an executor. MetricsTask find all checks in MongoDb that are enabled and live, both with true value. And of course, I re-use CheckRunner to store/send alert. In "Check Dialog" UI, if "live" option is checked, the method for checking the check will be that described above, otherwise a 'classic' http check is used (default behaviour), it is exclusive. Users are responsible to filter metrics in relay-rules.conf and not send all metrics to Seyren (to avoid performance problem). Example: How to send to Seyren only 'load' metrics, edit /opt/graphite/conf/relay-rules.conf like and restart carbon-relay: [seyren] pattern = .*load.* destinations = [ip seyren]:[port]:[instance] continue = true [default] default = true destinations = [ip carbon cache]:[port]:[instance] Thera are some limitations: * carbon-relay must have access Seyren (while http is easiest) * metric sended via carbon-relay are raw value, you can not access to graphite functions But with this feature, Seyren is more reactive. Enjoy :)
|
Well, this looks awesome...!! |
|
@obazoud, you legend you. I'm keen to see if a large number of metrics has much of an effect on CPU utilisation. (Noted your comment on filtering in relay-rules.conf). |
|
@paulcgt don't worry this feature is disabled by default :) Maybe I should write in other way. The point is here that my graphite instance contains ~ 20K metrics and metrics collectors (like collectd, jmxtrans, logster, ...) send metrics every 10s to carbon. And of course, I don't want to send 20K metrics each 10s to Seyren. So, a good practice is to filter data at the source (carbon-relay). |
|
I'm happy to pull. @neilprosser can you cross check please? We'll need to add the config variables to the README. As a side note it might be nice to extract the pickle code into it's own library so other Java/Graphite tools can get live update awesomeness too |
Done, see bcd7365 |
|
Lets get this in, awesome feature @obazoud 👍 |
Instead of pulling all metrics with http, this feature allows Seyren to listen carbon relay (see pickle protocol http://graphite.readthedocs.org/en/latest/feeding-carbon.html).
I introduce live package where CarbonPickleListener, the TCP server (quick implementation), is located.
It reads inputstream socket, decode pickle protocol (with the help of jython, see maven dependency), and add a MetricsTask to an executor.
MetricsTask find all checks in MongoDb that are enabled and live, both with true value.
And of course, I re-use CheckRunner to store/send alert.
In "Check Dialog" UI, if "live" option is checked, the method for checking the check will be that described above, otherwise a 'classic' http check is used (default behaviour), it is exclusive.
Users are responsible to filter metrics in relay-rules.conf and not send all metrics to Seyren (to avoid performance problem).
Example: How to send to Seyren only 'load' metrics, edit /opt/graphite/conf/relay-rules.conf like and restart carbon-relay:
[seyren]
pattern = .load.
destinations = [ip seyren]:[port]:[instance]
continue = true
[default]
default = true
destinations = [ip carbon cache]:[port]:[instance]
Thera are some limitations:
With this feature, Seyren is more reactive.
Enjoy :)