-
Notifications
You must be signed in to change notification settings - Fork 489
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Better Sensu alerts #1214
Better Sensu alerts #1214
Conversation
so now we have ID, Message, Hostname and Details
The first commit has already been integrated. For the auto_resolve, i'm not sure if it is the most common parameter but we use sensu like this. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change would be more appropriate if it only changed the Sensu handlers as this is not an option that all Alerts need. I have added comments on how and what to change to do that.
Thanks for the PR!
idTmpl *text.Template | ||
messageTmpl *text.Template | ||
detailsTmpl *html.Template | ||
hostnameTmpl *text.Template |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since this is specific to the Sensu handler the template logic should exists in services/sensu/service.go.
Have a look at the Alerta handler as it does something quite similar to this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok
@@ -355,6 +361,7 @@ func newAlertNode(wants EdgeType) *AlertNode { | |||
Id: defaultIDTmpl, | |||
Message: defaultMessageTmpl, | |||
Details: defaultDetailsTmpl, | |||
Hostname: defaultHostnameTmpl, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Again instead of making this a global option to all alerts add it the the SensuHandler type which can be found later on in this file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok
@@ -133,9 +136,10 @@ func (s *Service) prepareData(name, output string, level alert.Level) (*net.TCPA | |||
|
|||
postData := make(map[string]interface{}) | |||
postData["name"] = name | |||
postData["source"] = c.Source | |||
postData["source"] = hostname |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would source
be a better name for this option instead of hostname
. While most of the time the source is a hostname the use of the name hostname initially confused me because Kapacitor can only have a single hostname and it doesn't change. But this is not talking about Kapacitor's hostname but rather the hostname of the source.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok
postData["output"] = output | ||
postData["status"] = status | ||
postData["auto_resolve"] = false |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What does this change do?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If a node has a crit alert then a ok alert, the node is always in crit. In our case, this allow to find Flapping on network equipment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In that case we should create a new property on the SensuHandler called noAutoResolve(), as I think most users will want the auto resolve behavior.
For reference auto_resolve docs https://sensuapp.org/docs/0.23/reference/checks.html
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i'm sure you're right. In most cases, auto_resolve must be true.
Required for all non-trivial PRs
Required only if applicable
You can erase any checkboxes below this note if they are not applicable to your Pull Request.