Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send out JSON formatted emails for alerts #2892

Closed
iSignal opened this issue Nov 8, 2019 · 1 comment
Closed

Send out JSON formatted emails for alerts #2892

iSignal opened this issue Nov 8, 2019 · 1 comment
Assignees
Labels
area/platform Yugabyte Platform

Comments

@iSignal
Copy link
Contributor

iSignal commented Nov 8, 2019

No description provided.

@iSignal iSignal added the area/platform Yugabyte Platform label Nov 8, 2019
@iSignal iSignal self-assigned this Nov 8, 2019
iSignal added a commit that referenced this issue Jan 27, 2020
Summary:
Multiple improvements requested by customers for 2.1.
1/ API to disable alert emails per universe for a given time interval or permanently.
2/ Send out alerts email in JSON format
3/ App config to report only errors in alert emails

For (1) the API request looks like
```
curl --data 'disabled=true&disablePeriodSecs=3600' -X POST -H'X-AUTH-TOKEN: XXXXX' http://localhost:9000/api/v1/customers/f33e3c9b-75ab-4c30-80ad-cba85646ea39/universes/dadbd2bc-c202-4e8d-bf9d-a0a214ea3c4f/config_alerts
```
disablePeriodSecs is an optional parameter. If disabled=false, this parameter is ignored even if set.

For (2), the current email is now sent as a multipart MIME message. The plaintext part has the JSON email so users can retrieve and parse it if needed.

For (3), the new true/false config setting is a check box in the customer profile (customerconfig). The behavior of alert emails is now as follows:

Emails are sent in the following cases:
* Every status update period, a full list of nodes and checks is sent.

* Every health check interval, an email is sent only if errors are discovered. The email has the full list of nodes and checks unless report_only_errors is set in the profile, in which case, only errors are reported.
** The first success after errors clear also results in an email. The email has the full list of nodes and checks unless report_only_errors is set in the profile, in which case, the email body just says "No errors to report".

Test Plan:
Multiple test cases
For (1), there is a unit test that covers the key scenarios. I also verified manually that the correct arguments are passed to the python script.

For (2), I manually inspected the multipart MIME message and verified it had the right JSON text.

For (3), I simulated errors on the tserver in the python script and verified that only the errors were sent out with the setting enabled. I also verified that the success email gets sent out when errors stop.

Reviewers: arnav, ram, bogdan

Reviewed By: ram

Subscribers: jenkins-bot, yugaware

Differential Revision: https://phabricator.dev.yugabyte.com/D7816
@iSignal
Copy link
Contributor Author

iSignal commented Feb 21, 2020

The alert email is now a multi part MIME with the plain text part being in JSON.

@iSignal iSignal closed this as completed Feb 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/platform Yugabyte Platform
Projects
None yet
Development

No branches or pull requests

1 participant