Event log documentation

[arisonl]

We need documentation for the event log, preferably for GA.

[elasticmachine]

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

[pmuellr]

Is this for diagnostic purposes, or ???

We do have some doc in the plugin's README.md. I suspect the README.md could be beefed up a bit more - not sure it's really been updated since we added the event log, but also don't think we've really changed much of the API. It's also very much aimed at developers WRITING events to the event log, not READING the event log.

I don't see much mention of what Watcher's history index might contain, beside some Discuss posts: https://www.google.com/search?q=elasticsearch+%22watch+history%22 . Is there some technical reason we DIDN'T document it? I'm guessing we just never got around to it (like us with this).

Seems like the big hole in the README - and what would be of most interest to customers - are the actual events that are written out, for actions and alerts.

Also, if we're going to "promote" the event log, I wonder if we want to make it easier for people to access. I think you'd need admin or kibana_system privs to read it, given the name .kibana-* and the roles assigned to indices matching that pattern. Perhaps we could expand that with a new priv / role that could be assigned. We could also provide a Kibana index pattern, but that might be slightly weird if not many people can even "see" it (security-wise).

[arisonl]

@pmuellr we know of customers who are looking to use it. One example use case that was brought up is to investigate and understand why an alert did not fire, while it was expected to. This investigation may end up looking into the event log down the path. They have also provided an example of the equivalent current tool they are using, here is a screenshot:

[pmuellr]

So the customer built an event log viewer? :-) \o/

Seems like the current README contents are developer-oriented, so would go in the same kind of docs where we document the alertsClient and actionsClient (TS APIs).

Then a new section describing, which would be more in the place where HTTP APIs are documented, describing the index structure + ILM, the fields in the documents, and the specific event.provider / event.action combination of documents, what they mean, and what they contain.

These could be embedded within relevant Alerting docs, since only alerting is using the event log for now, but we could lift up to make it more general if we ever have more plugins use it.

[mikecote]

Moving from 7.x - Candidates to 8.x - Candidates (Backlog) after the latest 7.x planning session.

[ymao1]

@arisonl We've added some documentation for example event log queries as part of Alerting Troubleshooting: https://www.elastic.co/guide/en/kibana/master/event-log-index.html. Is that sufficient for this issue?

[arisonl]

@ymao1 we only have query examples there. Do you think that it would be helpful to list the schema and fields with a description of what each field stores?

[mikecote]

Closing in favour of the rule run history tab in the UI and lack of requests to GA event log access.