[Epic][Security Solution][Detections] Rule Execution Log - UI on the Rule Details page

[banderror]

Summary

Replace the existing "Failure History" tab with an enhanced Rule Execution Log UI on the Rule Details page.

New rule execution log documents are going have some standard ECS fields + some custom fields, which will make them technically similar to detection alerts and source events in terms of flexibility of analysis, showing in tables in the UI etc. This will allow us to implement a more advanced Rule Execution Log UI - not only with 5 last failures, but with all rule execution status updates (current statuses are going to run, succeeded, warning, failed) without limitation in their number, current execution metrics (querying time, indexing time, gaps, etc), any new execution metrics (if needed), any additional events with arbitrary data, and just generic log messages for observability purposes.

Ideas popped up during chatting with @yiyangliu9286 and @xcrzx:

• We likely need a normal log table (like we have now) to allow the user to troubleshoot issues with rule execution and find precise details about what happened and when.
• Additional view with charts and statistics might help the user to get a bigger picture. E.g. how many failures happened within a time frame etc.
• Probably it could be possible to use row renderers in the log table.

Resources

• Whimsical wireframe
• Figma

To do

First iteration (simple log UI with basic filtering and pagination, no visualizations):

• [Security Solution][Detections] New endpoint for Rule Execution Log #119598
• [Security Solution][Detections] New Rule Execution Log UI #119599

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[spong]

Discussed #124198 with team, holding for 8.2 to further iterate on implementation and UX.

[yiyangliu9286]

@spong @banderror updated the mvp design based on our discussion (link to Figma):

• Rule execution log view - default state UI:

There will be a tooltip on "Show metrics columns" toggle:

Turn on to show more metrics data in this table. Metrics columns are:

• Total alerts created
• Total alerts detected
• Gap duration
• Index duration
• Search duration
• Schedule delay

• When users want to filter by “rule execution id”, adding an action to each rule execution row:

• After filtering by rule execution id, will direct users to alerts tab w/ count (edge case: if nothing is found, will show an empty state like what we have in product currently):

[spong]

Awesome -- thank you @yiyangliu9286! 🙂 Will implement these in #126215 and ping you when ready for review!