[Security Solution][Detections] Adds rule execution log table #126215

spong · 2022-02-23T00:10:44Z

Summary

Resolves #119598, #119599, #101014

Adds Rule Execution Log table to Rule Details page:

Implementation notes

The useful metrics within event-log for a given rule execution are spread between a few different platform (execute-start, execute) and security (execution-metrics, status-change) events. In effort to provide consolidated metrics per rule execution (and avoiding a lot of empty cells and mis-matched statuses like in the image below)

these rule execution events are aggregated by their executionId, and then fields are merged from each different event. This PR was re-worked to take advantage of the new event-log aggregation support added in #126948, and is no longer implemented as an in-memory aggregation server side.

Due to restrictions around supplying search filters that may match multiple sub-agg buckets and missing data (see discussion here), it was decided that we'd disable the search bar for the time being. We have both a near-term (writing single rollup event) and long-term (ES|QL) solution that will allow us to re-enable this functionality.
Note, since a terms agg is used to fetch all execution events, an upper bound must be set. See this discussion for more details, but setting this max to 1000 events for the time being, and returning total cardinality of execution events back within total to allow the UI to inform the user that they should narrow their search further to better isolate and find possible issues. This should be a be a reasonable constraint for most all rules as a rule executing every 5 minutes, 1000 executions would cover over 3 days of execution time.

The Filter for alerts action will be available on all Succeeded/Partial Failure executions even if there weren't alerts generated until #126210 is merged and we can start returning the alert count, at which point we can programmatically enabled/disable this action based on alert count.

Todo:

Continue adding test coverage, but most stubbed out (see TODO's)
Resolve ID-joining to cleanup query logic and still support search filters (waiting on feedback from query team)
Follow-up UI review
- Spoke with @yiyangliu9286 and implemented initial feedback, will do final sync before merging

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
- TODO: Collaborate with @elastic/security-docs
Unit or functional tests were updated or added to match the most common scenarios
- Existing updated, additional tests required, including FTR suite for exercising total_alerts/total_hits changes
Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)

…tion-log-table

xcrzx · 2022-03-28T18:07:22Z

Sometimes filter by status behaves a bit strangely. When selecting "Partial failure," the table still shows "Failed" executions. I guess that happens because, despite the final "failed" status, there were warnings during executions?

Screen.Recording.2022-03-28.at.19.53.20.mov

kibana-ci · 2022-03-28T18:56:38Z

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`securitySolution`	2991	3000	+9

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`securitySolution`	4.8MB	4.8MB	+14.8KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`securitySolution`	247.4KB	249.2KB	+1.8KB

Unknown metric groups

API count

id	before	after	diff
`@kbn/securitysolution-rules`	23	24	+1

History

💔 Build #34034 failed 611a0c0
💔 Build #33789 failed 2b294ad
💚 Build #32913 succeeded ca6e3f4
💚 Build #32477 succeeded d3f7b70
💔 Build #32464 failed 1e8dbfb

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @spong

xcrzx

Fantastic work, @spong 🚀 🎉 Tested locally; the new event log functionality looks amazing!
I left a couple of comments, primarily nits, and neither is a blocker for this PR and could be addressed after FF if needed.

...y_solution/public/detections/containers/detection_engine/rules/use_rule_execution_events.tsx

...ecurity_solution/common/detection_engine/schemas/request/get_rule_execution_events_schema.ts

...olution/server/lib/detection_engine/rule_execution_log/client_for_routes/client_interface.ts

...curity_solution/server/lib/detection_engine/rule_execution_log/event_log/event_log_reader.ts

...r/lib/detection_engine/rule_execution_log/event_log/get_execution_event_aggregation/index.ts

xcrzx · 2022-03-28T18:46:43Z

...r/lib/detection_engine/rule_execution_log/event_log/get_execution_event_aggregation/index.ts

+  for (const field of sortFields) {
+    if (!Object.keys(SORT_FIELD_TO_AGG_MAPPING).includes(field)) {
+      throw new BadRequestError(
+        `Invalid sort field "${field}" - must be one of [${Object.keys(
+          SORT_FIELD_TO_AGG_MAPPING
+        ).join(',')}]`
+      );
+    }
+  }


Probably we should move this input validation to the route handler level 🤔 Ideally, helper functions like getExecutionEventAggregation shouldn't be aware of their execution context, i.e., they shouldn't throw BadRequestError errors.

I was following a similar pattern that the ResponseOps folks did in their PR where they still validated inputs at the getAggs layer in case anyone else consumed this function without validating their inputs. That said, I am throwing the wrong error here so if we want to keep this logic we should change this to a different error type. Btw, When switching to Decoded we'll have input validation at the route handler, so will be covered there 👍

xcrzx · 2022-03-28T18:52:40Z

...r/lib/detection_engine/rule_execution_log/event_log/get_execution_event_aggregation/index.ts

+    }
+  }
+
+  return {


I found this aggregation query pretty hard to follow. Maybe we can make it more readable by splitting it into multiple queries? As practice shows, there is no significant performance difference in executing one big aggregation vs. multiple smaller ones. But readability could be improved a lot.

We don't need to do anything with this query right now, but maybe that's something we should consider in the future.

Yeah, definitely agree with you that it's a monster agg -- I tried to help it with comments, but those can only do so much; a logical summary comment would be helpful here too so I'll add one of those. As for breaking it up into multiple queries, how could this be achieved while still maintaining functional sorting since there's two nested aggs that take the sort params (terms/bucket_sort for pagination)? And wouldn't you end back up with some in-memory aggs logic?

I might be missing something here, but am happy to rework this to multiple queries if we can -- let me know your thoughts!

spong · 2022-03-28T22:39:14Z

Thank you for the thorough review @xcrzx! I'm going to go ahead and merge as to not battle any flake getting a greenlight on CI again with the requested changes. Will post to each comment on this PR with the commit resolving it when ready. Thanks again!

spong · 2022-03-28T22:41:56Z

Sometimes filter by status behaves a bit strangely. When selecting "Partial failure," the table still shows "Failed" executions. I guess that happens because, despite the final "failed" status, there were warnings during executions?

Screen.Recording.2022-03-28.at.19.53.20.mov

Hrmm, perhaps? Usually the only time you should see weirdness here is with partial events as an execution is occurring (would like to resolve that case as well), but I'll take a peek and try to repro 👍

…n on Rule Execution Log (#128843) ## Summary One-liner fix for the `Show alerts for execution` action on the Rule Execution Log table. Had the wrong key after changing the response interface. Working on the follow-up feedback PR from #126215, and will be including additional test coverage there, but wanted to get this in before the first BC for testing purposes. <img width="700" src="https://user-images.githubusercontent.com/2946766/160720595-f00280d1-3928-4511-8e74-2f5d50b103e9.gif" />

kibanamachine · 2022-03-30T22:46:19Z

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create backports run node scripts/backport --pr 126215 or prevent reminders by adding the backport:skip label.

…#129003) ## Summary Addresses feedback and fixes identified in #126215 Feedback addressed includes: * Adds route validation via io-ts decode and schema tests * Fixed styling of max execution events error by wrapping text (#129321) * Fixed types within `view alerts for execution` action onClick * Caps auto-refresh minimum to `1min` (#129332) * Adds cardinality aggs to initial `execution_uuid` query to properly report total counts when filtering by status * Disabled `View Alerts from Execution` action as current UX was too cumbersome with erasing users existing filters --- Additional follow-ups for another PR: - [ ] UI Unit tests - [ ] Finalize API Integration tests for gap remediation events - [ ] Persist table state (DatePicker/StatusFilter/SortField/SortOrder/Pagination) when navigating to other tabs on the same page - [ ] Update global DatePicker to daterange of execution for `view alerts for execution` action (and clear all other filters) - [ ] Support `disabled rule` platform error #126215 (comment) - [ ] Verify StatusFilter issue #126215 (comment) --- ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [X] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/)) - [X] This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))

…#129003) ## Summary Addresses feedback and fixes identified in #126215 Feedback addressed includes: * Adds route validation via io-ts decode and schema tests * Fixed styling of max execution events error by wrapping text (#129321) * Fixed types within `view alerts for execution` action onClick * Caps auto-refresh minimum to `1min` (#129332) * Adds cardinality aggs to initial `execution_uuid` query to properly report total counts when filtering by status * Disabled `View Alerts from Execution` action as current UX was too cumbersome with erasing users existing filters --- Additional follow-ups for another PR: - [ ] UI Unit tests - [ ] Finalize API Integration tests for gap remediation events - [ ] Persist table state (DatePicker/StatusFilter/SortField/SortOrder/Pagination) when navigating to other tabs on the same page - [ ] Update global DatePicker to daterange of execution for `view alerts for execution` action (and clear all other filters) - [ ] Support `disabled rule` platform error #126215 (comment) - [ ] Verify StatusFilter issue #126215 (comment) --- ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [X] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/)) - [X] This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server)) (cherry picked from commit bc413c6)

…#129003) (#129518) ## Summary Addresses feedback and fixes identified in #126215 Feedback addressed includes: * Adds route validation via io-ts decode and schema tests * Fixed styling of max execution events error by wrapping text (#129321) * Fixed types within `view alerts for execution` action onClick * Caps auto-refresh minimum to `1min` (#129332) * Adds cardinality aggs to initial `execution_uuid` query to properly report total counts when filtering by status * Disabled `View Alerts from Execution` action as current UX was too cumbersome with erasing users existing filters --- Additional follow-ups for another PR: - [ ] UI Unit tests - [ ] Finalize API Integration tests for gap remediation events - [ ] Persist table state (DatePicker/StatusFilter/SortField/SortOrder/Pagination) when navigating to other tabs on the same page - [ ] Update global DatePicker to daterange of execution for `view alerts for execution` action (and clear all other filters) - [ ] Support `disabled rule` platform error #126215 (comment) - [ ] Verify StatusFilter issue #126215 (comment) --- ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [X] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/)) - [X] This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server)) (cherry picked from commit bc413c6) Co-authored-by: Garrett Spong <spong@users.noreply.github.com>

… Part Deux (#130072) ## Summary Addresses feedback and fixes identified in #126215 & #129003 ##### Feedback addressed includes: * Adds toast for restoring global query state after performing `view alerts for execution` action <img width="500" src="https://user-images.githubusercontent.com/2946766/164511565-b77d3dc8-a8b5-4927-a947-54966a58c74f.gif" /> * Updates global SuperDatePicker to daterange of execution (+/- day) for `view alerts for execution` action (and clear all other filters) * See above gif * Remove redundant `RuleExecutionStatusType` (#129003 (comment)) * Persist table state (DatePicker/StatusFilter/SortField/SortOrder/Pagination) when navigating to other tabs on the same page <img width="500" src="https://user-images.githubusercontent.com/2946766/164512498-59416601-d967-4a27-b0cc-0715cc0662c0.gif" /> * Fix duration hours bug (`7 hours (25033167ms)` as `06:417:13:000`) <img width="500" src="https://user-images.githubusercontent.com/2946766/164511478-bf0bb6d8-d8b7-4c86-8fbd-b60090f00555.png" /> * Support `disabled rule` platform error (#126215 (comment)) * Updated `getAggregateExecutionEvents` to fallback to platform status from `event.outcome` if `security_status` is empty, and also falls back to `error.message` is `security_message` is empty. This also now queries for corresponding `event.outcome` if filter is provided so that platform-only events can still be displayed when filtering. <img width="500" src="https://user-images.githubusercontent.com/2946766/164510056-1e0bce86-8360-4d46-b591-2041457e3244.png" /> * Verify StatusFilter issue #126215 (comment) * Unable to reproduce, I believe the query updates around first querying for status may've fixed this? * Provide helpful defaults for `to`/`from` and support datemath strings again (#129003 (comment)) * Created enhancement for this here: #131095 * Adds UI Unit tests for RuleExecutionLog Table * Finalize API Integration tests for gap remediation events * Test methods developed for injecting arbitrary execution events while still working with event-log RBAC. See last [API integration test](https://github.com/elastic/kibana/blob/22cc0c8dbd2a1300675caf4c6d471d211ed44858/x-pack/test/detection_engine_api_integration/security_and_spaces/tests/get_rule_execution_events.ts#L121-L166) for technique. This can further be used to inject many execution events and expand tests around pagination, sorting, filters, etc. * Fixes `gap_duration`'s of `1-499`ms showing up as `-` instead of `0` * Fixes restore filters action to restore either absolute or relative datepicker as it originally was * Resolves #130946 * Adds `min-height` to tab container * Removes scroll-pane from ExceptionsViewer to match Alerts/Execution Log --- ##### Remaining follow-ups: None! 🎉 ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [X] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/)) - [X] This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))

… Part Deux (elastic#130072) ## Summary Addresses feedback and fixes identified in elastic#126215 & elastic#129003 ##### Feedback addressed includes: * Adds toast for restoring global query state after performing `view alerts for execution` action <img width="500" src="https://user-images.githubusercontent.com/2946766/164511565-b77d3dc8-a8b5-4927-a947-54966a58c74f.gif" /> * Updates global SuperDatePicker to daterange of execution (+/- day) for `view alerts for execution` action (and clear all other filters) * See above gif * Remove redundant `RuleExecutionStatusType` (elastic#129003 (comment)) * Persist table state (DatePicker/StatusFilter/SortField/SortOrder/Pagination) when navigating to other tabs on the same page <img width="500" src="https://user-images.githubusercontent.com/2946766/164512498-59416601-d967-4a27-b0cc-0715cc0662c0.gif" /> * Fix duration hours bug (`7 hours (25033167ms)` as `06:417:13:000`) <img width="500" src="https://user-images.githubusercontent.com/2946766/164511478-bf0bb6d8-d8b7-4c86-8fbd-b60090f00555.png" /> * Support `disabled rule` platform error (elastic#126215 (comment)) * Updated `getAggregateExecutionEvents` to fallback to platform status from `event.outcome` if `security_status` is empty, and also falls back to `error.message` is `security_message` is empty. This also now queries for corresponding `event.outcome` if filter is provided so that platform-only events can still be displayed when filtering. <img width="500" src="https://user-images.githubusercontent.com/2946766/164510056-1e0bce86-8360-4d46-b591-2041457e3244.png" /> * Verify StatusFilter issue elastic#126215 (comment) * Unable to reproduce, I believe the query updates around first querying for status may've fixed this? * Provide helpful defaults for `to`/`from` and support datemath strings again (elastic#129003 (comment)) * Created enhancement for this here: elastic#131095 * Adds UI Unit tests for RuleExecutionLog Table * Finalize API Integration tests for gap remediation events * Test methods developed for injecting arbitrary execution events while still working with event-log RBAC. See last [API integration test](https://github.com/elastic/kibana/blob/22cc0c8dbd2a1300675caf4c6d471d211ed44858/x-pack/test/detection_engine_api_integration/security_and_spaces/tests/get_rule_execution_events.ts#L121-L166) for technique. This can further be used to inject many execution events and expand tests around pagination, sorting, filters, etc. * Fixes `gap_duration`'s of `1-499`ms showing up as `-` instead of `0` * Fixes restore filters action to restore either absolute or relative datepicker as it originally was * Resolves elastic#130946 * Adds `min-height` to tab container * Removes scroll-pane from ExceptionsViewer to match Alerts/Execution Log --- ##### Remaining follow-ups: None! 🎉 ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [X] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/)) - [X] This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server)) (cherry picked from commit 683463e) # Conflicts: # x-pack/plugins/security_solution/cypress/tasks/alerts.ts # x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/details/execution_log_table/execution_log_columns.tsx # x-pack/plugins/security_solution/server/lib/detection_engine/rule_execution_log/event_log/event_log_reader.ts # x-pack/plugins/security_solution/server/lib/detection_engine/rule_execution_log/event_log/get_execution_event_aggregation/index.test.ts # x-pack/plugins/security_solution/server/lib/detection_engine/rule_execution_log/event_log/get_execution_event_aggregation/index.ts # x-pack/test/detection_engine_api_integration/utils/index_event_log_execution_events.ts

… Fixes Part Deux (#130072) (#131574) * [Security Solution][Detections] Rule Execution Log Feedback and Fixes Part Deux (#130072) ## Summary Addresses feedback and fixes identified in #126215 & #129003 ##### Feedback addressed includes: * Adds toast for restoring global query state after performing `view alerts for execution` action <img width="500" src="https://user-images.githubusercontent.com/2946766/164511565-b77d3dc8-a8b5-4927-a947-54966a58c74f.gif" /> * Updates global SuperDatePicker to daterange of execution (+/- day) for `view alerts for execution` action (and clear all other filters) * See above gif * Remove redundant `RuleExecutionStatusType` (#129003 (comment)) * Persist table state (DatePicker/StatusFilter/SortField/SortOrder/Pagination) when navigating to other tabs on the same page <img width="500" src="https://user-images.githubusercontent.com/2946766/164512498-59416601-d967-4a27-b0cc-0715cc0662c0.gif" /> * Fix duration hours bug (`7 hours (25033167ms)` as `06:417:13:000`) <img width="500" src="https://user-images.githubusercontent.com/2946766/164511478-bf0bb6d8-d8b7-4c86-8fbd-b60090f00555.png" /> * Support `disabled rule` platform error (#126215 (comment)) * Updated `getAggregateExecutionEvents` to fallback to platform status from `event.outcome` if `security_status` is empty, and also falls back to `error.message` is `security_message` is empty. This also now queries for corresponding `event.outcome` if filter is provided so that platform-only events can still be displayed when filtering. <img width="500" src="https://user-images.githubusercontent.com/2946766/164510056-1e0bce86-8360-4d46-b591-2041457e3244.png" /> * Verify StatusFilter issue #126215 (comment) * Unable to reproduce, I believe the query updates around first querying for status may've fixed this? * Provide helpful defaults for `to`/`from` and support datemath strings again (#129003 (comment)) * Created enhancement for this here: #131095 * Adds UI Unit tests for RuleExecutionLog Table * Finalize API Integration tests for gap remediation events * Test methods developed for injecting arbitrary execution events while still working with event-log RBAC. See last [API integration test](https://github.com/elastic/kibana/blob/22cc0c8dbd2a1300675caf4c6d471d211ed44858/x-pack/test/detection_engine_api_integration/security_and_spaces/tests/get_rule_execution_events.ts#L121-L166) for technique. This can further be used to inject many execution events and expand tests around pagination, sorting, filters, etc. * Fixes `gap_duration`'s of `1-499`ms showing up as `-` instead of `0` * Fixes restore filters action to restore either absolute or relative datepicker as it originally was * Resolves #130946 * Adds `min-height` to tab container * Removes scroll-pane from ExceptionsViewer to match Alerts/Execution Log --- ##### Remaining follow-ups: None! 🎉 ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [X] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/)) - [X] This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server)) (cherry picked from commit 683463e) # Conflicts: # x-pack/plugins/security_solution/cypress/tasks/alerts.ts # x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/details/execution_log_table/execution_log_columns.tsx # x-pack/plugins/security_solution/server/lib/detection_engine/rule_execution_log/event_log/event_log_reader.ts # x-pack/plugins/security_solution/server/lib/detection_engine/rule_execution_log/event_log/get_execution_event_aggregation/index.test.ts # x-pack/plugins/security_solution/server/lib/detection_engine/rule_execution_log/event_log/get_execution_event_aggregation/index.ts # x-pack/test/detection_engine_api_integration/utils/index_event_log_execution_events.ts * Fixing import

… Part Deux (elastic#130072) ## Summary Addresses feedback and fixes identified in elastic#126215 & elastic#129003 ##### Feedback addressed includes: * Adds toast for restoring global query state after performing `view alerts for execution` action <img width="500" src="https://user-images.githubusercontent.com/2946766/164511565-b77d3dc8-a8b5-4927-a947-54966a58c74f.gif" /> * Updates global SuperDatePicker to daterange of execution (+/- day) for `view alerts for execution` action (and clear all other filters) * See above gif * Remove redundant `RuleExecutionStatusType` (elastic#129003 (comment)) * Persist table state (DatePicker/StatusFilter/SortField/SortOrder/Pagination) when navigating to other tabs on the same page <img width="500" src="https://user-images.githubusercontent.com/2946766/164512498-59416601-d967-4a27-b0cc-0715cc0662c0.gif" /> * Fix duration hours bug (`7 hours (25033167ms)` as `06:417:13:000`) <img width="500" src="https://user-images.githubusercontent.com/2946766/164511478-bf0bb6d8-d8b7-4c86-8fbd-b60090f00555.png" /> * Support `disabled rule` platform error (elastic#126215 (comment)) * Updated `getAggregateExecutionEvents` to fallback to platform status from `event.outcome` if `security_status` is empty, and also falls back to `error.message` is `security_message` is empty. This also now queries for corresponding `event.outcome` if filter is provided so that platform-only events can still be displayed when filtering. <img width="500" src="https://user-images.githubusercontent.com/2946766/164510056-1e0bce86-8360-4d46-b591-2041457e3244.png" /> * Verify StatusFilter issue elastic#126215 (comment) * Unable to reproduce, I believe the query updates around first querying for status may've fixed this? * Provide helpful defaults for `to`/`from` and support datemath strings again (elastic#129003 (comment)) * Created enhancement for this here: elastic#131095 * Adds UI Unit tests for RuleExecutionLog Table * Finalize API Integration tests for gap remediation events * Test methods developed for injecting arbitrary execution events while still working with event-log RBAC. See last [API integration test](https://github.com/elastic/kibana/blob/22cc0c8dbd2a1300675caf4c6d471d211ed44858/x-pack/test/detection_engine_api_integration/security_and_spaces/tests/get_rule_execution_events.ts#L121-L166) for technique. This can further be used to inject many execution events and expand tests around pagination, sorting, filters, etc. * Fixes `gap_duration`'s of `1-499`ms showing up as `-` instead of `0` * Fixes restore filters action to restore either absolute or relative datepicker as it originally was * Resolves elastic#130946 * Adds `min-height` to tab container * Removes scroll-pane from ExceptionsViewer to match Alerts/Execution Log --- ##### Remaining follow-ups: None! 🎉 ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [X] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/)) - [X] This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))

spong added 3 commits February 22, 2022 17:03

Adds Rule Execution Log table

2bcf458

Removes total alerts created and detected metrics

c7f91ae

Merge branch 'main' of github.com:elastic/kibana into adds-rule-execu…

e100c42

…tion-log-table

spong self-assigned this Feb 23, 2022

i18n fix

8699e4d

spong mentioned this pull request Feb 24, 2022

[Epic][Security Solution][Detections] Rule Execution Log - UI on the Rule Details page #101014

Closed

2 tasks

spong added 6 commits February 24, 2022 16:16

Merge branch 'main' of github.com:elastic/kibana into adds-rule-execu…

038a2d3

…tion-log-table

Merge branch 'main' of github.com:elastic/kibana into adds-rule-execu…

817adc8

…tion-log-table

Merge branch 'main' of github.com:elastic/kibana into adds-rule-execu…

fbe882b

…tion-log-table

Merge branch 'main' of github.com:elastic/kibana into adds-rule-execu…

5cc47c1

…tion-log-table

Merge branch 'main' of github.com:elastic/kibana into adds-rule-execu…

923bdd1

…tion-log-table

Merge branch 'main' of github.com:elastic/kibana into adds-rule-execu…

7207d0f

…tion-log-table

This was referenced Mar 9, 2022

[Event Log] Add support for aggregations #125645

Closed

[Response Ops] API to retrieve execution log entries from event log. #127339

Merged

spong added 10 commits March 10, 2022 17:12

Merge branch 'main' of github.com:elastic/kibana into adds-rule-execu…

7e3f96b

…tion-log-table

Sort type fix

373ba43

Adds FR translations

f0074d7

Reworks to use event-log aggregations

0ad3bb2

Merge branch 'main' of github.com:elastic/kibana into adds-rule-execu…

2fba083

…tion-log-table

Resolving duplicate i18n keys

de5e2e4

Fixing existing tests

49e7431

Merge branch 'main' of github.com:elastic/kibana into adds-rule-execu…

8331602

…tion-log-table

Move to EuiBasictable and support pagination & sorting through API

5c5c81c

Merge branch 'main' of github.com:elastic/kibana into adds-rule-execu…

9ec38ef

…tion-log-table

Merge branch 'main' of github.com:elastic/kibana into adds-rule-execu…

9f847a3

…tion-log-table

xcrzx approved these changes Mar 28, 2022

View reviewed changes

spong merged commit 9bc4c0c into elastic:main Mar 28, 2022

spong deleted the adds-rule-execution-log-table branch March 28, 2022 22:42

spong mentioned this pull request Mar 29, 2022

[Security Solution][Detections] Fixes show alerts for execution action on Rule Execution Log #128843

Merged

spong mentioned this pull request Mar 30, 2022

[Security Solution][Detections] New Rule Execution Log UI #119599

Closed

kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Mar 30, 2022

spong added needs_docs backport:skip This commit does not require backporting and removed backport missing Added to PRs automatically when the are determined to be missing a backport. labels Mar 31, 2022

spong mentioned this pull request Mar 31, 2022

[Security Solution][Detections] Rule Execution Log Feedback and Fixes #129003

Merged

12 tasks

joepeeples mentioned this pull request Mar 31, 2022

[DOCS] Rule Execution Logs elastic/security-docs#1794

Closed

spong mentioned this pull request Apr 5, 2022

[Unified search] Surface Recently used searches UI for showing the last few search queries/filters/dateranges that the user has performed #129542

Open

spong mentioned this pull request Apr 13, 2022

[Security Solution][Detections] Rule Execution Log Feedback and Fixes Part Deux #130072

Merged

4 tasks

joepeeples mentioned this pull request Apr 22, 2022

[DOCS] What's New in 8.2? elastic/security-docs#1827

Closed

joepeeples mentioned this pull request Jun 3, 2022

[DOCS] Rule execution logs: Add filter icon in Actions column elastic/security-docs#2043

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Security Solution][Detections] Adds rule execution log table #126215

[Security Solution][Detections] Adds rule execution log table #126215

spong commented Feb 23, 2022 •

edited

Loading

xcrzx commented Mar 28, 2022

kibana-ci commented Mar 28, 2022

API count

xcrzx left a comment

xcrzx Mar 28, 2022

spong Mar 28, 2022

xcrzx Mar 28, 2022

spong Mar 28, 2022

spong commented Mar 28, 2022

spong commented Mar 28, 2022

kibanamachine commented Mar 30, 2022

[Security Solution][Detections] Adds rule execution log table #126215

[Security Solution][Detections] Adds rule execution log table #126215

Conversation

spong commented Feb 23, 2022 • edited Loading

Summary

Implementation notes

Todo:

Checklist

xcrzx commented Mar 28, 2022

kibana-ci commented Mar 28, 2022

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

Page load bundle

API count

History

xcrzx left a comment

Choose a reason for hiding this comment

xcrzx Mar 28, 2022

Choose a reason for hiding this comment

spong Mar 28, 2022

Choose a reason for hiding this comment

xcrzx Mar 28, 2022

Choose a reason for hiding this comment

spong Mar 28, 2022

Choose a reason for hiding this comment

spong commented Mar 28, 2022

spong commented Mar 28, 2022

kibanamachine commented Mar 30, 2022

spong commented Feb 23, 2022 •

edited

Loading