[ML] Explain Log Rate Spikes: Fix error handling. #142047

walterra · 2022-09-28T08:50:58Z

Summary

Part of #138117.

Fixes error handling that before was not providing enough information for debugging purposes and support. This will now output more fine grained error information to the Kibana server log. The analysis is now more resilient to errors for individual queries. For example, we don't stop the analysis anymore if individual queries for p-values or histograms fail.
Moves the error callout above all other possible elements like empty prompts when the analysis doesn't return results.

Checklist

Unit or functional tests were updated or
This was checked for breaking API changes and was labeled appropriately

elasticmachine · 2022-09-29T16:29:27Z

Pinging @elastic/ml-ui (:ml)

...plugins/aiops/public/components/explain_log_rate_spikes/explain_log_rate_spikes_analysis.tsx

x-pack/plugins/aiops/server/routes/explain_log_rate_spikes.ts

qn895 · 2022-09-30T14:58:58Z

x-pack/plugins/aiops/server/routes/explain_log_rate_spikes.ts

+            logger.error(
+              `Failed to transform field/value pairs into groups, got: \n${e.toString()}`
+            );
+            pushError(`Failed to transform field/value pairs  into groups.`);


typo: there's an extra space here after 'pairs'

Fixed in c9eed74.

x-pack/plugins/aiops/server/routes/explain_log_rate_spikes.ts

qn895 · 2022-09-30T15:01:10Z

x-pack/plugins/aiops/server/routes/explain_log_rate_spikes.ts

+        } catch (e) {
+          logger.error(`Failed to fetch the overall histogram data, got: \n${e.toString()}`);
+          pushError(`Failed to fetch overall histogram data.`);
+          // Still continue the analysis even if loading the overall histogram failes.


typo: failes -> fails + suggestion to internationalize this message

Fixed typo in c9eed74. i18n will be done in a follow up.

walterra · 2022-10-03T07:39:01Z

@qn895 Thanks for the review, this is ready for another look!

qn895 · 2022-10-03T17:36:54Z

x-pack/plugins/aiops/common/api/stream_reducer.ts

@@ -35,6 +35,9 @@ export function streamReducer(
    return action.reduce(streamReducer, state);
  }

+  // eslint-disable-next-line no-console
+  console.log('action', action);


I'm assuming this console.log is here intentionally... In which case might be good to have a better log prefix message. But this is a super duper nit and is not an issue. Please feel free to merge 😄 .

Sorry for not mentioning this, it's just for some temporary cloud testing and will be deleted before merging!

Removed in 2bc9fce.

qn895 · 2022-10-03T17:37:24Z

Code LGTM 🎉

peteharverson

LGTM

kibana-ci · 2022-10-04T11:49:24Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 6c7fb9b

Failed CI Steps

FTR Configs #14

Test Failures

[job] [logs] FTR Configs #14 / alerting api integration spaces only Alerting runSoon should successfully run rule where scheduled task id is same as rule id

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`aiops`	765.9KB	765.9KB	+4.0B

History

💔 Build #77835 failed 1d1c714
💚 Build #77579 succeeded f0c58fb
💚 Build #77527 succeeded 19ce5ef
💔 Build #77514 failed bf0ac44
💛 Build #77397 was flaky 739ef4a
💚 Build #76906 succeeded c1ff302

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @walterra

- Fixes error handling that before was not providing enough information for debugging purposes and support. This will now output more fine grained error information to the Kibana server log. The analysis is now more resilient to errors for individual queries. For example, we don't stop the analysis anymore if individual queries for p-values or histograms fail. - Moves the error callout above all other possible elements like empty prompts when the analysis doesn't return results. (cherry picked from commit 4753d7c)

kibanamachine · 2022-10-04T11:54:46Z

💚 All backports created successfully

Status	Branch	Result
✅	8.5

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

- Fixes error handling that before was not providing enough information for debugging purposes and support. This will now output more fine grained error information to the Kibana server log. The analysis is now more resilient to errors for individual queries. For example, we don't stop the analysis anymore if individual queries for p-values or histograms fail. - Moves the error callout above all other possible elements like empty prompts when the analysis doesn't return results. (cherry picked from commit 4753d7c) Co-authored-by: Walter Rafelsberger <walter.rafelsberger@elastic.co>

- Fixes error handling that before was not providing enough information for debugging purposes and support. This will now output more fine grained error information to the Kibana server log. The analysis is now more resilient to errors for individual queries. For example, we don't stop the analysis anymore if individual queries for p-values or histograms fail. - Moves the error callout above all other possible elements like empty prompts when the analysis doesn't return results.

walterra added bug Fixes for quality problems that affect the customer experience release_note:fix ci:build-cloud-image labels Sep 28, 2022

walterra self-assigned this Sep 28, 2022

walterra mentioned this pull request Sep 28, 2022

[ML] Explain log rate spikes: MVP 8.5 improvements. #138117

Closed

9 tasks

walterra added v8.5.0 v8.6.0 :ml Feature:ML/AIOps ML AIOps features: Change Point Detection, Log Pattern Analysis, Log Rate Analysis labels Sep 29, 2022

walterra added 2 commits September 29, 2022 16:54

Fix error handling for p-value aggregation.

123612c

Improve error handling.

3f7b3b6

walterra force-pushed the ml-aiops-fix-error-handling-2 branch from d40f40d to 3f7b3b6 Compare September 29, 2022 14:55

Tweak frequent_items agg settings.

c1ff302

walterra requested review from peteharverson and qn895 September 29, 2022 16:28

walterra marked this pull request as ready for review September 29, 2022 16:29

walterra requested a review from a team as a code owner September 29, 2022 16:29

qn895 reviewed Sep 30, 2022

View reviewed changes

...plugins/aiops/public/components/explain_log_rate_spikes/explain_log_rate_spikes_analysis.tsx Outdated Show resolved Hide resolved

qn895 reviewed Sep 30, 2022

View reviewed changes

x-pack/plugins/aiops/server/routes/explain_log_rate_spikes.ts Show resolved Hide resolved

qn895 reviewed Sep 30, 2022

View reviewed changes

x-pack/plugins/aiops/server/routes/explain_log_rate_spikes.ts Show resolved Hide resolved

qn895 reviewed Sep 30, 2022

View reviewed changes

walterra added 3 commits October 3, 2022 09:26

Merge branch 'main' into ml-aiops-fix-error-handling-2

ff8d5de

Fix typos.

c9eed74

Fix tsx structure.

1bef5de

walterra added 3 commits October 3, 2022 10:32

Merge branch 'main' into ml-aiops-fix-error-handling-2

739ef4a

Ping action.

bf0ac44

Adds logger.info messages.

19ce5ef

Extends logger.info messages.

f0c58fb

qn895 approved these changes Oct 3, 2022

View reviewed changes

qn895 reviewed Oct 3, 2022

View reviewed changes

peteharverson approved these changes Oct 4, 2022

View reviewed changes

walterra added 2 commits October 4, 2022 11:23

Remove console.log.

2bc9fce

Merge branch 'main' into ml-aiops-fix-error-handling-2

1d1c714

walterra mentioned this pull request Oct 4, 2022

[ML] Explain Log Rate Spikes: Errors with long running streams. #142568

Closed

9 tasks

Merge branch 'main' into ml-aiops-fix-error-handling-2

6c7fb9b

walterra merged commit 4753d7c into elastic:main Oct 4, 2022

kibanamachine mentioned this pull request Oct 4, 2022

[8.5] [ML] Explain Log Rate Spikes: Fix error handling. (#142047) #142588

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ML] Explain Log Rate Spikes: Fix error handling. #142047

[ML] Explain Log Rate Spikes: Fix error handling. #142047

walterra commented Sep 28, 2022 •

edited by kibanamachine

Loading

elasticmachine commented Sep 29, 2022

qn895 Sep 30, 2022

walterra Oct 3, 2022

qn895 Sep 30, 2022

walterra Oct 3, 2022

walterra commented Oct 3, 2022

qn895 Oct 3, 2022

walterra Oct 4, 2022

walterra Oct 4, 2022

qn895 commented Oct 3, 2022

peteharverson left a comment

kibana-ci commented Oct 4, 2022

kibanamachine commented Oct 4, 2022

[ML] Explain Log Rate Spikes: Fix error handling. #142047

[ML] Explain Log Rate Spikes: Fix error handling. #142047

Conversation

walterra commented Sep 28, 2022 • edited by kibanamachine Loading

Summary

Checklist

elasticmachine commented Sep 29, 2022

qn895 Sep 30, 2022

Choose a reason for hiding this comment

walterra Oct 3, 2022

Choose a reason for hiding this comment

qn895 Sep 30, 2022

Choose a reason for hiding this comment

walterra Oct 3, 2022

Choose a reason for hiding this comment

walterra commented Oct 3, 2022

qn895 Oct 3, 2022

Choose a reason for hiding this comment

walterra Oct 4, 2022

Choose a reason for hiding this comment

walterra Oct 4, 2022

Choose a reason for hiding this comment

qn895 commented Oct 3, 2022

peteharverson left a comment

Choose a reason for hiding this comment

kibana-ci commented Oct 4, 2022

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Async chunks

History

kibanamachine commented Oct 4, 2022

💚 All backports created successfully

Questions ?

walterra commented Sep 28, 2022 •

edited by kibanamachine

Loading