[Logs UI] Logs overview queries for the observability dashboard

[afgomez]

Summary

This PR provides log overview data for the observability dashboard.

Checklist

Delete any items that are not applicable to this PR.

• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios

[jasonrhodes]

Looks like I'm getting an error that I can't quite track down. The error seems to be a vague one related to the esSearcher observable:

VM2611:1 POST http://localhost:5601/internal/search/es 400 (Bad Request)

logs_overview_fetchers.ts:44 Error: Bad Request
    at Fetch.fetchResponse (fetch.ts:162)
    at async interceptResponse (intercept.ts:58)
    at async http:/localhost:5601/9007199254740991/bundles/core/core.entry.js:50572

Is the esSearcher API documented anywhere or are you just piecing it together reading code?

I trigger this with the following patch:

diff --git a/x-pack/plugins/observability/public/application/index.tsx b/x-pack/plugins/observability/public/application/index.tsx
index 21a9fabf445..7e9e3bc6358 100644
--- a/x-pack/plugins/observability/public/application/index.tsx
+++ b/x-pack/plugins/observability/public/application/index.tsx
@@ -9,10 +9,28 @@ import { EuiThemeProvider } from '../../../../legacy/common/eui_styled_component
 import { AppMountParameters, CoreStart } from '../../../../../src/core/public';
 import { Home } from '../pages/home';
 import { PluginContext } from '../context/plugin_context';
+import { getDataHandler } from '../data_handler';
 
 export const renderApp = (core: CoreStart, { element }: AppMountParameters) => {
   const i18nCore = core.i18n;
   const isDarkMode = core.uiSettings.get('theme:darkMode');
+
+  const getData = async () => {
+    const now = new Date();
+    const params = {
+      startTime: '2020-06-29T11:38:23.747Z',
+      endTime: now.toString(),
+      bucketSize: '30s',
+    };
+    const data = await getDataHandler('infra_logs')?.fetchData(params);
+    console.log('logs data', { data });
+  };
+
+  getData().catch((e) => {
+    console.log('an error occurred while getting obs data');
+    console.log(e);
+  });
+
   ReactDOM.render(
     <PluginContext.Provider value={{ core }}>
       <EuiThemeProvider darkMode={isDarkMode}>

and then navigating to /app/observability

[cauemarcondes]

@afgomez as we discussed today can you return the logs rate per minute in the series, and the average in the stats, please?

I've done the same on APM: https://github.com/elastic/kibana/blob/master/x-pack/plugins/apm/server/lib/observability_dashboard/get_transaction_coordinates.ts#L50

[afgomez]

@jasonrhodes seems an issue with how the date is passed. The startTime is in ISO string, but the endTime is in epoch_millis. I'm not sure ES lets you mix and match (therefore the 400 error).

[jasonrhodes]

@jasonrhodes seems an issue with how the date is passed. The startTime is in ISO string, but the endTime is in epoch_millis. I'm not sure ES lets you mix and match (therefore the 400 error).

Oh I did .toString() so it's not millis but that's not the right string method, I need toISOString() 🤦

[jasonrhodes]

Looks great, nice work.

[jasonrhodes]

@afgomez if you look at the tests here that Chris added for metrics, could we duplicate those for some logs tests for our fetcher methods? #70529

We could merge this and do it immediately in another PR to unblock this for Caue if you want, but I'd like to get tests in ASAP.

[afgomez]

@jasonrhodes I was planning to :)

[jasonrhodes]

@jasonrhodes I was planning to :)

Yes but now I get credit for being the one who suggested it. #leadering

[cauemarcondes]

Observability registration LGTM.

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: c2d130c

Build metrics

@kbn/optimizer bundle module count

id	value	diff	baseline
infra	606	+3	603

History

• 💔 Build #58648 failed 929afde
• 💚 Build #58017 succeeded 0dfddfd
• 💚 Build #57798 succeeded 92607df

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[afgomez]

Merging now. There's a test missing that I will add in a separate PR.