Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Event Log] adds query support to the Event Log #62015

Merged
merged 26 commits into from
Apr 6, 2020
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
2d3f783
added Start api on Event Log plugin
gmmorris Mar 31, 2020
0263507
added empty skeleton for Event Log FTs
gmmorris Mar 31, 2020
a0e1dcd
Merge branch 'master' into event-log/query-support
gmmorris Mar 31, 2020
e4b601d
added functional test to public find events api
gmmorris Apr 1, 2020
cbb3a14
added test for pagination
gmmorris Apr 1, 2020
7805cc1
Merge branch 'master' into event-log/query-support
gmmorris Apr 1, 2020
1a32a00
fixed unit tests
gmmorris Apr 2, 2020
3cc9ed4
Merge branch 'master' into event-log/query-support
gmmorris Apr 2, 2020
66efd01
added support for date ranges
gmmorris Apr 2, 2020
cff6041
removed unused code
gmmorris Apr 2, 2020
711c098
replaces valdiation typing
gmmorris Apr 2, 2020
3eada98
Merge branch 'master' into event-log/query-support
gmmorris Apr 2, 2020
e893e01
Revert "replaces valdiation typing"
gmmorris Apr 3, 2020
b6ac43a
replaces match with term
gmmorris Apr 3, 2020
2bbac1a
added sorting
gmmorris Apr 3, 2020
a492453
Merge branch 'master' into event-log/query-support
gmmorris Apr 3, 2020
ea9f1fc
fixed saved objects nested query
gmmorris Apr 3, 2020
2b6a895
Merge branch 'master' into event-log/query-support
elasticmachine Apr 3, 2020
e166aa2
Merge branch 'master' into event-log/query-support
elasticmachine Apr 5, 2020
da25c0b
updated plugin FTs path
gmmorris Apr 6, 2020
503bea6
Merge branch 'event-log/query-support' of github.com:gmmorris/kibana …
gmmorris Apr 6, 2020
0fc6116
Update x-pack/plugins/encrypted_saved_objects/README.md
gmmorris Apr 6, 2020
8582fca
Update x-pack/plugins/encrypted_saved_objects/README.md
gmmorris Apr 6, 2020
2aaa21f
remofed validation from tests
gmmorris Apr 6, 2020
4b0d3c2
Merge branch 'event-log/query-support' of github.com:gmmorris/kibana …
gmmorris Apr 6, 2020
8cb2ee6
fixed typos
gmmorris Apr 6, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,8 @@
"examples/*",
"test/plugin_functional/plugins/*",
"test/interpreter_functional/plugins/*",
"x-pack/test/functional_with_es_ssl/fixtures/plugins/*"
"x-pack/test/functional_with_es_ssl/fixtures/plugins/*",
"x-pack/test/plugin_api_integration/plugins/*"
],
"nohoist": [
"**/@types/*",
Expand Down
1 change: 1 addition & 0 deletions test/scripts/jenkins_xpack_build_kibana.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ echo " -> building kibana platform plugins"
node scripts/build_kibana_platform_plugins \
--scan-dir "$XPACK_DIR/test/plugin_functional/plugins" \
--scan-dir "$XPACK_DIR/test/functional_with_es_ssl/fixtures/plugins" \
--scan-dir "$XPACK_DIR/test/plugin_api_integration/plugins" \
--verbose;

# doesn't persist, also set in kibanaPipeline.groovy
Expand Down
4 changes: 2 additions & 2 deletions x-pack/plugins/encrypted_saved_objects/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,10 +100,10 @@ $ node scripts/jest.js

In one shell, from `kibana-root-folder/x-pack`:
```bash
$ node scripts/functional_tests_server.js --config test/plugin_api_integration/config.js
$ node scripts/functional_tests_server.js --config test/plugin_api_integration/config.ts
gmmorris marked this conversation as resolved.
Show resolved Hide resolved
```

In another shell, from `kibana-root-folder/x-pack`:
```bash
$ node ../scripts/functional_test_runner.js --config test/plugin_api_integration/config.js --grep="{TEST_NAME}"
$ node ../scripts/functional_test_runner.js --config test/plugin_api_integration/config.ts --grep="{TEST_NAME}"
gmmorris marked this conversation as resolved.
Show resolved Hide resolved
```
7 changes: 7 additions & 0 deletions x-pack/plugins/event_log/common/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

export const BASE_EVENT_LOG_API_PATH = '/api/event_log';
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ const createClusterClientMock = () => {
createIndexTemplate: jest.fn(),
doesAliasExist: jest.fn(),
createIndex: jest.fn(),
queryEventsBySavedObject: jest.fn(),
};
return mock;
};
Expand Down
229 changes: 229 additions & 0 deletions x-pack/plugins/event_log/server/es/cluster_client_adapter.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@
import { ClusterClient, Logger } from '../../../../../src/core/server';
import { elasticsearchServiceMock, loggingServiceMock } from '../../../../../src/core/server/mocks';
import { ClusterClientAdapter, IClusterClientAdapter } from './cluster_client_adapter';
import moment from 'moment';
import { findOptionsSchema } from '../event_log_client';

type EsClusterClient = Pick<jest.Mocked<ClusterClient>, 'callAsInternalUser' | 'asScoped'>;

Expand Down Expand Up @@ -195,3 +197,230 @@ describe('createIndex', () => {
await clusterClientAdapter.createIndex('foo');
});
});

describe('queryEventsBySavedObject', () => {
const DEFAULT_OPTIONS = findOptionsSchema.validate({});

test('should call cluster with proper arguments', async () => {
clusterClient.callAsInternalUser.mockResolvedValue({
hits: {
hits: [],
total: { value: 0 },
},
});
await clusterClientAdapter.queryEventsBySavedObject(
'index-name',
'saved-object-type',
'saved-object-id',
DEFAULT_OPTIONS
);

const [method, query] = clusterClient.callAsInternalUser.mock.calls[0];
expect(method).toEqual('search');
expect(query).toMatchObject({
index: 'index-name',
body: {
from: 0,
size: 10,
sort: { 'event.start': { order: 'asc' } },
query: {
bool: {
must: [
{
nested: {
path: 'kibana.saved_objects',
query: {
bool: {
must: [
{
term: {
'kibana.saved_objects.type': {
value: 'saved-object-type',
},
},
},
{
term: {
'kibana.saved_objects.id': {
value: 'saved-object-id',
},
},
},
],
},
},
},
},
],
},
},
},
});
});

test('should call cluster with sort', async () => {
clusterClient.callAsInternalUser.mockResolvedValue({
hits: {
hits: [],
total: { value: 0 },
},
});
await clusterClientAdapter.queryEventsBySavedObject(
'index-name',
'saved-object-type',
'saved-object-id',
{ ...DEFAULT_OPTIONS, sort_field: 'event.end', sort_order: 'desc' }
);

const [method, query] = clusterClient.callAsInternalUser.mock.calls[0];
expect(method).toEqual('search');
expect(query).toMatchObject({
index: 'index-name',
body: {
sort: { 'event.end': { order: 'desc' } },
},
});
});

test('supports open ended date', async () => {
clusterClient.callAsInternalUser.mockResolvedValue({
hits: {
hits: [],
total: { value: 0 },
},
});

const start = moment()
.subtract(1, 'days')
.toISOString();

await clusterClientAdapter.queryEventsBySavedObject(
'index-name',
'saved-object-type',
'saved-object-id',
{ ...DEFAULT_OPTIONS, start }
);

const [method, query] = clusterClient.callAsInternalUser.mock.calls[0];
expect(method).toEqual('search');
expect(query).toMatchObject({
index: 'index-name',
body: {
query: {
bool: {
must: [
{
nested: {
path: 'kibana.saved_objects',
query: {
bool: {
must: [
{
term: {
'kibana.saved_objects.type': {
value: 'saved-object-type',
},
},
},
{
term: {
'kibana.saved_objects.id': {
value: 'saved-object-id',
},
},
},
],
},
},
},
},
{
range: {
'event.start': {
gte: start,
},
},
},
],
},
},
},
});
});

test('supports optional date range', async () => {
clusterClient.callAsInternalUser.mockResolvedValue({
hits: {
hits: [],
total: { value: 0 },
},
});

const start = moment()
.subtract(1, 'days')
.toISOString();
const end = moment()
.add(1, 'days')
.toISOString();

await clusterClientAdapter.queryEventsBySavedObject(
'index-name',
'saved-object-type',
'saved-object-id',
{ ...DEFAULT_OPTIONS, start, end }
);

const [method, query] = clusterClient.callAsInternalUser.mock.calls[0];
expect(method).toEqual('search');
expect(query).toMatchObject({
index: 'index-name',
body: {
query: {
bool: {
must: [
{
nested: {
path: 'kibana.saved_objects',
query: {
bool: {
must: [
{
term: {
'kibana.saved_objects.type': {
value: 'saved-object-type',
},
},
},
{
term: {
'kibana.saved_objects.id': {
value: 'saved-object-id',
},
},
},
],
},
},
},
},
{
range: {
'event.start': {
gte: start,
},
},
},
{
range: {
'event.end': {
lte: end,
},
},
},
],
},
},
},
});
});
});
91 changes: 91 additions & 0 deletions x-pack/plugins/event_log/server/es/cluster_client_adapter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,10 @@
* you may not use this file except in compliance with the Elastic License.
*/

import { reject, isUndefined } from 'lodash';
import { Logger, ClusterClient } from '../../../../../src/core/server';
import { IEvent } from '../types';
import { FindOptionsType } from '../event_log_client';

export type EsClusterClient = Pick<ClusterClient, 'callAsInternalUser' | 'asScoped'>;
export type IClusterClientAdapter = PublicMethodsOf<ClusterClientAdapter>;
Expand All @@ -14,6 +17,13 @@ export interface ConstructorOpts {
clusterClient: EsClusterClient;
}

export interface QueryEventsBySavedObjectResult {
page: number;
per_page: number;
total: number;
data: IEvent[];
}

export class ClusterClientAdapter {
private readonly logger: Logger;
private readonly clusterClient: EsClusterClient;
Expand Down Expand Up @@ -107,6 +117,87 @@ export class ClusterClientAdapter {
}
}

public async queryEventsBySavedObject(
index: string,
type: string,
id: string,
{ page, per_page: perPage, start, end, sort_field, sort_order }: FindOptionsType
): Promise<QueryEventsBySavedObjectResult> {
try {
const {
hits: {
hits,
total: { value: total },
},
} = await this.callEs('search', {
index,
body: {
gmmorris marked this conversation as resolved.
Show resolved Hide resolved
size: perPage,
from: (page - 1) * perPage,
sort: { [sort_field]: { order: sort_order } },
query: {
bool: {
must: reject(
gmmorris marked this conversation as resolved.
Show resolved Hide resolved
[
{
nested: {
path: 'kibana.saved_objects',
query: {
bool: {
must: [
{
term: {
'kibana.saved_objects.type': {
value: type,
},
},
},
{
term: {
'kibana.saved_objects.id': {
value: id,
},
},
},
],
},
},
},
},
start && {
range: {
'event.start': {
gte: start,
},
},
},
end && {
range: {
'event.end': {
lte: end,
},
},
},
],
isUndefined
),
},
},
},
});
return {
page,
per_page: perPage,
total,
data: hits.map((hit: any) => hit._source) as IEvent[],
};
} catch (err) {
throw new Error(
`querying for Event Log by for type "${type}" and id "${id}" failed with: ${err.message}`
);
}
}

private async callEs(operation: string, body?: any): Promise<any> {
try {
this.debug(`callEs(${operation}) calls:`, body);
Expand Down
Loading