Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SIEM][Detection Engine] Reduces flakiness within the tests using waitFor() and un-skips the tests #72479

Merged
merged 3 commits into from
Jul 20, 2020
Merged

[SIEM][Detection Engine] Reduces flakiness within the tests using waitFor() and un-skips the tests #72479

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
18e76dc
Unskips tests by adding additional checks that should reduce flakines…
FrankHassanabad Jul 20, 2020
709b61b
Updates the comments
FrankHassanabad Jul 20, 2020
3dbc682
Merge branch 'master' into unskip-tests
FrankHassanabad Jul 20, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions x-pack/test/detection_engine_api_integration/basic/tests/add_prepackaged_rules.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,15 @@ import {
deleteAllAlerts,
deleteAllTimelines,
deleteSignalsIndex,
waitFor,
} from '../../utils';

// eslint-disable-next-line import/no-default-export
export default ({ getService }: FtrProviderContext): void => {
const supertest = getService('supertest');
const es = getService('es');

// FLAKY: https://github.com/elastic/kibana/issues/71867
describe.skip('add_prepackaged_rules', () => {
describe('add_prepackaged_rules', () => {
describe('validation errors', () => {
it('should give an error that the index must exist first if it does not exist before adding prepackaged rules', async () => {
const { body } = await supertest
Expand Down Expand Up @@ -91,6 +91,16 @@ export default ({ getService }: FtrProviderContext): void => {
.send()
.expect(200);

// NOTE: I call the GET call until eventually it becomes consistent and that the number of rules to install are zero.
// This is to reduce flakiness where it can for a short period of time try to install the same rule twice.
await waitFor(async () => {
const { body } = await supertest
.get(`${DETECTION_ENGINE_PREPACKAGED_URL}/_status`)
.set('kbn-xsrf', 'true')
.expect(200);
return body.rules_not_installed === 0;
});

const { body } = await supertest
.put(DETECTION_ENGINE_PREPACKAGED_URL)
.set('kbn-xsrf', 'true')
Expand Down
14 changes: 12 additions & 2 deletions .../test/detection_engine_api_integration/security_and_spaces/tests/add_prepackaged_rules.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,15 @@ import {
deleteAllAlerts,
deleteAllTimelines,
deleteSignalsIndex,
waitFor,
} from '../../utils';

// eslint-disable-next-line import/no-default-export
export default ({ getService }: FtrProviderContext): void => {
const supertest = getService('supertest');
const es = getService('es');

// FLAKY: https://github.com/elastic/kibana/issues/71814
describe.skip('add_prepackaged_rules', () => {
describe('add_prepackaged_rules', () => {
describe('validation errors', () => {
it('should give an error that the index must exist first if it does not exist before adding prepackaged rules', async () => {
const { body } = await supertest
Expand Down Expand Up @@ -91,6 +91,16 @@ export default ({ getService }: FtrProviderContext): void => {
.send()
.expect(200);

// NOTE: I call the GET call until eventually it becomes consistent and that the number of rules to install are zero.
// This is to reduce flakiness where it can for a short period of time try to install the same rule the same rule twice.
await waitFor(async () => {
const { body } = await supertest
.get(`${DETECTION_ENGINE_PREPACKAGED_URL}/_status`)
.set('kbn-xsrf', 'true')
.expect(200);
return body.rules_not_installed === 0;
});

const { body } = await supertest
.put(DETECTION_ENGINE_PREPACKAGED_URL)
.set('kbn-xsrf', 'true')
Expand Down
3 changes: 1 addition & 2 deletions x-pack/test/detection_engine_api_integration/security_and_spaces/tests/create_rules_bulk.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,7 @@ export default ({ getService }: FtrProviderContext): void => {
const supertest = getService('supertest');
const es = getService('es');

// Failing ES promotion: https://github.com/elastic/kibana/issues/71612
describe.skip('create_rules_bulk', () => {
describe('create_rules_bulk', () => {
describe('validation errors', () => {
it('should give a 200 even if the index does not exist as all bulks return a 200 but have an error of 409 bad request in the body', async () => {
const { body } = await supertest
Expand Down