Skip to content

Commit

Permalink
[SIEM][Detection Engine] Reduces flakiness within the tests using wai…
Browse files Browse the repository at this point in the history
…tFor() and un-skips the tests (#72479) (#72508)

## Summary

* Utilizes the `waitFor` so that the tests are less flaky and more resilient
* Unskips a test that was due to a regression within Elastic Search that should be fixed now
* #71867
* #71814
* #71612

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios
  • Loading branch information
FrankHassanabad authored Jul 20, 2020
1 parent a9b3777 commit 6130e8b
Show file tree
Hide file tree
Showing 3 changed files with 25 additions and 6 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,15 @@ import {
deleteAllAlerts,
deleteAllTimelines,
deleteSignalsIndex,
waitFor,
} from '../../utils';

// eslint-disable-next-line import/no-default-export
export default ({ getService }: FtrProviderContext): void => {
const supertest = getService('supertest');
const es = getService('es');

// FLAKY: https://github.com/elastic/kibana/issues/71867
describe.skip('add_prepackaged_rules', () => {
describe('add_prepackaged_rules', () => {
describe('validation errors', () => {
it('should give an error that the index must exist first if it does not exist before adding prepackaged rules', async () => {
const { body } = await supertest
Expand Down Expand Up @@ -91,6 +91,16 @@ export default ({ getService }: FtrProviderContext): void => {
.send()
.expect(200);

// NOTE: I call the GET call until eventually it becomes consistent and that the number of rules to install are zero.
// This is to reduce flakiness where it can for a short period of time try to install the same rule twice.
await waitFor(async () => {
const { body } = await supertest
.get(`${DETECTION_ENGINE_PREPACKAGED_URL}/_status`)
.set('kbn-xsrf', 'true')
.expect(200);
return body.rules_not_installed === 0;
});

const { body } = await supertest
.put(DETECTION_ENGINE_PREPACKAGED_URL)
.set('kbn-xsrf', 'true')
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,15 @@ import {
deleteAllAlerts,
deleteAllTimelines,
deleteSignalsIndex,
waitFor,
} from '../../utils';

// eslint-disable-next-line import/no-default-export
export default ({ getService }: FtrProviderContext): void => {
const supertest = getService('supertest');
const es = getService('es');

// FLAKY: https://github.com/elastic/kibana/issues/71814
describe.skip('add_prepackaged_rules', () => {
describe('add_prepackaged_rules', () => {
describe('validation errors', () => {
it('should give an error that the index must exist first if it does not exist before adding prepackaged rules', async () => {
const { body } = await supertest
Expand Down Expand Up @@ -91,6 +91,16 @@ export default ({ getService }: FtrProviderContext): void => {
.send()
.expect(200);

// NOTE: I call the GET call until eventually it becomes consistent and that the number of rules to install are zero.
// This is to reduce flakiness where it can for a short period of time try to install the same rule the same rule twice.
await waitFor(async () => {
const { body } = await supertest
.get(`${DETECTION_ENGINE_PREPACKAGED_URL}/_status`)
.set('kbn-xsrf', 'true')
.expect(200);
return body.rules_not_installed === 0;
});

const { body } = await supertest
.put(DETECTION_ENGINE_PREPACKAGED_URL)
.set('kbn-xsrf', 'true')
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,7 @@ export default ({ getService }: FtrProviderContext): void => {
const supertest = getService('supertest');
const es = getService('es');

// Failing ES promotion: https://github.com/elastic/kibana/issues/71612
describe.skip('create_rules_bulk', () => {
describe('create_rules_bulk', () => {
describe('validation errors', () => {
it('should give a 200 even if the index does not exist as all bulks return a 200 but have an error of 409 bad request in the body', async () => {
const { body } = await supertest
Expand Down

0 comments on commit 6130e8b

Please sign in to comment.