[TESTING][Detection Engine] Testing Prebuilt Security Rules Package - DO NOT MERGE

[terrancedejesus]

Related Issue

• https://github.com/elastic/ia-trade-team/issues/136
• [Security Solution] Intercept individual package installation via Fleet #161859

Summary

This PR is used to use the Kibana CI testing for the security solution Cypress tests. This PR is NOT MEANT to be merged. Once tests are successful, this PR will be closed.

Prerelease packages:
v8.7.9-beta.1
v8.8.7-beta.1
v8.9.2-beta.1

[terrancedejesus]

8.7.9-beta.1 package

This package failed some UI testing, but the rules load correctly by manual testing. The package installs correctly, rules updates load correctly and there seem to be no UI issues.

It seems that the prebuilt rules package is not installing correctly in the Cypress tests: https://buildkite.com/elastic/kibana-pull-request/builds/141978#01894afe-a70f-40fa-bed0-9466c841cd19/6-4226

@spong - Do you have any additional information on why this might be happening?

I am curious if this is because new workflows exist in main, that do not exist in 8.7 where this package would be installed? Therefore when running cypress tests I need to point to the matching branch?

[kibana-ci]

💔 Build Failed

• Buildkite Build
• Commit: 5e259ae
• Interpreting CI Failures

Failed CI Steps

• Security Solution Tests #1
• Security Solution Tests #1
• Security Solution Tests #2
• Security Solution Tests #2
• Security Solution Tests #6

Test Failures

• [job] [logs] Security Solution Tests #1 / Detection rules, Prebuilt Rules Installation and Update workflow Installation of prebuilt rules package via Fleet should install package from Fleet in the background
• [job] [logs] Security Solution Tests #1 / Detection rules, Prebuilt Rules Installation and Update workflow Installation of prebuilt rules package via Fleet should install package from Fleet in the background
• [job] [logs] Security Solution Tests #1 / Detection rules, Prebuilt Rules Installation and Update workflow Installation of prebuilt rules should install selected rules when user clicks on Install selected rules
• [job] [logs] Security Solution Tests #6 / Export rules exports only custom rules
• [job] [logs] Security Solution Tests #2 / Prebuilt rules Actions with prebuilt rules Rules table Deletes and recovers more than one rule
• [job] [logs] Security Solution Tests #2 / Prebuilt rules Actions with prebuilt rules Rules table Deletes and recovers more than one rule
• [job] [logs] Security Solution Tests #2 / Prebuilt rules Actions with prebuilt rules Rules table Deletes and recovers one rule
• [job] [logs] Security Solution Tests #2 / Prebuilt rules Actions with prebuilt rules Rules table Deletes and recovers one rule

Metrics [docs]

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
enterpriseSearch	14	16	+2
securitySolution	409	413	+4
total			+6

Total ESLint disabled count

id	before	after	diff
enterpriseSearch	15	17	+2
securitySolution	488	492	+4
total			+6

History

• 💔 Build #141978 failed 4fb860a

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[terrancedejesus]

[job] [logs] Security Solution Tests #1 / Detection rules, Prebuilt Rules Installation and Update workflow Installation of prebuilt rules should install selected rules when user clicks on Install selected rules

8.9.0 stack loaded 8.9.1 package. Then force installed 8.7.9-beta.1 with rule updates. Selected rules and selected Install rule. Rules installed fine.

[terrancedejesus]

Security Solution Tests #6 / Export rules exports only custom rules

I believe this is expected as prebuilt rules should not be exported.

[terrancedejesus]

Security Solution Tests #1 / Detection rules, Prebuilt Rules Installation and Update workflow Installation of prebuilt rules package via Fleet should install package from Fleet in the background

This failure is concerning as it seems to indicate that the package is not being installed via Fleet. Test is from x-pack/plugins/security_solution/cypress/e2e/detection_rules/prebuilt_rules_install_update_workflows.cy.ts It is difficult to re-create this with a beta package as we have to force install beta packages rather than rely on Fleet to auto-pull these in.

  describe('Installation of prebuilt rules package via Fleet', () => {
    beforeEach(() => {
      cy.intercept('POST', '/api/fleet/epm/packages/_bulk*').as('installPackage');
      waitForRulesTableToBeLoaded();
    });

    it('should install package from Fleet in the background', () => {
      /* Assert that the package in installed from Fleet by checking that
      /* the installSource is "registry", as opposed to "bundle" */
      cy.wait('@installPackage', {
        timeout: 60000,
      }).then(({ response }) => {
        cy.wrap(response?.statusCode).should('eql', 200);

        const packages = response?.body.items.map(({ name, result }: BulkInstallPackageInfo) => ({
          name,
          installSource: result.installSource,
        }));

        expect(packages.length).to.have.greaterThan(0);
        expect(packages).to.deep.include.members([
          { name: 'security_detection_engine', installSource: 'registry' },
        ]);
      });
    });

[terrancedejesus]

Suggested change

	`--xpack.securitySolution.prebuiltRulesPackageVersion=8.7.9-beta.1`,
	`--xpack.securitySolution.prebuiltRulesPackageVersion=8.8.7-beta.1`,

[terrancedejesus]

Closing, packages have all been released.