Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SIEM] Run Cypress Tests Against Elastic Cloud & Cypress Command Line / Reporting #42804

Commits on Aug 14, 2019

  1. # [SIEM] Run Cypress Tests Against Elastic Cloud & Cypress Command Li…

    …ne / Reporting
    
    This PR contains enhancements to the `Cypress` end-to-end (e2e) testing support in the SIEM app, as discussed with @spalger
    
    Note: this PR depends on test refactorings in elastic#42766
    
    ## New: Run `Cypress` Tests Against Elastic Cloud
    
    Cypress tests may now be run against a remote Elastic Cloud instance
    (override `baseUrl`), interactively or via the command line.
    
    Credentials are specified via `kibna.dev.yml` or environment variables.
    
    ## New: Run `Cypress` Tests On the Command Line
    
    Run tests on the command line (override `baseUrl` and pass credentials via
    the `CYPRESS_ELASTICSEARCH_USERNAME` and `CYPRESS_ELASTICSEARCH_PASSWORD`
    environment variables), via command line.
    
    ## New: Reporting
    
    Reports are configured to include:
    
    - An HTML report, suitable for e-mail notifications
    - Screenshots
    - Junit reports (for integration with Kibana CI)
    - Videos (optional)
    
    Reports generated when `Cypress` tests are run on the command line are output
    to the `target` directory, which follows specific conventions used by the
    Kibana CI process:
    
    * Any directory under `target` that begins with `kibana-`, e.g. `kibana-siem` will be uploaded as an artifact to a bucket
    * Junit reports are picked up from the `target/junit` directory
    
    The artifacts generated by running `Cypress` tests on the command line conform to the above conventions.
    
    ### HTML Reports
    
    An HTML report (e.g. for email notifications) is output to:
    
    ```
    target/kibana-siem/cypress/results/output.html
    ```
    
    ### Screenshots
    
    Screenshots of failed tests are output to:
    
    ```
    target/kibana-siem/cypress/screenshots
    ```
    
    ### `junit` Reports
    
    The Kibana CI process reports `junit` test results from the `target/junit` directory.
    
    Cypress `junit` reports are generated in `target/kibana-siem/cypress/results`
    and copied to the `target/junit` directory.
    
    ### Videos (optional)
    
    Videos are disabled by default, but can optionally be enabled by setting the
    `CYPRESS_video=true` environment variable:
    
    ```
    CYPRESS_video=true yarn cypress:run
    ```
    
    Videos are (optionally) output to:
    
    ```
    target/kibana-siem/cypress/videos
    ```
    
    # Updated `README`
    
    The full details and instructions for running Cypress tests are in the
    `siem/cypress/README.md`, which was updated to reflect the changes
    in this in this PR, but in a nutshell, you may run tests interactively with
    the following (new) environment variables:
    
    ```sh
    cd x-pack/legacy/plugins/siem
    CYPRESS_baseUrl=http://localhost:5601 CYPRESS_ELASTICSEARCH_USERNAME=elastic CYPRESS_ELASTICSEARCH_PASSWORD=<password> yarn cypress:open
    ```
    
    Running the command line version of the tests, which will output
    the reports described above, is a similar experence:
    
    ```sh
    cd x-pack/legacy/plugins/siem
    CYPRESS_baseUrl=http://localhost:5601 CYPRESS_ELASTICSEARCH_USERNAME=elastic CYPRESS_ELASTICSEARCH_PASSWORD=<password> yarn cypress:run
    ```
    
    elastic/siem-team#435
    elastic/siem-team#437
    andrew-goldstein committed Aug 14, 2019
    Configuration menu
    Copy the full SHA
    79c5b97 View commit details
    Browse the repository at this point in the history