Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add AD cypress tests for uncovered workflows and pages #95

Merged
merged 13 commits into from
Mar 7, 2022

Conversation

ohltyler
Copy link
Member

@ohltyler ohltyler commented Feb 28, 2022

Description

This PR adds a significant amount of tests for the Anomaly Detection Dashboards plugin. Main changes include:

  • adding tests for:
    • create detector workflow
    • sample detector workflow (creating/deleting)
    • real-time results
    • historical analysis
    • detector configuration page
    • overview page
    • additional filtering tests for dashboard and detector list pages
  • adding global utility functions including getElementByTestId(), and AD-specific helpers
  • cleaning up existing flaky tests
  • cleaning up existing tests by relying more on AD-controlled data-test-subj tags, instead of internal EUI tags
  • cleaning up AD-specific constants
  • cleaning up unused or outdated fixtures
  • adding test results to ingest to the test cluster

Related PR which adds the test IDs used in these tests: link

Check List

  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@ohltyler ohltyler requested a review from a team as a code owner February 28, 2022 17:56
@ohltyler
Copy link
Member Author

@amitgalitz @ylwu @kaituo this adds a lot of test cases for AD Dashboards - if some of you have time to take a look that would be helpful. Details in the PR description

@ohltyler ohltyler changed the title Add AD cypress tests for remaining workflows and pages Add AD cypress tests for uncovered workflows and pages Feb 28, 2022
@ohltyler ohltyler added the 1.3 1.3 release label Feb 28, 2022
@ohltyler
Copy link
Member Author

Tests failing until related AD PR is merged first: opensearch-project/anomaly-detection-dashboards-plugin#183

@tianleh
Copy link
Member

tianleh commented Mar 1, 2022

Tests failing until related AD PR is merged first: opensearch-project/anomaly-detection-dashboards-plugin#183

Can we get some reviewers to approve your related PR? @ohltyler

@ohltyler
Copy link
Member Author

ohltyler commented Mar 1, 2022

Tests failing until related AD PR is merged first: opensearch-project/anomaly-detection-dashboards-plugin#183

Can we get some reviewers to approve your related PR? @ohltyler

I'm trying :)

@ohltyler
Copy link
Member Author

ohltyler commented Mar 1, 2022

AD PR merged, re-running tests now @tianleh

@ohltyler
Copy link
Member Author

ohltyler commented Mar 2, 2022

@tianleh @Tengda-He the CI workflow is using 1.2.0 bundle so these tests will fail, since it's dependent on AD plugin changes specific to 1.3.0. Lmk how you want to proceed.

I think we can either start versioning and bump to pull in a 1.3.0 candidate tarball, or consider the failures acceptable until main itself bumps to 1.3.0

@tianleh
Copy link
Member

tianleh commented Mar 2, 2022

@tianleh @Tengda-He the CI workflow is using 1.2.0 bundle so these tests will fail, since it's dependent on AD plugin changes specific to 1.3.0. Lmk how you want to proceed.

I think we can either start versioning and bump to pull in a 1.3.0 candidate tarball, or consider the failures acceptable until main itself bumps to 1.3.0

cc @seraphjiang for visibility

@tianleh
Copy link
Member

tianleh commented Mar 3, 2022

@ohltyler Extended our previous conversation to a RFC #45

@tianleh
Copy link
Member

tianleh commented Mar 4, 2022

We have upgraded the main branch to 1.3.0 #104 There are some known failures where we are working with owners to fix them. #104 (comment)

By syncing the latest main branch into your PR, you will be able to see your test results against latest 1.3.0 and address any failures related to your change in this PR.

@amitgalitz amitgalitz mentioned this pull request Mar 4, 2022
1 task
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
@ohltyler
Copy link
Member Author

ohltyler commented Mar 7, 2022

Rebased with main and all AD tests are passing:

      Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✔  plugins/anomaly-detection-dashboard      00:22        1        1        -        -        - │
  │    s-plugin/create_detector_spec.js                                                            │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  plugins/anomaly-detection-dashboard      00:24        8        8        -        -        - │
  │    s-plugin/dashboard_spec.js                                                                  │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  plugins/anomaly-detection-dashboard      00:27        5        5        -        -        - │
  │    s-plugin/detector_configuration_spe                                                         │
  │    c.js                                                                                        │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  plugins/anomaly-detection-dashboard      00:31       11       11        -        -        - │
  │    s-plugin/detector_list_spec.js                                                              │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  plugins/anomaly-detection-dashboard      01:20        6        6        -        -        - │
  │    s-plugin/historical_analysis_spec.j                                                         │
  │    s                                                                                           │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  plugins/anomaly-detection-dashboard      00:21        6        6        -        -        - │
  │    s-plugin/overview_spec.js                                                                   │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  plugins/anomaly-detection-dashboard      00:20        3        3        -        -        - │
  │    s-plugin/real_time_results_spec.js                                                          │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  plugins/anomaly-detection-dashboard      00:36        3        3        -        -        - │
  │    s-plugin/sample_detector_spec.js                                                            │
 

cc @tianleh @Tengda-He

Copy link
Member

@tianleh tianleh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! Looks like that this issue to have coverage report is more needed as more plugins are on this repo. #88

@tianleh tianleh merged commit 25afb70 into opensearch-project:main Mar 7, 2022
@ohltyler ohltyler deleted the add-ad-it branch March 7, 2022 18:27
@ohltyler ohltyler mentioned this pull request Mar 15, 2022
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1.3 1.3 release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants