Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Runs GitHub workflow unit tests in band #1306

Merged
merged 1 commit into from
Mar 4, 2022

Conversation

tmarkley
Copy link
Contributor

@tmarkley tmarkley commented Mar 2, 2022

Description

  • Removes caching from GitHub test workflow to simplify.
  • Removes "bad apples" check from GitHub workflow since those tests now pass consistently.
  • Runs unit tests in band to reduce execution time from ~90 minutes to ~20 minutes.
  • Removes the logic that ignored failing unit tests.
  • Adds mocha tests to the workflow.
  • Adds ci-specific scripts to simplify the commands.
    • yarn test:jest:ci and yarn test:jest_integration:ci are now used in the workflow.

Issues Resolved

Resolves #1231

Check List

  • New functionality includes testing.
    • All tests pass
      • yarn test:jest
      • yarn test:jest_integration
      • yarn test:ftr
  • New functionality has been documented.
  • Commits are signed per the DCO using --signoff

@tmarkley tmarkley requested a review from a team as a code owner March 2, 2022 17:51
@tmarkley tmarkley requested review from ananzh and kavilla March 2, 2022 17:57
@tmarkley
Copy link
Contributor Author

tmarkley commented Mar 2, 2022

@tmarkley tmarkley requested a review from ashwin-pc March 2, 2022 18:07
@kavilla
Copy link
Member

kavilla commented Mar 2, 2022

@kavilla I'm testing this for 1.x in my forked repo: https://github.com/tmarkley/OpenSearch-Dashboards/runs/5396077731?check_suite_focus=true

Looks like we need the jest upgrade to be able to use the runInBand.

@tmarkley
Copy link
Contributor Author

tmarkley commented Mar 2, 2022

@kavilla I'm testing this for 1.x in my forked repo: https://github.com/tmarkley/OpenSearch-Dashboards/runs/5396077731?check_suite_focus=true

Looks like we need the jest upgrade to be able to use the runInBand.

Actually I think the UnhandledPromiseRejectionWarning doesn't require the upgrade. I think those tests just aren't handling promises correctly. I made changes to fix this in the jest upgrade PR, I can do that here too: https://github.com/opensearch-project/OpenSearch-Dashboards/pull/1301/files#diff-bef055a839e9b33dd53173ea66b575211e8e794624ae65a4c011c78e28ca2858

Update: I was wrong. That still didn't work. So yeah, I don't think we can backport this to 1.x.

* Removes caching from GitHub test workflow to simplify.
* Removes "bad apples" check from GitHub workflow since those tests now
  pass consistently.
* Runs unit tests in band to reduce execution time from ~90 minutes to
  ~20 minutes.
* Removes the logic that ignored failing unit tests.
* Adds mocha tests to the workflow.
* Adds ci-specific scripts to simplify the commands.
  * `yarn test:jest:ci` and `yarn test:jest_integration:ci` are now used
    in the workflow.

Resolves opensearch-project#1231

Signed-off-by: Tommy Markley <markleyt@amazon.com>
Copy link
Member

@kavilla kavilla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Thanks!

@tmarkley tmarkley requested a review from AMoo-Miki March 4, 2022 00:28
Copy link
Member

@ananzh ananzh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tmarkley tmarkley merged commit fef8af0 into opensearch-project:main Mar 4, 2022
@tmarkley tmarkley deleted the github-unit-tests branch March 4, 2022 00:57
AMoo-Miki added a commit to AMoo-Miki/OpenSearch-Dashboards that referenced this pull request Jan 20, 2023
* Run GitHub workflow unit tests in band (opensearch-project#1306)
* Fix JUnit Reporter test (opensearch-project#1338)
* Add code coverage reporting (opensearch-project#1478)
* Add release build verification workflow (opensearch-project#1502)
* Add Windows CI to the workflow (opensearch-project#2966)
* Add retrying flaky jest tests (opensearch-project#2967)
* Record functional test artifacts in case of failure (opensearch-project#3190)
* Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194)
* Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10

Signed-off-by: Miki <miki@amazon.com>
AMoo-Miki added a commit to AMoo-Miki/OpenSearch-Dashboards that referenced this pull request Jan 20, 2023
* Run GitHub workflow unit tests in band (opensearch-project#1306)
* Fix JUnit Reporter test (opensearch-project#1338)
* Add code coverage reporting (opensearch-project#1478)
* Add release build verification workflow (opensearch-project#1502)
* Add Windows CI to the workflow (opensearch-project#2966)
* Add retrying flaky jest tests (opensearch-project#2967)
* Record functional test artifacts in case of failure (opensearch-project#3190)
* Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193)
* Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194)
* Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10

Signed-off-by: Miki <miki@amazon.com>
AMoo-Miki added a commit to AMoo-Miki/OpenSearch-Dashboards that referenced this pull request Jan 20, 2023
* Run GitHub workflow unit tests in band (opensearch-project#1306)
* Fix JUnit Reporter test (opensearch-project#1338)
* Add code coverage reporting (opensearch-project#1478)
* Add release build verification workflow (opensearch-project#1502)
* Add Windows CI to the workflow (opensearch-project#2966)
* Add retrying flaky jest tests (opensearch-project#2967)
* Record functional test artifacts in case of failure (opensearch-project#3190)
* Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193)
* Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194)
* Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10

Signed-off-by: Miki <miki@amazon.com>
AMoo-Miki added a commit to AMoo-Miki/OpenSearch-Dashboards that referenced this pull request Jan 20, 2023
* Run GitHub workflow unit tests in band (opensearch-project#1306)
* Fix JUnit Reporter test (opensearch-project#1338)
* Add code coverage reporting (opensearch-project#1478)
* Add release build verification workflow (opensearch-project#1502)
* Add Windows CI to the workflow (opensearch-project#2966)
* Add retrying flaky jest tests (opensearch-project#2967)
* Record functional test artifacts in case of failure (opensearch-project#3190)
* Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193)
* Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194)
* Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10
* Fix unit tests of ExecutionContract throwing AbortError

Signed-off-by: Miki <miki@amazon.com>
AMoo-Miki added a commit to AMoo-Miki/OpenSearch-Dashboards that referenced this pull request Jan 20, 2023
* Run GitHub workflow unit tests in band (opensearch-project#1306)
* Fix JUnit Reporter test (opensearch-project#1338)
* Add code coverage reporting (opensearch-project#1478)
* Add release build verification workflow (opensearch-project#1502)
* Add Windows CI to the workflow (opensearch-project#2966)
* Add retrying flaky jest tests (opensearch-project#2967)
* Record functional test artifacts in case of failure (opensearch-project#3190)
* Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193)
* Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194)
* Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10
* Fix unit tests of ExecutionContract throwing AbortError

Signed-off-by: Miki <miki@amazon.com>
AMoo-Miki added a commit to AMoo-Miki/OpenSearch-Dashboards that referenced this pull request Jan 20, 2023
* Run GitHub workflow unit tests in band (opensearch-project#1306)
* Fix JUnit Reporter test (opensearch-project#1338)
* Add src/test_utils/ and src/core paths to the target clean patterns (opensearch-project#1442)
* Add code coverage reporting (opensearch-project#1478)
* Add release build verification workflow (opensearch-project#1502)
* Add Windows CI to the workflow (opensearch-project#2966)
* Add retrying flaky jest tests (opensearch-project#2967)
* Record functional test artifacts in case of failure (opensearch-project#3190)
* Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193)
* Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194)
* Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10
* Fix unit tests of ExecutionContract throwing AbortError

Signed-off-by: Miki <miki@amazon.com>
AMoo-Miki added a commit to AMoo-Miki/OpenSearch-Dashboards that referenced this pull request Jan 20, 2023
* Run GitHub workflow unit tests in band (opensearch-project#1306)
* Fix JUnit Reporter test (opensearch-project#1338)
* Add src/test_utils/ and src/core paths to the target clean patterns (opensearch-project#1442)
* Add code coverage reporting (opensearch-project#1478)
* Add release build verification workflow (opensearch-project#1502)
* Add Windows CI to the workflow (opensearch-project#2966)
* Add retrying flaky jest tests (opensearch-project#2967)
* Record functional test artifacts in case of failure (opensearch-project#3190)
* Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193)
* Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194)
* Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10
* Prevent `AbortError` from breaking unit tests for ExecutionContract

Signed-off-by: Miki <miki@amazon.com>
AMoo-Miki added a commit to AMoo-Miki/OpenSearch-Dashboards that referenced this pull request Jan 20, 2023
* Run GitHub workflow unit tests in band (opensearch-project#1306)
* Fix JUnit Reporter test (opensearch-project#1338)
* Add src/test_utils/ and src/core paths to the target clean patterns (opensearch-project#1442)
* Add code coverage reporting (opensearch-project#1478)
* Add release build verification workflow (opensearch-project#1502)
* Add Windows CI to the workflow (opensearch-project#2966)
* Add retrying flaky jest tests (opensearch-project#2967)
* Record functional test artifacts in case of failure (opensearch-project#3190)
* Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193)
* Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194)
* Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10
* Prevent `AbortError` from breaking unit tests for ExecutionContract

Signed-off-by: Miki <miki@amazon.com>
AMoo-Miki added a commit to AMoo-Miki/OpenSearch-Dashboards that referenced this pull request Jan 20, 2023
* Run GitHub workflow unit tests in band (opensearch-project#1306)
* Fix JUnit Reporter test (opensearch-project#1338)
* Add src/test_utils/ and src/core paths to the target clean patterns (opensearch-project#1442)
* Add code coverage reporting (opensearch-project#1478)
* Add release build verification workflow (opensearch-project#1502)
* Add Windows CI to the workflow (opensearch-project#2966)
* Add retrying flaky jest tests (opensearch-project#2967)
* Record functional test artifacts in case of failure (opensearch-project#3190)
* Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193)
* Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194)
* Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10
* Prevent `AbortError` from breaking unit tests for ExecutionContract

Signed-off-by: Miki <miki@amazon.com>
AMoo-Miki added a commit to AMoo-Miki/OpenSearch-Dashboards that referenced this pull request Jan 20, 2023
* Run GitHub workflow unit tests in band (opensearch-project#1306)
* Fix JUnit Reporter test (opensearch-project#1338)
* Add src/test_utils/ and src/core paths to the target clean patterns (opensearch-project#1442)
* Add code coverage reporting (opensearch-project#1478)
* Add release build verification workflow (opensearch-project#1502)
* Add Windows CI to the workflow (opensearch-project#2966)
* Add retrying flaky jest tests (opensearch-project#2967)
* Record functional test artifacts in case of failure (opensearch-project#3190)
* Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193)
* Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194)
* Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10
* Prevent `AbortError` from breaking unit tests for ExecutionContract

Signed-off-by: Miki <miki@amazon.com>
AMoo-Miki added a commit to AMoo-Miki/OpenSearch-Dashboards that referenced this pull request Jan 20, 2023
* Run GitHub workflow unit tests in band (opensearch-project#1306)
* Fix JUnit Reporter test (opensearch-project#1338)
* Add src/test_utils/ and src/core paths to the target clean patterns (opensearch-project#1442)
* Add code coverage reporting (opensearch-project#1478)
* Add release build verification workflow (opensearch-project#1502)
* Add Windows CI to the workflow (opensearch-project#2966)
* Add retrying flaky jest tests (opensearch-project#2967)
* Record functional test artifacts in case of failure (opensearch-project#3190)
* Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193)
* Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194)
* Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10
* Prevent `AbortError` from breaking unit tests for ExecutionContract

Signed-off-by: Miki <miki@amazon.com>
AMoo-Miki added a commit to AMoo-Miki/OpenSearch-Dashboards that referenced this pull request Jan 20, 2023
* Run GitHub workflow unit tests in band (opensearch-project#1306)
* Fix JUnit Reporter test (opensearch-project#1338)
* Add src/test_utils/ and src/core paths to the target clean patterns (opensearch-project#1442)
* Add code coverage reporting (opensearch-project#1478)
* Add release build verification workflow (opensearch-project#1502)
* Add Windows CI to the workflow (opensearch-project#2966)
* Add retrying flaky jest tests (opensearch-project#2967)
* Record functional test artifacts in case of failure (opensearch-project#3190)
* Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193)
* Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194)
* Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10
* Prevent `AbortError` from breaking unit tests for ExecutionContract

Signed-off-by: Miki <miki@amazon.com>
AMoo-Miki added a commit to AMoo-Miki/OpenSearch-Dashboards that referenced this pull request Jan 20, 2023
* Run GitHub workflow unit tests in band (opensearch-project#1306)
* Fix JUnit Reporter test (opensearch-project#1338)
* Add src/test_utils/ and src/core paths to the target clean patterns (opensearch-project#1442)
* Add code coverage reporting (opensearch-project#1478)
* Add release build verification workflow (opensearch-project#1502)
* Add Windows CI to the workflow (opensearch-project#2966)
* Add retrying flaky jest tests (opensearch-project#2967)
* Record functional test artifacts in case of failure (opensearch-project#3190)
* Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193)
* Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194)
* Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10
* Prevent `AbortError` from breaking unit tests for ExecutionContract

Signed-off-by: Miki <miki@amazon.com>
ananzh pushed a commit that referenced this pull request Jan 23, 2023
* Run GitHub workflow unit tests in band (#1306)
* Fix JUnit Reporter test (#1338)
* Add src/test_utils/ and src/core paths to the target clean patterns (#1442)
* Add code coverage reporting (#1478)
* Add release build verification workflow (#1502)
* Add Windows CI to the workflow (#2966)
* Add retrying flaky jest tests (#2967)
* Record functional test artifacts in case of failure (#3190)
* Increase timeout waiting for the exiting of an optimizer worker (#3193)
* Improve yarn's performance in workflows by caching yarn's cache folder (#3194)
* Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10
* Prevent `AbortError` from breaking unit tests for ExecutionContract

Signed-off-by: Miki <miki@amazon.com>

Signed-off-by: Miki <miki@amazon.com>
AMoo-Miki added a commit to AMoo-Miki/OpenSearch-Dashboards that referenced this pull request Jan 23, 2023
…ject#3299)

* Run GitHub workflow unit tests in band (opensearch-project#1306)
* Fix JUnit Reporter test (opensearch-project#1338)
* Add src/test_utils/ and src/core paths to the target clean patterns (opensearch-project#1442)
* Add code coverage reporting (opensearch-project#1478)
* Add release build verification workflow (opensearch-project#1502)
* Add Windows CI to the workflow (opensearch-project#2966)
* Add retrying flaky jest tests (opensearch-project#2967)
* Record functional test artifacts in case of failure (opensearch-project#3190)
* Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193)
* Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194)
* Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10
* Prevent `AbortError` from breaking unit tests for ExecutionContract

Signed-off-by: Miki <miki@amazon.com>
Signed-off-by: Miki <amoo_miki@yahoo.com>
AMoo-Miki added a commit to AMoo-Miki/OpenSearch-Dashboards that referenced this pull request Jan 24, 2023
…ject#3299)

* Run GitHub workflow unit tests in band (opensearch-project#1306)
* Fix JUnit Reporter test (opensearch-project#1338)
* Add src/test_utils/ and src/core paths to the target clean patterns (opensearch-project#1442)
* Add code coverage reporting (opensearch-project#1478)
* Add release build verification workflow (opensearch-project#1502)
* Add Windows CI to the workflow (opensearch-project#2966)
* Add retrying flaky jest tests (opensearch-project#2967)
* Record functional test artifacts in case of failure (opensearch-project#3190)
* Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193)
* Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194)
* Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10
* Prevent `AbortError` from breaking unit tests for ExecutionContract

Signed-off-by: Miki <miki@amazon.com>
Signed-off-by: Miki <amoo_miki@yahoo.com>
ananzh pushed a commit to ananzh/OpenSearch-Dashboards that referenced this pull request Jan 24, 2023
* Run GitHub workflow unit tests in band (opensearch-project#1306)
* Fix JUnit Reporter test (opensearch-project#1338)
* Add src/test_utils/ and src/core paths to the target clean patterns (opensearch-project#1442)
* Add code coverage reporting (opensearch-project#1478)
* Add release build verification workflow (opensearch-project#1502)
* Add Windows CI to the workflow (opensearch-project#2966)
* Add retrying flaky jest tests (opensearch-project#2967)
* Record functional test artifacts in case of failure (opensearch-project#3190)
* Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193)
* Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194)
* Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10
* Prevent `AbortError` from breaking unit tests for ExecutionContract

Signed-off-by: Miki <miki@amazon.com>
ananzh pushed a commit to ananzh/OpenSearch-Dashboards that referenced this pull request Jan 24, 2023
* Run GitHub workflow unit tests in band (opensearch-project#1306)
* Fix JUnit Reporter test (opensearch-project#1338)
* Add src/test_utils/ and src/core paths to the target clean patterns (opensearch-project#1442)
* Add code coverage reporting (opensearch-project#1478)
* Add release build verification workflow (opensearch-project#1502)
* Add Windows CI to the workflow (opensearch-project#2966)
* Add retrying flaky jest tests (opensearch-project#2967)
* Record functional test artifacts in case of failure (opensearch-project#3190)
* Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193)
* Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194)
* Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10
* Prevent `AbortError` from breaking unit tests for ExecutionContract

Signed-off-by: Miki <miki@amazon.com>
ananzh pushed a commit to ananzh/OpenSearch-Dashboards that referenced this pull request Jan 24, 2023
* Run GitHub workflow unit tests in band (opensearch-project#1306)
* Fix JUnit Reporter test (opensearch-project#1338)
* Add src/test_utils/ and src/core paths to the target clean patterns (opensearch-project#1442)
* Add code coverage reporting (opensearch-project#1478)
* Add release build verification workflow (opensearch-project#1502)
* Add Windows CI to the workflow (opensearch-project#2966)
* Add retrying flaky jest tests (opensearch-project#2967)
* Record functional test artifacts in case of failure (opensearch-project#3190)
* Increase timeout waiting for the exiting of an optimizer worker (opensearch-project#3193)
* Improve yarn's performance in workflows by caching yarn's cache folder (opensearch-project#3194)
* Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10
* Prevent `AbortError` from breaking unit tests for ExecutionContract

Backport PR: opensearch-project#3299
Signed-off-by: Miki <miki@amazon.com>
ananzh pushed a commit that referenced this pull request Jan 25, 2023
* Run GitHub workflow unit tests in band (#1306)
* Fix JUnit Reporter test (#1338)
* Add src/test_utils/ and src/core paths to the target clean patterns (#1442)
* Add code coverage reporting (#1478)
* Add release build verification workflow (#1502)
* Add Windows CI to the workflow (#2966)
* Add retrying flaky jest tests (#2967)
* Record functional test artifacts in case of failure (#3190)
* Increase timeout waiting for the exiting of an optimizer worker (#3193)
* Improve yarn's performance in workflows by caching yarn's cache folder (#3194)
* Fix the version of Chrome to 107 to match `chromedriver@107.0.1`, the last that works on NodeJS v10
* Prevent `AbortError` from breaking unit tests for ExecutionContract

Signed-off-by: Miki <miki@amazon.com>
Signed-off-by: Miki <amoo_miki@yahoo.com>

Signed-off-by: Miki <miki@amazon.com>
Signed-off-by: Miki <amoo_miki@yahoo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CI] Unit tests are failing in GitHub Action for all PRs and Pushes
3 participants