Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix planning conflict e2e test errors #3310

Merged
merged 1 commit into from
Nov 25, 2020

Conversation

ysf-simsoft
Copy link
Contributor

@ysf-simsoft ysf-simsoft commented Nov 19, 2020

wdio e2e test were failing randomly. I have continuously run the tests on BrowserStack in order to observe the problem. After detailed investigation it turns out that createReportWithPlanningConflict tests were suffering from timeout because this particular suit took 4-5 minutes (on BrowserStack) to run on average. Particularly "Should create second draft report with conflicts" spec was intermittently taking more than 90 seconds to run. I made changes to element queries in order to reduce run time and increased the timeout duration for test runner. As a result timeout related failures almost disappeared.

image

As can be seen in the screenshot above out of 1315 spec (~77 runs for whole suit) runs only 1 failure happened. And that particular failure is also happened due to a timeout but as can be seen in the raw webdriver logs in the following screenshot and BrowserStack logs on the latter secreenshot, webdriver waits for more than 60 seconds between two consecutive requests for querying an element. I couldn't figure out the reason for that as it is almost impossible to reproduce and debug any further.

image

image

Consequnetly createReportWithPlanningConflict suit's run time has decreased to ~3-3.5 minutes and "Should create second draft report with conflicts" specs run time has decreased by ~20 seconds. Even though there was an intermittent test failure, this PR will significantly reduce the timeout related issues, hopefully.

I also had to tweak editLocations and editPositions specs because they were also suffering from an unexpected alert open error on BrowserStack.

UPDATE:
I have merged these changes into my local keycloak-integration branch and wdio-tests have been running on BrowserStack for almost 24 hours. All wdio-tests ran more than 100 times without any failures. Last two sessions in the following screenshot doesn't have any failures. (Both sessions are timed out by BrowserStack btw 🙂 hence the yellow warning icons).

image

Closes #

User changes

Super User changes

Admin changes

System admin changes

  • anet.yml or anet-dictionary.yml needs change
  • db needs migration
  • documentation has changed
  • graphql schema has changed

Checklist

  • Described the user behavior in PR body
  • Referenced/updated all related issues
  • commits follow a repo#issue: Title title format and these 7 rules
  • commits have a clean history, otherwise PR may be squash-merged
  • Added and/or updated unit tests
  • Added and/or updated e2e tests
  • Added and/or updated data migrations
  • Updated documentation
  • Resolved all build errors and warnings
  • Opened debt issues for anything not resolved here

@ysf-simsoft ysf-simsoft force-pushed the fix-planning-conflict-e2e-test-errors branch from 4a57377 to fc6b1c0 Compare November 19, 2020 07:50
@ysf-simsoft ysf-simsoft marked this pull request as draft November 19, 2020 07:51
@ysf-simsoft ysf-simsoft marked this pull request as ready for review November 20, 2020 12:30
@gjvoosten gjvoosten merged commit 81897b7 into candidate Nov 25, 2020
@gjvoosten gjvoosten deleted the fix-planning-conflict-e2e-test-errors branch November 25, 2020 08:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants