Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Fix flakiness caused by the notification permission dialog which caused confirmation test flakiness #9610

Merged
merged 1 commit into from
May 10, 2024

Conversation

cortisiko
Copy link
Member

@cortisiko cortisiko commented May 10, 2024

Description

The purpose of this PR is to address sporadic test flakiness within the confirmations workflow. The test, at times, would timeout because it was unable to log into the app. Sometimes, this would result in long-running confirmation e2e workflows in Bitrise.

What caused the failure?

The approve-custom-erc20.spec.js and approve-default-erc20.spec.js tests assert that the submitted the txn is confirmed before making the test as passed.

The notification dialog appears whenever a successful transaction is made. See below
image

Unfortunately, the system dialog lingers on the simulator.

Whenever new tests run on the simulator, the system dialog prevents the tests from moving forward. See the below screenshot:

image

Solution

The solution is to set the notifications permission to true. This will prevent the notification dialog from appearing within the tests. Detox's api allows us to set this permission whenever we launch the app for testing. For the curious minds, please read up more here.

Related issues

Fixes:

Manual testing steps

  1. Go to this page...

Screenshots/Recordings

Before

After

Pre-merge author checklist

  • I’ve followed MetaMask Coding Standards.
  • I've completed the PR template to the best of my ability
  • I’ve included tests if applicable
  • I’ve documented my code using JSDoc format if applicable
  • I’ve applied the right labels on the PR (see labeling guidelines). Not required for external contributors.

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

@cortisiko cortisiko added No QA Needed Apply this label when your PR does not need any QA effort. team-mobile-platform Run Smoke E2E Triggers smoke e2e on Bitrise labels May 10, 2024
@cortisiko cortisiko self-assigned this May 10, 2024
@cortisiko cortisiko requested a review from a team as a code owner May 10, 2024 16:32
@cortisiko cortisiko linked an issue May 10, 2024 that may be closed by this pull request
9 tasks
Copy link
Contributor

github-actions bot commented May 10, 2024

https://bitrise.io/ Bitrise

✅✅✅ pr_smoke_e2e_pipeline passed on Bitrise! ✅✅✅

Commit hash: daca1d9
Build link: https://app.bitrise.io/app/be69d4368ee7e86d/pipelines/cd1f4c78-1034-42cd-8031-15b35f804598

Note

  • You can kick off another pr_smoke_e2e_pipeline on Bitrise by removing and re-applying the Run Smoke E2E label on the pull request

Copy link

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

Copy link
Contributor

@Cal-L Cal-L left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cortisiko cortisiko merged commit 78dd5f1 into main May 10, 2024
46 checks passed
@cortisiko cortisiko deleted the e2e-fix-flakiness-within-confirmations-workflow branch May 10, 2024 17:53
@github-actions github-actions bot locked and limited conversation to collaborators May 10, 2024
@metamaskbot metamaskbot added the release-7.23.0 Issue or pull request that will be included in release 7.23.0 label May 10, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
No QA Needed Apply this label when your PR does not need any QA effort. release-7.23.0 Issue or pull request that will be included in release 7.23.0 Run Smoke E2E Triggers smoke e2e on Bitrise team-mobile-platform
Projects
None yet
Development

Successfully merging this pull request may close these issues.

E2E: Investigate inconsistent E2E failures within confirmations workflow
3 participants