Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: Enable Regression tests on Bitrise #7645

Merged
merged 44 commits into from
Nov 1, 2023
Merged

test: Enable Regression tests on Bitrise #7645

merged 44 commits into from
Nov 1, 2023

Conversation

cortisiko
Copy link
Member

@cortisiko cortisiko commented Oct 30, 2023 •
edited
Loading

Description

The purpose of this PR is to set up the regression test pipeline in Bitrise. Currently, we conduct regression tests on our local machines, which has its limitations. An improved approach involves running these tests on a virtual machine, which offers more flexibility and scalability.

The regression pipeline in bitrise is called: pr_regression_e2e_tests

Also, I removed all flaky/failing tests from the regression tag and placed them in a quarantine folder.

Related issues

Fixes: #

Manual testing steps

  1. Go to this page...

Screenshots/Recordings

Before

After

Pre-merge author checklist

  • I’ve followed MetaMask Coding Standards.
  • I've clearly explained what problem this PR is solving and how it is solved.
  • I've linked related issues
  • I've included manual testing steps
  • I've included screenshots/recordings if applicable
  • I’ve included tests if applicable
  • I’ve documented my code using JSDoc format if applicable
  • I’ve applied the right labels on the PR (see labeling guidelines). Not required for external contributors.
  • I’ve properly set the pull request status:
    • In case it's not yet "ready for review", I've set it to "draft".
    • In case it's "ready for review", I've changed it from "draft" to "non-draft".

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

@cortisiko
set smoke by default
2e1f45d
@cortisiko
set default to smoke
e986c53
@cortisiko
test conditional statement
4e9b169
@cortisiko
test config conditional
bfaefcd
@cortisiko
fix spacing
70d4aa8
@cortisiko
test smoke output
6bb92b7
@cortisiko
fix echo
c3f2552
@cortisiko
create regression pipeline
27d6c09
@cortisiko
enable detox yarn script
40c75c5
@cortisiko
disable approve custom ERC20 test
c830ee8
@cortisiko
temporarily add push trigger for regression
4d2b83c
@cortisiko
changing to smoke to investigate cmake issue
2ea157b
@cortisiko
order of operations
81e5da9
@cortisiko
hard coding test suite for now
acfa1eb
@cortisiko
Revert "hard coding test suite for now"
54f353f 
This reverts commit acfa1eb.
@cortisiko
temporarily changing condition statement
974542d
@cortisiko
Revert "temporarily changing condition statement"
f61bd5d 
This reverts commit 974542d.
@cortisiko
add script to install missing dependencies
d40c0f2
@cortisiko
script for dependency
79d4161
@cortisiko
script for updating the SDK manager.
81d0256
@cortisiko
revert
00a7000
@cortisiko
temporarily removing cmake check
495a51f
@cortisiko
Revert "temporarily removing cmake check"
3dd270a 
This reverts commit 495a51f.
@cortisiko
fix meta tag for build machine
d4950e0
@cortisiko
Revert "fix meta tag for build machine"
8cb0e46 
This reverts commit d4950e0.
@cortisiko
set machine for regression workflow
69723b2
@cortisiko
temporarily moving type-sign tests from regression tag to monitor ove…
e7be2aa 
…rall test run
@cortisiko cortisiko added No QA Needed Apply this label when your PR does not need any QA effort. team-mobile-client labels Oct 30, 2023
@cortisiko cortisiko marked this pull request as ready for review November 1, 2023 02:30
@cortisiko cortisiko requested a review from a team as a code owner November 1, 2023 02:30
@cortisiko cortisiko added the needs-dev-review PR needs reviews from other engineers (in order to receive required approvals) label Nov 1, 2023
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 1, 2023

E2E test started on Bitrise: https://app.bitrise.io/app/be69d4368ee7e86d/pipelines/4a705f22-3bf1-49cc-a633-6b30d27e9954
You can also kick off another Bitrise E2E smoke test by removing and re-applying the (Run Smoke E2E) label

Andepande
Andepande previously approved these changes Nov 1, 2023
View reviewed changes
SamuelSalas
SamuelSalas previously approved these changes Nov 1, 2023
View reviewed changes
@cortisiko cortisiko dismissed stale reviews from SamuelSalas and Andepande via c2c8921 November 1, 2023 16:55
@sonarqubecloud SonarQubeCloud
Copy link

sonarqubecloud bot commented Nov 1, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@cortisiko cortisiko removed the needs-dev-review PR needs reviews from other engineers (in order to receive required approvals) label Nov 1, 2023
Andepande
Andepande approved these changes Nov 1, 2023
View reviewed changes
Copy link
Member

@Andepande Andepande left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG

@cortisiko cortisiko merged commit 24a1d3f into main Nov 1, 2023
26 checks passed
@cortisiko cortisiko deleted the detox-add-regreesion branch November 1, 2023 19:08
@github-actions github-actions bot locked and limited conversation to collaborators Nov 1, 2023
@metamaskbot metamaskbot added the release-7.11.0 Issue or pull request that will be included in release 7.11.0 label Nov 1, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@Andepande Andepande Andepande approved these changes

@SamuelSalas SamuelSalas SamuelSalas left review comments

Assignees
No one assigned
Labels
bitrise No QA Needed Apply this label when your PR does not need any QA effort. release-7.11.0 Issue or pull request that will be included in release 7.11.0 team-mobile-platform
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@cortisiko @codecov-commenter @SamuelSalas @Andepande @gauthierpetetin @metamaskbot