Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Ganache errors that was reported in the Custom Network tests #27131

Open
hjetpoluru opened this issue Sep 13, 2024 · 1 comment
Open

Fix Ganache errors that was reported in the Custom Network tests #27131

hjetpoluru opened this issue Sep 13, 2024 · 1 comment
Labels
flaky tests Sev2-normal Normal severity; minor loss of service or inconvenience. team-extension-platform type-bug

Comments

@hjetpoluru
Copy link
Contributor

hjetpoluru commented Sep 13, 2024

CI failure reported on Sept 20
https://app.circleci.com/pipelines/github/MetaMask/metamask-extension/101484/workflows/02584d3e-9e9d-49d9-9953-0944167dace6/jobs/3778719/tests

Build - Webpack
Browser - Chrome
Test Path - /test/e2e/tests/transaction/send-eth.spec.js
Test Scenario -

  • Send ETH from inside MetaMask finds the transaction in the transactions list using advanced gas modal
  • Send ETH from inside MetaMask finds the transaction in the transactions list when sending to a Multisig Address
  • Send ETH from inside MetaMask shows no error when cancel transaction when sending via QR code
  • Send ETH from inside MetaMask from dapp using advanced gas controls should display the correct gas price on the legacy transaction
  • Send ETH from inside MetaMask from dapp using advanced gas controls should display correct gas values for EIP-1559 transaction
  • Send ETH from inside MetaMask to non-contract address with data that matches ERC20 transfer data signature renders the correct recipient on the confirmation screen

CI failure reported on Sept 20
https://app.circleci.com/pipelines/github/MetaMask/metamask-extension/101369/workflows/6c8551cb-1b8c-4ff5-a912-889d65e289b2/jobs/3774012/tests

Build - mmi
Browser - Chrome
Test Path - /test/e2e/tests/signature/signature-request.spec.js
Test Scenario -

  • Sign Typed Data Signature Request can queue multiple Signature Requests of Sign Typed Data V3 and reject
  • Sign Typed Data Signature Request can queue multiple Signature Requests of Sign Typed Data V4 and reject

CI failure reported on Sept 19
https://app.circleci.com/pipelines/github/MetaMask/metamask-extension/101259/workflows/506f3637-17c1-42dd-82b0-8e62ee9e6ff3/jobs/3770240/tests

Build - Regular
Browser - Chrome
Test Path - /test/e2e/tests/metrics/dapp-viewed.spec.js
Test Scenario -

  • Dapp viewed Event @no-mmi is sent when refreshing dapp with one account connected
  • Dapp viewed Event @no-mmi is sent when navigating to a connected dapp
  • Dapp viewed Event @no-mmi is sent when connecting dapp with two accounts
  • Dapp viewed Event @no-mmi is sent when reconnect to a dapp that has been connected before

CI failure reported on Sept 19
https://app.circleci.com/pipelines/github/MetaMask/metamask-extension/101117/workflows/42636518-13bc-42c7-8624-9334fe1ca94f/jobs/3764726/tests

Build - mmi
Browser - Chrome
Test Path - /test/e2e/tests/settings/settings-search.spec.js
Test Scenario -

  • Settings Search should find element inside the "Security & privacy" tab
  • Settings Search should find element inside the Alerts tab
  • Settings Search should find element inside the Networks tab
  • Settings Search should find element inside the Experimental tab
  • Settings Search should find element inside the About tab
  • Settings Search should display "Element not found" for a non-existing element

CI failure reported on Sept 17
https://app.circleci.com/pipelines/github/MetaMask/metamask-extension/100638/workflows/39f21f3a-0ac3-454b-a34a-3abe8ec618ac/jobs/3747023/tests

Build - webpack
Browser - Chrome
Test Path - /test/e2e/tests/simulation-details/simulation-details.spec.ts
Test Scenarios -

  • Simulation Details renders send eth transaction
  • Simulation Details renders buy ERC20 transaction
  • Simulation Details renders buy ERC721 transaction
  • Simulation Details renders buy ERC1155 transaction
  • Simulation Details renders no changes transaction
  • Simulation Details displays error message if transaction will fail or revert
  • Simulation Details does not display if chain is not supported
  • Simulation Details displays generic error message

Another CI failure reported on Sept 17
https://app.circleci.com/pipelines/github/MetaMask/metamask-extension/100523/workflows/8e4175f9-0b34-416f-815d-2524eb92820e/jobs/3742442/tests

Build - webpack
Browser - Chrome
Test Path - /test/e2e/tests/transaction/send-eth.spec.js
Test Scenario -

  • Send ETH from inside MetaMask finds the transaction in the transactions list using advanced gas modal
  • Send ETH from inside MetaMask finds the transaction in the transactions list when sending to a Multisig Address
  • Send ETH from inside MetaMask shows no error when cancel transaction when sending via QR code
  • Send ETH from inside MetaMask from dapp using advanced gas controls should display the correct gas price on the legacy transaction

CI failure reported on Sept 13
https://app.circleci.com/pipelines/github/MetaMask/metamask-extension/100198/workflows/7d530039-f9aa-486c-ab61-17e60c5647ca/jobs/3730323/tests

Build - mmi
Browser - Chrome
Test Path - /test/e2e/tests/network/add-custom-network.spec.js
Test scenarios -

  • Custom network JSON-RPC API don't validate bad rpc custom network when toggle is off
  • Custom network JSON-RPC API don't add unreachable custom network
  • Custom network Popular Networks List add custom network and switch the network
  • Custom network Popular Networks List add custom network and not switch the network
  • Custom network Popular Networks List delete the Arbitrum network
  • Custom network Popular Networks List when the network details validation toggle is turned on, validate user inserted details against data from 'chainid.network'
  • Custom network Popular Networks List when the network details validation toggle is turned off, don't validate user inserted details
  • Custom network customNetwork should add mainnet network
  • Custom network customNetwork should check symbol and show warnings
  • Custom network customNetwork should add collision network

Initial error message for the first test scenario

Error: Timeout of 80000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/home/circleci/project/test/e2e/tests/network/add-custom-network.spec.js)
    at listOnTimeout (node:internal/timers:581:17)
    at process.processTimers (node:internal/timers:519:7)
Error: listen EADDRINUSE: address already in use 127.0.0.1:8545.
  (Ran on CircleCI Node 1 of 12, Job test-e2e-chrome-mmi)
    at /home/circleci/project/node_modules/ganache/dist/node/1.js:2:5133
    at async Promise.allSettled (index 1)
    at async Ganache.start (test/e2e/seeder/ganache.ts:2:1285)
    at async withFixtures (test/e2e/helpers.js:105:7)
    at async Context.<anonymous> (test/e2e/tests/network/add-custom-network.spec.js:393:7)
@hjetpoluru hjetpoluru added type-bug Sev2-normal Normal severity; minor loss of service or inconvenience. team-extension-platform flaky tests labels Sep 13, 2024
@HowardBraham
Copy link
Contributor

@hjetpoluru I found a very simple reproduction for this error.

In any suite that has multiple tests, add this.timeout(5000); right under the describe( line.

The first test will fail with Error: Timeout of 5000ms exceeded....
The subsequent tests will fail with Error: listen EADDRINUSE: address already in use 127.0.0.1:8545

I did a little digging, and what's happening is that if mocha causes the test to quit (rather than the test internally throwing an error), the blocks in withFixtures() for catch and finally don't run. So the teardown doesn't happen properly, and the next tests fail.

This is not an easy thing to fix, it's a huge funnel cake.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
flaky tests Sev2-normal Normal severity; minor loss of service or inconvenience. team-extension-platform type-bug
Projects
Status: To be fixed
Status: To be fixed
Development

No branches or pull requests

2 participants