Codecept with Playwright e2e tests throws connection refused error in Github Actions

[RashmiRamdas]

Context:

• Playwright Version: 1.18.0
• Operating System: Linux
• Node.js version: 14
• Browser: [Chromium, Firefox]

I'm trying to run e2e tests in parallel with workers (run-workers) in Github Actions in headless mode and getting this error
browserContext.storageState: net::ERR_FAILED at https://localhost:3000/ at Playwright._after

Some of the tests run successfully before throwing this error.
I have observed this error only on Github Actions and works fine on local

browserContext.storageState: net::ERR_FAILED at https://localhost:3000/
    at Playwright._after (/data/runners/node_modules/codeceptjs/lib/helper/Playwright.js:434:50) {
  name: 'Error'
}
  Click on close icon in nav bar when the form is in dirty state then Save the changes- Crystal
    --- STARTED "before each" hook: Before for "Click on close icon in nav bar when the form is in dirty state then Save the changes- Crystal" ---
    I clear cookie 
    › Screenshot is saving to /data/runners/test/output/record_2ade0e9c5f1ca5cebfabd11e1e1ae3fa/0000.png
    I am on page "/home"
    [1] Error | Error: page.goto: net::ERR_CONNECTION_REFUSED at https://localhost:3000/showcase/home
=========================== logs ===========================
navigating to "https://localhost:3000/showcase/home", waiting until "networkidle"
============================================================
    › Screenshot is saving to /data/runners/test/output/record_2ade0e9c5f1ca5cebfabd11e1e1ae3fa/0001.png
    [1] Error | Error: page.goto: net::ERR_CONNECTION_REFUSED at https://localhost:3000/showcase/home
=========================== logs ===========================
navigating to "https://localhost:3000/showcase/home", waiting until "networkidle"
============================================================
    [1] Error | Error: page.goto: net::ERR_CONNECTION_REFUSED at https://localhost:3000/showcase/home
=========================== logs ===========================
navigating to "https://localhost:3000/showcase/home", waiting until "networkidle"
============================================================
    [1] Error | Error: page.goto: net::ERR_CONNECTION_REFUSED at https://localhost:3000/showcase/home
=========================== logs ===========================
navigating to "https://localhost:3000/showcase/home", waiting until "networkidle"
============================================================
    [1] Error | Error: page.goto: net::ERR_CONNECTION_REFUSED at https://localhost:3000/showcase/home
=========================== logs ===========================
navigating to "https://localhost:3000/showcase/home", waiting until "networkidle"
============================================================
    [1] Error | Error: page.goto: net::ERR_CONNECTION_REFUSED at https://localhost:3000/showcase/home
=========================== logs ===========================
navigating to "https://localhost:3000/showcase/home", waiting until "networkidle"
============================================================
    [1] Error | Error: page.goto: net::ERR_CONNECTION_REFUSED at https://localhost:3000/showcase/home
=========================== logs ===========================
navigating to "https://localhost:3000/showcase/home", waiting until "networkidle"
============================================================
    [1] Error | Error: page.goto: net::ERR_CONNECTION_REFUSED at https://localhost:3000/showcase/home
=========================== logs ===========================
navigating to "https://localhost:3000/showcase/home", waiting until "networkidle"
============================================================
    [1] Error | Error: page.goto: net::ERR_CONNECTION_REFUSED at https://localhost:3000/showcase/home
=========================== logs ===========================
navigating to "https://localhost:3000/showcase/home", waiting until "networkidle"
============================================================
    [1] Error | Error: page.goto: net::ERR_CONNECTION_REFUSED at https://localhost:3000/showcase/home
=========================== logs ===========================
navigating to "https://localhost:3000/showcase/home", waiting until "networkidle"
============================================================
    [1] Error | Error: page.goto: net::ERR_CONNECTION_REFUSED at https://localhost:3000/showcase/home
=========================== logs ===========================
navigating to "https://localhost:3000/showcase/home", waiting until "networkidle"
============================================================
    [1] Error | Error: page.goto: net::ERR_CONNECTION_REFUSED at https://localhost:3000/showcase/home
=========================== logs ===========================
navigating to "https://localhost:3000/showcase/home", waiting until "networkidle"
============================================================
    [1] <teardown> Stopping recording promises
  ✖ FAILED in 212ms

configuration file

const { setHeadlessWhen, setWindowSize } = require("@codeceptjs/configure");
const { integration } = require(`./../package.json`);
const page = require("./utils/pageObject");
const bootstrap = require("./utils/boostrap");
const users = require("./utils/users");

setHeadlessWhen(process.env.CI || integration.headless);
setWindowSize(1920, 1080);

exports.config = {
    tests: `./**/*.+(e2e|visual).+(js|ts)`,
    output: `./output`,
    helpers: {
        Playwright: {
            url: process.env.SHOWCASE_URL || integration.url || "https://localhost:3000/showcase",
            browser: process.profile || integration.browser || "chromium",
            show: !integration.headless,
            coloredLogs: true,
            smartWait: integration.waitInSeconds * 1000 || 5 * 1000,
            uniqueScreenshotNames: true,
            fullPageScreenshots: true,
            restart: false,
            keepBrowserState: true,
            keepCookies: true,
            waitForNavigation: "networkidle0",
            timeouts: {
                script: 60000,
                "page load": 10000,
            },
            ignoreHTTPSErrors: true,
            chromium: {
                args: [
                    "--disable-web-security",
                    "--allow-insecure-localhost",
                    "--ignore-certificate-errors",
                    "--no-sandbox",
                    "-–allow-file-access-from-files",
                ],
            },
            firefox: {
                args: [
                    "--disable-web-security",
                    "--allow-insecure-localhost",
                    "--ignore-certificate-errors",
                    "--ignore-ssl-errors",
                    "--no-sandbox",
                    "-–allow-file-access-from-files",
                ],
            },
        },
        Actions: {
            require: "./utils/helpers/actionsHelper.js",
        },
    },
    include: {
        I: "./utils/customSteps.js",
        Selectors: "./utils/pageObject",
    },
    multiple: {
        e2e: {
            browsers: [{ browser: "firefox" }, { browser: "chromium" }],
        },
    },
    bootstrap: bootstrap,
    mocha: {},
    name: "showcase",
    plugins: {
        retryFailedStep: {
            enabled: true,
        },
        stepByStepReport: {
            enabled: true,
        },
        screenshotOnFail: {
            enabled: false,
        },
        customLocator: {
            enabled: true,
            prefix: "testid=",
            attribute: "data-testid",
        },
        allure: {
            enabled: false,
            enableScreenshotDiffPlugin: false,
        },
        autoLogin: {
            enabled: true,
            saveToFile: true,
            inject: "loginAs",
            users: users,
        },
        stepTimeout:{
            enabled: true,
            timeout: 10
        }
    },
};

Describe the bug

Add any other details about the problem here.

[yury-s]

at Playwright._after (/data/runners/node_modules/codeceptjs/lib/helper/Playwright.js:434:50) {

The problem is in codeceptjs module, please open an issue in their repository instead, there is no Playwright._after in the stock Playwright library.

[achisholm]

Not entirely sure, but I think this is the same issue I got when I installed Playwright using the default settings after installing and committing to GH.

GH Actions log:

Run npx playwright test
  

Running 9 tests using 1 worker
××F··××F··××F··

  1) [chromium] › basket.spec.ts:3:1 › meta is correct =============================================

    page.goto: net::ERR_CONNECTION_REFUSED at http://localhost:3000/basket
    =========================== logs ===========================
    navigating to "http://localhost:3000/basket", waiting until "load"
    ============================================================

      2 |
      3 | test('meta is correct', async ({ page }) => {
    > [4](https://github.com/achisholm/proto-parallax/actions/runs/3981080717/jobs/6824578955#step:6:4) |     await page.goto("http://localhost:3000/basket");
        |                ^
      [5](https://github.com/achisholm/proto-parallax/actions/runs/3981080717/jobs/6824578955#step:6:6) |
      [6](https://github.com/achisholm/proto-parallax/actions/runs/3981080717/jobs/6824578955#step:6:7) |     await expect(page).toHaveTitle('Basket');
      [7](https://github.com/achisholm/proto-parallax/actions/runs/3981080717/jobs/6824578955#step:6:8) | });

        at file:///home/runner/work/proto-parallax/proto-parallax/tests/basket.spec.ts:4:16

    Retry #1 ---------------------------------------------------------------------------------------

    page.goto: net::ERR_CONNECTION_REFUSED at http://localhost:3000/basket
    =========================== logs ===========================
    navigating to "http://localhost:3000/basket", waiting until "load"
    ============================================================

      2 |
      3 | test('meta is correct', async ({ page }) => {
    > 4 |     await page.goto("http://localhost:3000/basket");
        |                ^
      5 |
      6 |     await expect(page).toHaveTitle('Basket');
      7 | });

        at file:///home/runner/work/proto-parallax/proto-parallax/tests/basket.spec.ts:4:[16](https://github.com/achisholm/proto-parallax/actions/runs/3981080717/jobs/6824578955#step:6:17)

    attachment #1: trace (application/zip) ---------------------------------------------------------
    test-results/basket-meta-is-correct-chromium-retry1/trace.zip
    Usage:

        npx playwright show-trace test-results/basket-meta-is-correct-chromium-retry1/trace.zip

    ------------------------------------------------------------------------------------------------

    Retry #2 ---------------------------------------------------------------------------------------

    page.goto: net::ERR_CONNECTION_REFUSED at http://localhost:3000/basket
    =========================== logs ===========================
    navigating to "http://localhost:3000/basket", waiting until "load"
    ============================================================

      2 |
      3 | test('meta is correct', async ({ page }) => {
    > 4 |     await page.goto("http://localhost:3000/basket");
        |                ^
      5 |
      6 |     await expect(page).toHaveTitle('Basket');
      7 | });

        at file:///home/runner/work/proto-parallax/proto-parallax/tests/basket.spec.ts:4:16

  2) [firefox] › basket.spec.ts:3:1 › meta is correct ==============================================

    page.goto: NS_ERROR_CONNECTION_REFUSED
    =========================== logs ===========================
    navigating to "http://localhost:3000/basket", waiting until "load"
    ============================================================

      2 |
      3 | test('meta is correct', async ({ page }) => {
    > 4 |     await page.goto("http://localhost:3000/basket");
        |                ^
      5 |
      6 |     await expect(page).toHaveTitle('Basket');
      7 | });

        at file:///home/runner/work/proto-parallax/proto-parallax/tests/basket.spec.ts:4:16

    Retry #1 ---------------------------------------------------------------------------------------

    page.goto: NS_ERROR_CONNECTION_REFUSED
    =========================== logs ===========================
    navigating to "http://localhost:[30](https://github.com/achisholm/proto-parallax/actions/runs/3981080717/jobs/6824578955#step:6:31)00/basket", waiting until "load"
    ============================================================

      2 |
      3 | test('meta is correct', async ({ page }) => {
    > 4 |     await page.goto("http://localhost:3000/basket");
        |                ^
      5 |
      6 |     await expect(page).toHaveTitle('Basket');
      7 | });

        at file:///home/runner/work/proto-parallax/proto-parallax/tests/basket.spec.ts:4:16

    attachment #1: trace (application/zip) ---------------------------------------------------------
    test-results/basket-meta-is-correct-firefox-retry1/trace.zip
    Usage:

        npx playwright show-trace test-results/basket-meta-is-correct-firefox-retry1/trace.zip

    ------------------------------------------------------------------------------------------------

    Retry #2 ---------------------------------------------------------------------------------------

    page.goto: NS_ERROR_CONNECTION_REFUSED
    =========================== logs ===========================
    navigating to "http://localhost:3000/basket", waiting until "load"
    ============================================================

      2 |
      3 | test('meta is correct', async ({ page }) => {
    > 4 |     await page.goto("http://localhost:3000/basket");
        |                ^
      5 |
      6 |     await expect(page).toHaveTitle('Basket');
      7 | });

        at file:///home/runner/work/proto-parallax/proto-parallax/tests/basket.spec.ts:4:16

  3) [webkit] › basket.spec.ts:3:1 › meta is correct ===============================================

    page.goto: Could not connect: Connection refused
    =========================== logs ===========================
    navigating to "http://localhost:3000/basket", waiting until "load"
    ============================================================

      2 |
      3 | test('meta is correct', async ({ page }) => {
    > 4 |     await page.goto("http://localhost:3000/basket");
        |                ^
      5 |
      6 |     await expect(page).toHaveTitle('Basket');
      7 | });

        at file:///home/runner/work/proto-parallax/proto-parallax/tests/basket.spec.ts:4:16

    Retry #1 ---------------------------------------------------------------------------------------

    page.goto: Could not connect: Connection refused
    =========================== logs ===========================
    navigating to "http://localhost:3000/basket", waiting until "load"
    ============================================================

      2 |
      3 | test('meta is correct', async ({ page }) => {
    > 4 |     await page.goto("http://localhost:3000/basket");
        |                ^
      5 |
      6 |     await expect(page).toHaveTitle('Basket');
      7 | });

        at file:///home/runner/work/proto-parallax/proto-parallax/tests/basket.spec.ts:4:16

    attachment #1: trace (application/zip) ---------------------------------------------------------
    test-results/basket-meta-is-correct-webkit-retry1/trace.zip
    Usage:

        npx playwright show-trace test-results/basket-meta-is-correct-webkit-retry1/trace.zip

    ------------------------------------------------------------------------------------------------

    Retry #2 ---------------------------------------------------------------------------------------

    page.goto: Could not connect: Connection refused
    =========================== logs ===========================
    navigating to "http://localhost:3000/basket", waiting until "load"
    ============================================================

      2 |
      3 | test('meta is correct', async ({ page }) => {
    > 4 |     await page.goto("http://localhost:3000/basket");
        |                ^
      5 |
      6 |     await expect(page).toHaveTitle('Basket');
      7 | });

        at file:///home/runner/work/proto-parallax/proto-parallax/tests/basket.spec.ts:4:16


  3 failed
    [chromium] › basket.spec.ts:3:1 › meta is correct ==============================================
    [firefox] › basket.spec.ts:3:1 › meta is correct ===============================================
    [webkit] › basket.spec.ts:3:1 › meta is correct ================================================
  6 passed (30s)
Error: Process completed with exit code 1.

[RashmiRamdas]

@achisholm Are you using playwright with codeceptjs for your testing?
Playwright team mentioned that the issue is with codeceptjs module.

[achisholm]

Actuallly, no. I've come to Playwright from the Astro framework so this might be a separate issue. I've basically followed the documentation here and hit the issue, using the bootstrap code that was generated for me:

https://docs.astro.build/en/guides/testing/

[RomarioNijimAlmond]

@achisholm did you solve the issue? I'm getting this error myself

Error: page.goto: net::ERR_CONNECTION_REFUSED at http://localhost:3000/users/sign_in
=========================== logs ===========================
navigating to "http://localhost:3000/users/sign_in", waiting until "load"
============================================================

   6 |     // console.log('Page Title:', await page.title());
   7 |     await page.waitForTimeout(5000);
>  8 |     await page.goto('http://localhost:3000/users/sign_in');
     |                ^
   9 |     const title = await page.title();
  10 |     expect(title).toBe(process.env.TITLE);

[dguo]

@RomarioNijimAlmond (and for anyone else who runs into a similar problem and ends up here), check out this issue: #20784

Upgrading your dependencies may be enough to fix the problem.