fix(electron): tracing with @playwright/test #31437

mxschmitt · 2024-06-25T14:05:48Z

This regressed in e7a11c0#diff-d7c041137e763edae5c4d18bc8ded9a1ac668078e310712341c444ef3aac423b - v1.45.0.

When Electron browser context gets closed, the request object gets disposed. But tracing was never started on the Electron request instance. This throws now:

➜  playwright git:(bugfix/electron-pwt) ✗ npm run ctest --  --trace=on wheel:18

> playwright-internal@1.46.0-next ctest
> playwright test --config=tests/library/playwright.config.ts --project=chromium-* --trace=on wheel:18


Running 1 test using 1 worker
  1) [chromium-page] › page/wheel.spec.ts:18:5 › should work ───────────────────────────────────────

    Error: ENOENT: no such file or directory, open '/Users/maxschmitt/Developer/playwright/test-results/.playwright-artifacts-0/fd9b718e68bb993159f60c54492d3af1.zip'

    attachment #1: trace (application/zip) ─────────────────────────────────────────────────────────
    test-results/wheel-should-work-chromium-page/electron-trace.zip
    Usage:

        npx playwright show-trace test-results/wheel-should-work-chromium-page/electron-trace.zip

    ────────────────────────────────────────────────────────────────────────────────────────────────

  1 failed
    [chromium-page] › page/wheel.spec.ts:18:5 › should work ────────────────────────────────────────

To open last HTML report run:

  npx playwright show-report

➜  playwright git:(bugfix/electron-pwt) ✗

In normal PWT setup, this is not a problem, since we already start the tracing before. But since we don't check if the Tracing has been started, it throws now, that we didn't start it before.

I see two solutions:

a) We call didCreateContext in Electron, see footnote

Tracing support in Electron

Risk which we don't want right now.

b) just ignore the stop call, it wasn't marked with the start symbol.

diff --git a/packages/playwright-core/src/client/electron.ts b/packages/playwright-core/src/client/electron.ts
index a9e90b77f..bb92fdfa5 100644
--- a/packages/playwright-core/src/client/electron.ts
+++ b/packages/playwright-core/src/client/electron.ts
@@ -30,6 +30,7 @@ import { ConsoleMessage } from './consoleMessage';
 import type { Env, WaitForEventOptions, Headers, BrowserContextOptions } from './types';
 import { Waiter } from './waiter';
 import { TargetClosedError } from './errors';
+import { Playwright } from './playwright';
 
 type ElectronOptions = Omit<channels.ElectronLaunchOptions, 'env'|'extraHTTPHeaders'|'recordHar'|'colorScheme'|'acceptDownloads'> & {
   env?: Env,
@@ -42,6 +43,8 @@ type ElectronOptions = Omit<channels.ElectronLaunchOptions, 'env'|'extraHTTPHead
 type ElectronAppType = typeof import('electron');
 
 export class Electron extends ChannelOwner<channels.ElectronChannel> implements api.Electron {
+  _playwright!: Playwright;
+
   static from(electron: channels.ElectronChannel): Electron {
     return (electron as any)._object;
   }
@@ -57,6 +60,7 @@ export class Electron extends ChannelOwner<channels.ElectronChannel> implements
       tracesDir: options.tracesDir,
     };
     const app = ElectronApplication.from((await this._channel.launch(params)).electronApplication);
+    await this._playwright.chromium._didCreateContext(app.context(), params, options, this._logger);
     app._context._setOptions(params, options);
     return app;
   }

pavelfeldman · 2024-06-28T03:45:39Z

I think you are on the right track, but I don't think that is the right place for the fix. Consider a situation where tracing is started, but no trace events are emitted. As the trace file is created lazily, it won't be there when stopping the trace either. So the fix should rather remove the non-existing temporary files from the list before iterating over them in mergeTraceFiles.

You can probably test it using the ttest rather than an installation test btw.

This reverts commit 8f67ee1.

github-actions · 2024-07-01T17:23:09Z

Test results for "tests 1"

8 flaky

⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [firefox-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-page] › page/page-event-request.spec.ts:138:3 › should report navigation requests and responses handled by service worker with routing
⚠️ [playwright-test] › ui-mode-test-screencast.spec.ts:21:5 › should show screenshots
⚠️ [webkit-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture

28408 passed, 655 skipped
✔️✔️✔️

Merge workflow run.

github-actions · 2024-07-01T17:29:42Z

Test results for "tests others"

2 flaky

⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture

17787 passed, 482 skipped
✔️✔️✔️

Merge workflow run.

github-actions · 2024-07-01T18:54:13Z

Test results for "tests 2"

4 fatal errors, not part of any test

92 flaky

⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/beforeunload.spec.ts:20:3 › should close browser with beforeunload page
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-viewport.spec.ts:145:12 › should drag with high dpi
⚠️ [chromium-page] › page/page-event-request.spec.ts:110:3 › should report navigation requests and responses handled by service worker
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/inspector/cli-codegen-1.spec.ts:197:7 › cli codegen › should not target selector preview by text regexp
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/inspector/cli-codegen-3.spec.ts:383:7 › cli codegen › should generate getByTestId
⚠️ [chromium-page] › page/locator-misc-1.spec.ts:28:3 › should hover when Node is removed
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/selector-generator.spec.ts:454:5 › selector generator › should generate label selector
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/trace-viewer.spec.ts:726:1 › should follow redirects
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [firefox-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [firefox-page] › page/page-event-request.spec.ts:169:3 › should return response body when Cross-Origin-Opener-Policy is set
⚠️ [firefox-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [firefox-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [firefox-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [firefox-library] › library/video.spec.ts:189:5 › screencast › should capture static page
⚠️ [firefox-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [firefox-page] › page/page-click.spec.ts:230:3 › should click on checkbox input and toggle
⚠️ [firefox-library] › library/browsercontext-basic.spec.ts:331:3 › should emulate media in cross-process iframe
⚠️ [firefox-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [firefox-page] › page/page-mouse.spec.ts:193:3 › should trigger hover state with removed window.Node
⚠️ [firefox-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [firefox-page] › page/page-click.spec.ts:97:3 › should click the 1x1 div
⚠️ [firefox-page] › page/page-clock.frozen.spec.ts:24:3 › clock should be realtime
⚠️ [firefox-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [firefox-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [firefox-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [firefox-library] › library/tracing.spec.ts:412:14 › should produce screencast frames scale
⚠️ [firefox-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [firefox-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [firefox-page] › page/workers.spec.ts:106:3 › should clear upon navigation
⚠️ [firefox-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-viewport.spec.ts:145:12 › should drag with high dpi
⚠️ [chromium-library] › library/inspector/cli-codegen-1.spec.ts:502:7 › cli codegen › should check with keyboard
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/selector-generator.spec.ts:527:5 › selector generator › should generate relative selector
⚠️ [chromium-library] › library/trace-viewer.spec.ts:264:1 › should have network request overrides 2
⚠️ [chromium-library] › library/video.spec.ts:381:5 › screencast › should capture navigation
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/inspector/cli-codegen-1.spec.ts:23:7 › cli codegen › should click
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [chromium-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [firefox-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [webkit-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [webkit-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [webkit-library] › library/inspector/cli-codegen-1.spec.ts:553:7 › cli codegen › should select
⚠️ [webkit-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [webkit-library] › library/inspector/cli-codegen-1.spec.ts:341:7 › cli codegen › should press
⚠️ [webkit-library] › library/popup.spec.ts:115:3 › should inherit viewport size from browser context
⚠️ [webkit-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [webkit-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [webkit-library] › library/inspector/cli-codegen-1.spec.ts:106:7 › cli codegen › should make a positioned click on a canvas
⚠️ [webkit-page] › page/elementhandle-misc.spec.ts:20:3 › should hover
⚠️ [webkit-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [webkit-library] › library/browsercontext-viewport.spec.ts:145:12 › should drag with high dpi
⚠️ [webkit-library] › library/inspector/cli-codegen-1.spec.ts:76:7 › cli codegen › should click after same-document navigation
⚠️ [webkit-library] › library/inspector/cli-codegen-1.spec.ts:341:7 › cli codegen › should press
⚠️ [webkit-library] › library/inspector/cli-codegen-2.spec.ts:107:7 › cli codegen › should upload a single file
⚠️ [webkit-library] › library/inspector/cli-codegen-3.spec.ts:383:7 › cli codegen › should generate getByTestId
⚠️ [webkit-page] › page/workers.spec.ts:243:3 › should support offline
⚠️ [webkit-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [webkit-library] › library/tracing.spec.ts:412:14 › should produce screencast frames scale
⚠️ [webkit-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [webkit-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture
⚠️ [webkit-library] › library/browsercontext-fetch-happy-eyeballs.spec.ts:49:3 › get should work on request fixture

206326 passed, 9230 skipped, 1390 did not run
✔️✔️✔️

Merge workflow run.

mxschmitt marked this pull request as draft June 25, 2024 14:15

This comment has been minimized.

Sign in to view

mxschmitt force-pushed the bugfix/electron-pwt branch from b2a2232 to a49e958 Compare June 27, 2024 18:55

mxschmitt marked this pull request as ready for review June 27, 2024 18:56

fix(electron): tracing with @playwright/test

a7b0496

mxschmitt force-pushed the bugfix/electron-pwt branch from a49e958 to a7b0496 Compare June 27, 2024 19:07

This comment has been minimized.

Sign in to view

pavelfeldman approved these changes Jul 1, 2024

View reviewed changes

mxschmitt added 2 commits July 1, 2024 18:52

add logging

8f67ee1

Revert "add logging"

061e90d

This reverts commit 8f67ee1.

This comment has been minimized.

Sign in to view

mxschmitt added the CQ1 label Jul 1, 2024

mxschmitt merged commit 9a3e096 into microsoft:main Jul 1, 2024
101 of 107 checks passed

mxschmitt added a commit that referenced this pull request Jul 1, 2024

cherry-pick(#31437): fix(electron): tracing with @playwright/test

0fe7a10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(electron): tracing with @playwright/test #31437

fix(electron): tracing with @playwright/test #31437

mxschmitt commented Jun 25, 2024 •

edited by pavelfeldman

Loading

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

pavelfeldman commented Jun 28, 2024

This comment has been minimized.

github-actions bot commented Jul 1, 2024

github-actions bot commented Jul 1, 2024

github-actions bot commented Jul 1, 2024

fix(electron): tracing with @playwright/test #31437

fix(electron): tracing with @playwright/test #31437

Conversation

mxschmitt commented Jun 25, 2024 • edited by pavelfeldman Loading

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

pavelfeldman commented Jun 28, 2024

This comment has been minimized.

github-actions bot commented Jul 1, 2024

Test results for "tests 1"

github-actions bot commented Jul 1, 2024

Test results for "tests others"

github-actions bot commented Jul 1, 2024

Test results for "tests 2"

mxschmitt commented Jun 25, 2024 •

edited by pavelfeldman

Loading