Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(puppeteer): support trace recording #3981

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions docs/helpers/Puppeteer.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ Type: [object][4]
- `disableScreenshots` **[boolean][20]?** don't save screenshot on failure.
- `fullPageScreenshots` **[boolean][20]?** make full page screenshots on failure.
- `uniqueScreenshotNames` **[boolean][20]?** option to prevent screenshot override if you have scenarios with the same name in different suites.
- `trace` **[boolean][20]?** record [tracing information][25] with screenshots.
- `keepTraceForPassedTests` **[boolean][20]?** save trace for passed tests.
- `keepBrowserState` **[boolean][20]?** keep browser state between tests when `restart` is set to false.
- `keepCookies` **[boolean][20]?** keep cookies between tests when `restart` is set to false.
- `waitForAction` **[number][11]?** how long to wait after click, doubleClick or PressKey actions in ms. Default: 100.
Expand All @@ -55,11 +57,19 @@ Type: [object][4]
- `userAgent` **[string][6]?** user-agent string.
- `manualStart` **[boolean][20]?** do not start browser before a test, start it manually inside a helper with `this.helpers["Puppeteer"]._startBrowser()`.
- `browser` **[string][6]?** can be changed to `firefox` when using [puppeteer-firefox][2].
- `chrome` **[object][4]?** pass additional [Puppeteer run options][25].
- `chrome` **[object][4]?** pass additional [Puppeteer run options][26].
- `highlightElement` **[boolean][20]?** highlight the interacting elements. Default: false. Note: only activate under verbose mode (--verbose).



#### Trace Recording Customization

Trace recording provides complete information on test execution and includes screenshots, and network requests logged during run.
Traces will be saved to `output/trace`

- `trace`: enables trace recording for failed tests; trace are saved into `output/trace` folder
- `keepTraceForPassedTests`: - save trace for passed tests

#### Example #1: Wait for 0 network connections.

```js
Expand Down Expand Up @@ -2263,4 +2273,6 @@ Returns **[Promise][7]<void>** automatically synchronized promise through #re

[24]: https://codecept.io/react

[25]: https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#puppeteerlaunchoptions
[25]: https://pptr.dev/api/puppeteer.tracing

[26]: https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#puppeteerlaunchoptions
47 changes: 45 additions & 2 deletions lib/helper/Puppeteer.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ const fsExtra = require('fs-extra');
const path = require('path');

const Helper = require('@codeceptjs/helper');
const { v4: uuidv4 } = require('uuid');
const Locator = require('../locator');
const recorder = require('../recorder');
const store = require('../store');
Expand All @@ -19,6 +20,7 @@ const {
fileExists,
chunkArray,
toCamelCase,
clearString,
convertCssPropertiesToCamelCase,
screenshotOutputFolder,
getNormalizedKeyAttributeValue,
Expand Down Expand Up @@ -57,6 +59,8 @@ const consoleLogStore = new Console();
* @prop {boolean} [disableScreenshots=false] - don't save screenshot on failure.
* @prop {boolean} [fullPageScreenshots=false] - make full page screenshots on failure.
* @prop {boolean} [uniqueScreenshotNames=false] - option to prevent screenshot override if you have scenarios with the same name in different suites.
* @prop {boolean} [trace=false] - record [tracing information](https://pptr.dev/api/puppeteer.tracing) with screenshots.
* @prop {boolean} [keepTraceForPassedTests=false] - save trace for passed tests.
* @prop {boolean} [keepBrowserState=false] - keep browser state between tests when `restart` is set to false.
* @prop {boolean} [keepCookies=false] - keep cookies between tests when `restart` is set to false.
* @prop {number} [waitForAction=100] - how long to wait after click, doubleClick or PressKey actions in ms. Default: 100.
Expand Down Expand Up @@ -92,6 +96,14 @@ const config = {};
*
* <!-- configuration -->
*
* #### Trace Recording Customization
*
* Trace recording provides complete information on test execution and includes screenshots, and network requests logged during run.
* Traces will be saved to `output/trace`
*
* * `trace`: enables trace recording for failed tests; trace are saved into `output/trace` folder
* * `keepTraceForPassedTests`: - save trace for passed tests
*
* #### Example #1: Wait for 0 network connections.
*
* ```js
Expand Down Expand Up @@ -281,8 +293,9 @@ class Puppeteer extends Helper {
}
}

async _before() {
async _before(test) {
this.sessionPages = {};
this.currentRunningTest = test;
recorder.retry({
retries: 3,
when: err => {
Expand Down Expand Up @@ -647,6 +660,14 @@ class Puppeteer extends Helper {
}
}

if (this.options.trace) {
const fileName = `${`${global.output_dir}${path.sep}trace${path.sep}${uuidv4()}_${clearString(this.currentRunningTest.title)}`.slice(0, 245)}.json`;
const dir = path.dirname(fileName);
if (!fileExists(dir)) fs.mkdirSync(dir);
await this.page.tracing.start({ screenshots: true, path: fileName });
this.currentRunningTest.artifacts.trace = fileName;
}

await this.page.goto(url, { waitUntil: this.options.waitForNavigation });

const performanceTiming = JSON.parse(await this.page.evaluate(() => JSON.stringify(window.performance.timing)));
Expand Down Expand Up @@ -1898,8 +1919,30 @@ class Puppeteer extends Helper {
}
}

async _failed() {
async _failed(test) {
await this._withinEnd();

if (this.options.trace) {
await this.page.tracing.stop();
const _traceName = this.currentRunningTest.artifacts.trace.replace('.json', '.failed.json');
fs.renameSync(this.currentRunningTest.artifacts.trace, _traceName);
test.artifacts.trace = _traceName;
}
}

async _passed(test) {
await this._withinEnd();

if (this.options.trace) {
await this.page.tracing.stop();
if (this.options.keepTraceForPassedTests) {
const _traceName = this.currentRunningTest.artifacts.trace.replace('.json', '.passed.json');
fs.renameSync(this.currentRunningTest.artifacts.trace, _traceName);
test.artifacts.trace = _traceName;
} else {
fs.unlinkSync(this.currentRunningTest.artifacts.trace);
}
}
}

/**
Expand Down
47 changes: 47 additions & 0 deletions test/helper/Puppeteer_test.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,15 @@ const path = require('path');

const puppeteer = require('puppeteer');

const fs = require('fs');
const TestHelper = require('../support/TestHelper');
const Puppeteer = require('../../lib/helper/Puppeteer');

const AssertionFailedError = require('../../lib/assert/error');
const webApiTests = require('./webapi');
const FileSystem = require('../../lib/helper/FileSystem');
const Secret = require('../../lib/secret');
const { deleteDir } = require('../../lib/utils');
global.codeceptjs = require('../../lib');

let I;
Expand Down Expand Up @@ -1037,3 +1039,48 @@ describe('Puppeteer (remote browser)', function () {
});
});
});

describe('Puppeteer - Trace', () => {
const test = { title: 'a failed test', artifacts: {} };
before(() => {
global.codecept_dir = path.join(__dirname, '/../data');
global.output_dir = path.join(`${__dirname}/../data/output`);

I = new Puppeteer({
url: siteUrl,
windowSize: '500x700',
show: false,
trace: true,
});
I._init();
return I._beforeSuite();
});

beforeEach(async () => {
webApiTests.init({
I, siteUrl,
});
deleteDir(path.join(global.output_dir, 'trace'));
return I._before(test).then(() => {
page = I.page;
browser = I.browser;
});
});

afterEach(async () => {
return I._after();
});

it('checks that trace is recorded', async () => {
await I.amOnPage('/');
await I.dontSee('this should be an error');
await I.click('More info');
await I.dontSee('this should be an error');
await I._failed(test);
assert(test.artifacts);
expect(Object.keys(test.artifacts)).to.include('trace');

assert.ok(fs.existsSync(test.artifacts.trace));
expect(test.artifacts.trace).to.include(path.join(global.output_dir, 'trace'));
});
});