Evaluates Sentry & Replay impact on website performance by running a web app in Chromium via Playwright and collecting various metrics.
The general idea is to run a web app without Sentry, and then run the same app again with Sentry and another one with Sentry+Replay included. For the three scenarios, we collect some metrics (CPU, memory, vitals) and later compare them and post as a comment in a PR. Changes in the metrics, compared to previous runs from the main branch, should be evaluated on case-by-case basis when preparing and reviewing the PR.
The web-benchmark
script will instanciate all classes that are exported from your scenario module and call the run()
method in each scenario run. The run()
method is called with two arguments: browser and page. If you are familiar with Playwright you can go ahead a write a benchmarking scenario, otherwise, you can use Playwright's Codegen to assist in writing one.
Note: Only ESM is supported, your scenario must be in a .mjs
file.
export class MyBaseScenario {
async run(browser, page) {
await page.goto('https://sentry.io/welcome/')
await page.getByRole('navigation').getByRole('link', { name: 'Pricing' }).click()
}
}
To run:
npx @sentry-internal/web-benchmark -f myScenario.mjs
arg | required | default | description |
---|---|---|---|
--file / -f | true | The path to your scenario file | |
--headless | false | true | Run in headless mode |
--runs / -c | false | 1 | The number of times to run the scenario |
--cpu | false | 1 | The CPU throttle factor. e.g. "2" represents a 2x throttle |
--network | false | false | The network speed to emulate (e.g. "Fast 3G" or "Slow 3G") |