Release performance tests are failing since v14.8.0

[WunderBart]

The release performance tests have been failing since 14.8.0 RC (Dec 14). Let's fix that.

/cc @youknowriad @dmsnell

[WunderBart]

Looks like the manual dispatch (custom perf comparison) jobs are failing as well: https://github.com/WordPress/gutenberg/actions/workflows/performance.yml?query=event%3Aworkflow_dispatch

[dmsnell]

I'm getting these errors when running the tests but I'm not certain if those are because I'm running them over an SSH connection to my laptop. I'll confirm later.

Failed to fire hook: while creating logrus local file hook: user: Current requires cgo or $USER set in environment

WARNING: Some service image(s) must be built from source by running:
    docker compose build %s wordpress

[dmsnell]

On a separate server I don't see those errors, but the tests in trunk fail here:

Error: ENOENT: no such file or directory, copyfile '/tmp/ef9cab1c-9634-4822-99ef-5db9b93a8751/tests/packages/e2e-tests/specs/performance/post-editor.test.results.json' -> './__test-results/tests/measure-perf-test-runtimes_post-editor_run-0.results.json'
    at Object.copyFileSync (node:fs:2884:3)
    at runTestSuite (/home/exouser/gutenberg/bin/plugin/commands/performance.js:208:5)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async runPerformanceTests (/home/exouser/gutenberg/bin/plugin/commands/performance.js:414:33)
    at async Command.<anonymous> (/home/exouser/gutenberg/bin/plugin/cli.js:11:4) {
  errno: -2,
  syscall: 'copyfile',
  code: 'ENOENT',
  path: '/tmp/ef9cab1c-9634-4822-99ef-5db9b93a8751/tests/packages/e2e-tests/specs/performance/post-editor.test.results.json',
  dest: './__test-results/tests/measure-perf-test-runtimes_post-editor_run-0.results.json'
}

[WunderBart]

I'm getting these errors when running the tests but I'm not certain if those are because I'm running them over an SSH connection to my laptop. I'll confirm later.

Failed to fire hook: while creating logrus local file hook: user: Current requires cgo or $USER set in environment

WARNING: Some service image(s) must be built from source by running:
    docker compose build %s wordpress

Did you get to resolve it? I'm actually facing the same issue

[WunderBart]

I've triggered a release job manually from the refactor/perf-test-results-path branch (PR #48684) and got more info about the issue:

FAIL packages/e2e-tests/specs/performance/post-editor.test.js (219.513 s)


  ● Test suite failed to run

    ENOENT: no such file or directory, open '/home/runner/work/gutenberg/gutenberg/artifacts/post-editor_wp/6.1_run-0.performance-results.json'

      26 | 		path.basename( testFilename, '.js' ) + '.performance-results.json';
      27 |
    > 28 | 	return writeFileSync(
         | 	       ^
      29 | 		path.join( process.env.WP_ARTIFACTS_PATH, resultsFilename ),
      30 | 		JSON.stringify( results, null, 2 )
      31 | 	);

Looks like it's caused by branch names like release/15.2 that break the filename because the slash is not escaped. I'll see if kebab-casing it will suffice.

[WunderBart]

Sanitizing the branch name to be used in a path did the trick.

Let me know if we should extract that fix from #48684. It is related to other changes in that PR, so it wouldn't be unreasonable to keep it there, IMO.

/cc @youknowriad

Edit: also added the fix in #49068