Repair benchmarks

[steveluscher]

Summary

Because we moved a build-time contstant to the top level in #3340, we can no longer run these scripts without replacing the build time constants. That means that running this with tsx will yield a __NODEJS__ is not defined error.

In this PR, we change the benchmarks to run on the built source, and teach Turborepo to build it beforehand. You now run benchmarks with pnpm turbo benchmark.

[changeset-bot]

⚠️ No Changeset found

Latest commit: 97d27f9

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[steveluscher]

• Repair benchmarks #3347 👈
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @steveluscher and the rest of your teammates on Graphite

[steveluscher]

Unless anyone knows of a better way to run a script with tsx and replace build-time constants on the fly?

[steveluscher]

These ignore directives imply that these benchmarks will break frequently without warning, but I don't know of any other way to:

1. Refer to the built source
2. Before it exists
3. In a way that will typecheck

[mcintyre94]

Could we run these benchmarks in the CI, perhaps only on main if they're expensive? Just so we know if we break them, rather than having to debug when we need to run them and find them bust.

[steveluscher]

I figured it out.

[github-actions]

Because there has been no activity on this PR for 14 days since it was merged, it has been automatically locked. Please open a new issue if it requires a follow up.