Investigate and implement a solution for readFileSync of static files (known at build time)

[dario-piotrowicz]

Intro

The repository current goal is to get a minimal hello-world Next.js api app running using wrangler dev.

At the time of writing this issue (current commit) the hello-world app is still not successfully running, but it should hopefully be close to it.

To get the application to the current state a series of temporary hacks/workaround were introduced, those should be cleaned up and re-implemented in a stable way.

This issue is for the removal of one of such hacks/workarounds.

The hack/workaround

There are files that get read by the Next.js server using readFileSync like the following:

• https://github.com/vercel/next.js/blob/15aeb92efb34c09a36779a42e5e01b7d62b34498/packages/next/src/server/next-server.ts#L441
• https://github.com/vercel/next.js/blob/15aeb92efb34c09a36779a42e5e01b7d62b34498/packages/next/src/server/load-manifest.ts#L44

These files are known during build time and (I presume) not supposed to change during the server's lifespan.

Since workerd doesn't support readFileSync we're currently taking the worker built code and replacing code using readFileSync to just return the files' contents directly (without any runtime reads):

poc-build-nextjs-app-for-cf-workers/nextjs-worker-builder/src/build/build-worker/index.ts

Lines 131 to 170 in d380bbf

	// The next-server code gets the buildId from the filesystem, resulting in a `[unenv] fs.readFileSync is not implemented yet!` error
	// so we add an early return to the `getBuildId` function so that the `readyFileSync` is never encountered
	// (source: https://github.com/vercel/next.js/blob/15aeb92efb34c09a36/packages/next/src/server/next-server.ts#L438-L451)
	// Note: we could/should probably just patch readFileSync here or something!
	updatedWorkerContents = updatedWorkerContents.replace(
	"getBuildId() {",
	`getBuildId() {
	return ${JSON.stringify(
	readFileSync(
	`${nextjsAppPaths.standaloneAppDotNextDir}/BUILD_ID`,
	"utf-8"
	)
	)};
	`
	);
	// Same as above, the next-server code loads the manifests with `readyFileSync` and we want to avoid that
	// (source: https://github.com/vercel/next.js/blob/15aeb92e/packages/next/src/server/load-manifest.ts#L34-L56)
	// Note: we could/should probably just patch readFileSync here or something!
	const manifestJsons = globSync(
	`${nextjsAppPaths.standaloneAppDotNextDir}/**/*-manifest.json`
	).map((file) => file.replace(nextjsAppPaths.standaloneAppDir + "/", ""));
	updatedWorkerContents = updatedWorkerContents.replace(
	/function loadManifest\((.+?), .+?\) {/,
	`$&
	${manifestJsons
	.map(
	(manifestJson) => `
	if ($1.endsWith("${manifestJson}")) {
	return ${readFileSync(
	`${nextjsAppPaths.standaloneAppDir}/${manifestJson}`,
	"utf-8"
	)};
	}
	`
	)
	.join("\n")}
	throw new Error("Unknown loadManifest: " + $1);
	`
	);

We should investigate a more proper solution for this as the sort of code replacement that we have in place now is very brittle.

(I would imagine that we could patch/polyfill readFileSync so that it could return know file contents and throw otherwise)

[dario-piotrowicz]

evalManifest is another function that relies on readFileSync:

poc-build-nextjs-app-for-cf-workers/nextjs-worker-builder/src/build/build-worker/index.ts

Lines 305 to 306 in ce26f03

	// `evalManifest` relies on readFileSync so we need to patch the function so that it instead returns the content of the manifest files
	// which are known at build time

[vicb]

@vercel/nft might help to resolve asset file paths.