-
Notifications
You must be signed in to change notification settings - Fork 26.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve compile time on large application #50792
Improve compile time on large application #50792
Conversation
Current dependencies on/for this PR:
This comment was auto-generated by Graphite. |
Allow CI Workflow Run
Note: this should only be enabled once the PR is ready to go and can only be enabled by a maintainer |
a57540d
to
4ea9575
Compare
## What? While investigating slow compilation for a page on vercel.com in development I found that there was close to 10 seconds of time unaccounted for in `.next/trace`. Ran a profile and found that time was spent in watchpack `batch`, specifically calling `close` many times. When I tried to debug this further by running unbundled webpack I noticed the same issue didn't exists. ### Before <img width="1329" alt="Before" src="https://github.com/vercel/next.js/assets/6324199/9ace4628-db04-4de7-993f-65aef9dffc55"> ### After <img width="1278" alt="After" src="https://github.com/vercel/next.js/assets/6324199/55d5e58b-4a27-4235-8dea-723a7a78c117"> ## Raw numbers <table> <tr> <td>Before</td> <td>After</td> <td>Delta</td> <td>Delta (percent)</td> </tr> <tr> <td>13840 ms</td> <td>3580 ms</td> <td>-10260 ms</td> <td>-74.13%</td> </tr> </table> ## How? Investigated further and found that specifically not minifying watchpack solved the issue. <!-- Thanks for opening a PR! Your contribution is much appreciated. To make sure your PR is handled as smoothly as possible we request that you follow the checklist sections below. Choose the right checklist for the change(s) that you're making: ## For Contributors ### Improving Documentation or adding/fixing Examples - The "examples guidelines" are followed from our contributing doc https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md - Make sure the linting passes by running `pnpm build && pnpm lint`. See https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md ### Fixing a bug - Related issues linked using `fixes #number` - Tests added. See: https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs - Errors have a helpful link attached, see https://github.com/vercel/next.js/blob/canary/contributing.md ### Adding a feature - Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR. (A discussion must be opened, see https://github.com/vercel/next.js/discussions/new?category=ideas) - Related issues/discussions are linked using `fixes #number` - e2e tests added (https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs - Documentation added - Telemetry added. In case of a feature if it's used or not. - Errors have a helpful link attached, see https://github.com/vercel/next.js/blob/canary/contributing.md ## For Maintainers - Minimal description (aim for explaining to someone not on the team to understand the PR) - When linking to a Slack thread, you might want to share details of the conclusion - Link both the Linear (Fixes NEXT-xxx) and the GitHub issues - Add review comments if necessary to explain to the reviewer the logic behind a change ### What? ### Why? ### How? Closes NEXT- Fixes # -->
We noticed many fs close calls in our app and switched to polling in watchpack(since it does not relies on fs watcher). Polling is still faster (~12s now vs ~20s in watcher) in our app even after this PR's changes. We added this to next.config.js
This issue likely appeared in node 14 onwards nodejs/node#29949, not sure how precompiling watchpack is increasing fs call. |
Update: we found the root cause: #51826
What?
While investigating slow compilation for a page on vercel.com in development I found that there was close to 10 seconds of time unaccounted for in
.next/trace
. Ran a profile and found that time was spent in watchpackbatch
, specifically callingclose
many times. When I tried to debug this further by running unbundled webpack I noticed the same issue didn't exists.Before
After
Raw numbers
How?
Investigated further and found that specifically not minifying watchpack solved the issue.