Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

more efficient node.js process startup #7661

Merged
merged 2 commits into from
Mar 13, 2024
Merged

more efficient node.js process startup #7661

merged 2 commits into from
Mar 13, 2024

Conversation

sokra
Copy link
Member

@sokra sokra commented Mar 7, 2024

Description

  • Slow start for the concurrency of parallel booting node.js processes (1, 2, 4, ...)
  • Depending on historic statistics avoid booting up a new process, but wait for one to finish computing

Loading the /home route on 16-core machine

Before:

16 x Node.js process startup 9.09s avg 568.07ms
16 x Node.js initialization 10.58s avg 660.96ms
276 x Node.js evaluation 30.17s avg 109.30ms

After:

7 x Node.js process startup 1261.79ms avg 180.26ms
7 x Node.js initialization 3.15s avg 450.21ms
276 x Node.js evaluation 29.41s avg 106.55ms

Testing Instructions

Closes PACK-2680

Copy link

vercel bot commented Mar 7, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
examples-native-web 🔄 Building (Inspect) Visit Preview 💬 Add feedback Mar 7, 2024 2:21pm
examples-nonmonorepo ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 7, 2024 2:21pm
rust-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 7, 2024 2:21pm
turbo-site ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 7, 2024 2:21pm
7 Ignored Deployments
Name Status Preview Comments Updated (UTC)
examples-basic-web ⬜️ Ignored (Inspect) Visit Preview Mar 7, 2024 2:21pm
examples-designsystem-docs ⬜️ Ignored (Inspect) Visit Preview Mar 7, 2024 2:21pm
examples-gatsby-web ⬜️ Ignored (Inspect) Visit Preview Mar 7, 2024 2:21pm
examples-kitchensink-blog ⬜️ Ignored (Inspect) Visit Preview Mar 7, 2024 2:21pm
examples-svelte-web ⬜️ Ignored (Inspect) Visit Preview Mar 7, 2024 2:21pm
examples-tailwind-web ⬜️ Ignored (Inspect) Visit Preview Mar 7, 2024 2:21pm
examples-vite-web ⬜️ Ignored (Inspect) Visit Preview Mar 7, 2024 2:21pm

Copy link
Contributor

github-actions bot commented Mar 7, 2024

🟢 Turbopack Benchmark CI successful 🟢

Thanks

@sokra sokra marked this pull request as ready for review March 7, 2024 13:23
@sokra sokra requested a review from a team as a code owner March 7, 2024 13:23
Copy link
Contributor

github-actions bot commented Mar 7, 2024

⚠️ This change may fail to build next-swc.

Logs

error: failed to select a version for `turbopack-binding`.
    ... required by package `next-core v0.1.0 (/root/actions-runner/_work/turbo/turbo/packages/next-swc/crates/next-core)`
    ... which satisfies path dependency `next-core` (locked to 0.1.0) of package `next-swc-napi v0.0.0 (/root/actions-runner/_work/turbo/turbo/packages/next-swc/crates/napi)`
versions that meet the requirements `*` (locked to 0.1.0) are: 0.1.0

the package `next-core` depends on `turbopack-binding`, with features: `__turbopack_build` but `turbopack-binding` does not have these features.


failed to select a version for `turbopack-binding` which could resolve this conflict

See job summary for details

Copy link
Contributor

github-actions bot commented Mar 7, 2024

⚠️ CI failed ⚠️

The following steps have failed in CI:

  • Turbopack Rust tests (mac/win, non-blocking)

See workflow summary for details

let expected_completed_task_until_full_workers = {
let millis = max(1, warm_process_time.as_millis());
let ready_workers = workers - booting_workers;
(expected_full_workers_in.as_millis() / millis) as u32 * ready_workers
Copy link
Member

@kdy1 kdy1 Mar 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it better to multiply first before dividing? (Not important as the numbers are millis, which are not very small)

crates/turbopack-node/src/pool.rs Show resolved Hide resolved
@sokra sokra merged commit a139056 into main Mar 13, 2024
48 of 51 checks passed
@sokra sokra deleted the sokra/node-js-bootup branch March 13, 2024 08:12
kdy1 added a commit to vercel/next.js that referenced this pull request Mar 13, 2024
# Turbopack

* vercel/turborepo#7719 <!-- Tobias Koppers - accept
css files outside of the project as virtual assets -->
* vercel/turborepo#7661 <!-- Tobias Koppers - more
efficient node.js process startup -->
* vercel/turborepo#7720 <!-- Tobias Koppers -
generate correct async module handling for side effects optimization -->
* vercel/turborepo#7718 <!-- Donny/강동윤 -
fix(turbopack): Fix CSS Modules class name for lightningcss mode -->

### What?

Update tests to make CSS Modules of `lightningcss` work with CSS grids.

### Why?

`lightningcss` enforces the class name to end with `[local]`.

See: https://lightningcss.dev/css-modules.html#css-grid


### How?

Closes PACK-2731
ForsakenHarmony pushed a commit to vercel/next.js that referenced this pull request Jul 25, 2024
### Description

* Slow start for the concurrency of parallel booting node.js processes
(1, 2, 4, ...)
* Depending on historic statistics avoid booting up a new process, but
wait for one to finish computing

Loading the `/home` route on 16-core machine

Before:

```
16 x Node.js process startup 9.09s avg 568.07ms
16 x Node.js initialization 10.58s avg 660.96ms
276 x Node.js evaluation 30.17s avg 109.30ms
```

After:

```
7 x Node.js process startup 1261.79ms avg 180.26ms
7 x Node.js initialization 3.15s avg 450.21ms
276 x Node.js evaluation 29.41s avg 106.55ms
```


### Testing Instructions

<!--
  Give a quick description of steps to test your changes.
-->


Closes PACK-2680
ForsakenHarmony pushed a commit to vercel/next.js that referenced this pull request Jul 29, 2024
### Description

* Slow start for the concurrency of parallel booting node.js processes
(1, 2, 4, ...)
* Depending on historic statistics avoid booting up a new process, but
wait for one to finish computing

Loading the `/home` route on 16-core machine

Before:

```
16 x Node.js process startup 9.09s avg 568.07ms
16 x Node.js initialization 10.58s avg 660.96ms
276 x Node.js evaluation 30.17s avg 109.30ms
```

After:

```
7 x Node.js process startup 1261.79ms avg 180.26ms
7 x Node.js initialization 3.15s avg 450.21ms
276 x Node.js evaluation 29.41s avg 106.55ms
```


### Testing Instructions

<!--
  Give a quick description of steps to test your changes.
-->


Closes PACK-2680
ForsakenHarmony pushed a commit to vercel/next.js that referenced this pull request Jul 29, 2024
### Description

* Slow start for the concurrency of parallel booting node.js processes
(1, 2, 4, ...)
* Depending on historic statistics avoid booting up a new process, but
wait for one to finish computing

Loading the `/home` route on 16-core machine

Before:

```
16 x Node.js process startup 9.09s avg 568.07ms
16 x Node.js initialization 10.58s avg 660.96ms
276 x Node.js evaluation 30.17s avg 109.30ms
```

After:

```
7 x Node.js process startup 1261.79ms avg 180.26ms
7 x Node.js initialization 3.15s avg 450.21ms
276 x Node.js evaluation 29.41s avg 106.55ms
```


### Testing Instructions

<!--
  Give a quick description of steps to test your changes.
-->


Closes PACK-2680
ForsakenHarmony pushed a commit to vercel/next.js that referenced this pull request Aug 1, 2024
### Description

* Slow start for the concurrency of parallel booting node.js processes
(1, 2, 4, ...)
* Depending on historic statistics avoid booting up a new process, but
wait for one to finish computing

Loading the `/home` route on 16-core machine

Before:

```
16 x Node.js process startup 9.09s avg 568.07ms
16 x Node.js initialization 10.58s avg 660.96ms
276 x Node.js evaluation 30.17s avg 109.30ms
```

After:

```
7 x Node.js process startup 1261.79ms avg 180.26ms
7 x Node.js initialization 3.15s avg 450.21ms
276 x Node.js evaluation 29.41s avg 106.55ms
```


### Testing Instructions

<!--
  Give a quick description of steps to test your changes.
-->


Closes PACK-2680
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants