Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only serialize and send shared references to workers that need them #8589

Merged
merged 16 commits into from
Nov 1, 2022

Conversation

devongovett
Copy link
Member

Closes #8491

This is based on #8491, and in addition to packaging on the main thread, it also transforms on the main thread if there is only one file change which improves performance even more.

It also refactors the code a little to follow a slightly different approach: rather than avoiding creating the shared reference and needing a different way of passing the bundle graph around, it changes the way shared references are implemented in the worker farm. Instead of serializing the bundle graph upfront and always sending it to every worker, it lazily serializes and sends shared references to workers only when needed. This way, if using the main thread, no serialization occurs but the shared reference can still be accessed using the same API. In addition, if you only have 2 bundles but 8 workers, we only send the bundle graph to the 2 workers that need it rather than all 8.

@@ -145,6 +145,7 @@ export type RunAPI = {|
getRequestResult<T>(contentKey: ContentKey): Async<?T>,
getPreviousResult<T>(ifMatch?: string): Async<?T>,
getSubRequests(): Array<StoredRequest>,
getInvalidSubRequests(): Array<StoredRequest>,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new API is subtly different than api.getSubRequest().filter(req => !api.canSkipSubrequest(req.id)) in that it doesn't have the side effect of preserving all previous sub requests even if not used.

@@ -110,7 +110,7 @@ export default class Parcel {
await resolvedOptions.cache.ensure();

let {dispose: disposeOptions, ref: optionsRef} =
await this.#farm.createSharedReference(resolvedOptions);
await this.#farm.createSharedReference(resolvedOptions, false);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Serializing the options aren't cacheable because of the MemoryFS which has a side effect in the serialize() method... 😬

@parcel-benchmark
Copy link

parcel-benchmark commented Oct 31, 2022

Benchmark Results

Kitchen Sink ✅

Timings

Description Time Difference
Cold 1.91s -32.00ms
Cached 324.00ms -10.00ms

Cold Bundles

Bundle Size Difference Time Difference
dist/legacy/parcel.7cdb0fad.webp 102.94kb +0.00b 105.00ms -207.00ms 🚀
dist/legacy/parcel.7cdb0fad.webp 102.94kb +0.00b 106.00ms -207.00ms 🚀
dist/modern/parcel.7cdb0fad.webp 102.94kb +0.00b 105.00ms -207.00ms 🚀

Cached Bundles

No bundle changes detected.

React HackerNews ✅

Timings

Description Time Difference
Cold 12.24s +184.00ms
Cached 563.00ms +7.00ms

Cold Bundles

No bundle changes detected.

Cached Bundles

No bundle changes detected.

AtlasKit Editor ✅

Timings

Description Time Difference
Cold 2.10m -4.07s
Cached 2.66s +35.00ms

Cold Bundles

Bundle Size Difference Time Difference
dist/pl.4643976c.js 2.37kb +0.00b 54.85s -6.70s 🚀
dist/table.71e953fa.js 1.08kb +0.00b 50.94s -3.91s 🚀
dist/simpleHasher.28ade3b5.js 742.00b +0.00b 1.01m -34.46s 🚀

Cached Bundles

Bundle Size Difference Time Difference
dist/ConfigPanelFieldsLoader.b6029f4e.js 305.39kb +0.00b 56.87s +2.99s ⚠️
dist/DatePicker.52462510.js 46.99kb +0.00b 56.87s +2.99s ⚠️
dist/DatePicker.31948e54.js 27.22kb +0.00b 56.87s +2.99s ⚠️
dist/ResourcedEmojiComponent.bef36b85.js 2.68kb +0.00b 56.87s +2.99s ⚠️
dist/cs.08737142.js 2.28kb +0.00b 56.87s +2.99s ⚠️
dist/de.c2e79abd.js 2.26kb +0.00b 56.88s +3.01s ⚠️
dist/es.983af340.js 2.24kb +0.00b 56.88s +3.01s ⚠️
dist/ja.b56014f3.js 2.24kb +0.00b 56.89s +3.02s ⚠️
dist/fr.aaa4d0bf.js 2.20kb +0.00b 56.89s +3.01s ⚠️
dist/hu.3d2e30a0.js 2.17kb +0.00b 56.89s +3.01s ⚠️
dist/fi.080f52aa.js 2.13kb +0.00b 56.89s +3.01s ⚠️
dist/ko.51863560.js 2.13kb +0.00b 56.89s +3.02s ⚠️
dist/it.7bb93510.js 2.12kb +0.00b 56.89s +3.01s ⚠️
dist/nb.67163f41.js 2.10kb +0.00b 56.89s +3.02s ⚠️
dist/nl.336a2549.js 2.09kb +0.00b 56.89s +3.02s ⚠️
dist/da.f39b0c8c.js 2.07kb +0.00b 56.88s +3.01s ⚠️
dist/feedback.2ade51a8.js 1.92kb +0.00b 56.87s +2.99s ⚠️
dist/heading6.e0c2f3cf.js 1.52kb +0.00b 56.87s +2.99s ⚠️
dist/heading3.5b4663e8.js 1.51kb +0.00b 56.87s +2.99s ⚠️
dist/heading5.df611011.js 1.39kb +0.00b 56.87s +2.99s ⚠️
dist/expand.655beb16.js 1.33kb +0.00b 56.87s +2.99s ⚠️
dist/heading2.54f14b16.js 1.33kb +0.00b 56.87s +2.99s ⚠️
dist/heading4.3ae44c0a.js 1.28kb +0.00b 56.87s +2.99s ⚠️
dist/heading1.216107b4.js 1.17kb +0.00b 56.87s +2.99s ⚠️
dist/et.03b90e09.js 778.00b +0.00b 56.88s +3.01s ⚠️
dist/simpleHasher.28ade3b5.js 742.00b +0.00b 1.05m -30.19s 🚀
dist/is.0d0b2897.js 638.00b +0.00b 56.89s +3.01s ⚠️
dist/en_GB.a4eaa606.js 623.00b +0.00b 56.88s +3.01s ⚠️
dist/en.dced70ab.js 620.00b +0.00b 56.88s +3.01s ⚠️

Three.js ✅

Timings

Description Time Difference
Cold 8.70s +6.00ms
Cached 348.00ms -19.00ms 🚀

Cold Bundles

No bundle changes detected.

Cached Bundles

No bundle changes detected.

Click here to view a detailed benchmark overview.

@devongovett devongovett merged commit 3480705 into v2 Nov 1, 2022
@devongovett devongovett deleted the single-thread branch November 1, 2022 04:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants