This benchmark suite provides comprehensive performance comparisons of ES Module Shims versus native modules across browsers.
- ES Module Shims Chrome Passthrough results in ~5ms extra initialization time over native for ES Module Shims fetching, execution and initialization, and on a slow connection the additional non-blocking bandwidth cost of its 10.5KB download as expected.
- ES Module Shims Polyfilling is on average 1.4x - 1.5x slower than native module loading, and up to 1.8x slower on slow networks (most likely due to the browser preloader), both for cached and uncached loads, and this result scales linearly.
- Very large import maps (100s of entries) cost only a few extra milliseconds upfront for the additional loading cost.
The test being benchmarked is loading and executing n versions of Preact plus a hello world component, where each n corresponds to a unique instance of Preact. This way real work is being benchmarked and not just some hypothetical n module load test but actual non-trivial execution is also associated with each module load to more closely resemble a real world scenario.
The base test varies these loads from 10 to 100 modules (separate Preact + hello world component for each n being rendered). n=10 corrsponds to 100KB of code being loaded and executed (10 unique instances of the 10KB Preact with the small component being rendered), and n=100 corresponds to 1MB of 100 module component / Preact pairs loading and executing in parallel. In addition a 1000 module load case (10MB) is also included for stress testing.
Each test includes the full network cost of loading the initial HTML by wrapping the top-level runner to call the network server for the actual test start. This way all benchmarks include the full cost of loading and initializing ES module shims itself.
Tests are run via Tachometer which then starts benchmarking and loads an IFrame to a custom HTTP/2 server. This ensures full end-to-end performance is benchmarked.
Network variations include:
- Fastest: Fully cold loads, running at maximum throughput over the Node.js HTTP/2 server.
- Throttled: 750Kb/s / 25ms latency throttling with Brotli enabled, handled by the Node.js HTTP/2 server.
Browser tests done on a modern Windows machine with versions:
- Chrome: 100.0.4896.60
- Firefox: 98.0.2
When running in latest Chromium browser (~70% of users), ES Module Shims fully delegates to the native loader, and users will get native-level performance.
These benchmarks verify this by comparing page loads with and without ES Module Shims, where the benchmark includes the time it takes to load ES Module Shims, run the feature detections, then have importShim() delegate to the native loader.
| n | Chrome Import Maps (ms) | Chrome Import Maps ES Module Shims (ms) | Difference (ms, x) |
|---|---|---|---|
| 10 (100 KB) | 54 (51 - 58) | 62 (59 - 66) | 8 (1.15x) |
| 20 (200 KB) | 75 (72 - 78) | 81 (79 - 84) | 7 (1.09x) |
| 30 (300 KB) | 96 (94 - 98) | 102 (99 - 104) | 6 (1.06x) |
| 40 (400 KB) | 119 (115 - 123) | 125 (123 - 131) | 6 (1.05x) |
| 50 (500 KB) | 140 (135 - 145) | 147 (143 - 157) | 7 (1.05x) |
| 60 (600 KB) | 162 (158 - 175) | 168 (164 - 179) | 6 (1.04x) |
| 70 (700 KB) | 182 (180 - 185) | 190 (185 - 204) | 8 (1.04x) |
| 80 (800 KB) | 202 (199 - 206) | 208 (203 - 215) | 6 (1.03x) |
| 90 (900 KB) | 225 (217 - 236) | 231 (225 - 242) | 6 (1.03x) |
| 100 (1000 KB) | 246 (241 - 256) | 251 (247 - 257) | 5 (1.02x) |
| n | Chrome Throttled Import Maps (ms) | Chrome Throttled Import Maps ES Module Shims (ms) | Difference (ms, x) |
|---|---|---|---|
| 10 (100 KB) | 149 (143 - 164) | 168 (161 - 171) | 19 (1.13x) |
| 20 (200 KB) | 207 (194 - 210) | 209 (207 - 216) | 3 (1.01x) |
| 30 (300 KB) | 254 (246 - 269) | 270 (267 - 272) | 16 (1.06x) |
| 40 (400 KB) | 311 (304 - 320) | 317 (310 - 327) | 6 (1.02x) |
| 50 (500 KB) | 364 (358 - 415) | 362 (357 - 370) | -2 (1x) |
| 60 (600 KB) | 418 (416 - 427) | 418 (414 - 428) | 0 (1x) |
| 70 (700 KB) | 475 (469 - 488) | 481 (470 - 492) | 6 (1.01x) |
| 80 (800 KB) | 548 (539 - 554) | 547 (539 - 552) | -1 (1x) |
| 90 (900 KB) | 603 (597 - 611) | 606 (597 - 610) | 3 (1x) |
| 100 (1000 KB) | 662 (659 - 670) | 666 (660 - 672) | 4 (1.01x) |
To compare native v polyfill performance we compare execution without import maps to execution with import maps and ES Module Shims in Firefox and Safari (which don't support import maps).
| n | Firefox (ms) | Firefox Import Maps ES Module Shims (ms) | Difference (ms, x) |
|---|---|---|---|
| 10 (100 KB) | 76 (46 - 114) | 132 (118 - 171) | 56 (1.74x) |
| 20 (200 KB) | 109 (100 - 124) | 186 (138 - 218) | 77 (1.71x) |
| 30 (300 KB) | 111 (78 - 133) | 187 (158 - 234) | 76 (1.68x) |
| 40 (400 KB) | 134 (101 - 153) | 206 (164 - 248) | 71 (1.53x) |
| 50 (500 KB) | 145 (112 - 168) | 244 (200 - 272) | 99 (1.68x) |
| 60 (600 KB) | 158 (123 - 207) | 237 (208 - 269) | 78 (1.5x) |
| 70 (700 KB) | 171 (143 - 195) | 259 (238 - 294) | 88 (1.52x) |
| 80 (800 KB) | 185 (150 - 218) | 285 (257 - 356) | 100 (1.54x) |
| 90 (900 KB) | 205 (179 - 233) | 303 (278 - 338) | 98 (1.48x) |
| 100 (1000 KB) | 219 (191 - 248) | 319 (297 - 373) | 100 (1.46x) |
| n | Firefox Throttled (ms) | Firefox Throttled Import Maps ES Module Shims (ms) | Difference (ms, x) |
|---|---|---|---|
| 10 (100 KB) | 203 (196 - 213) | 253 (241 - 297) | 50 (1.24x) |
| 20 (200 KB) | 254 (245 - 266) | 303 (287 - 346) | 49 (1.19x) |
| 30 (300 KB) | 308 (297 - 323) | 411 (394 - 420) | 103 (1.33x) |
| 40 (400 KB) | 357 (347 - 364) | 409 (390 - 455) | 52 (1.15x) |
| 50 (500 KB) | 418 (411 - 433) | 461 (455 - 506) | 44 (1.11x) |
| 60 (600 KB) | 471 (463 - 488) | 516 (507 - 567) | 45 (1.1x) |
| 70 (700 KB) | 525 (515 - 539) | 573 (555 - 604) | 48 (1.09x) |
| 80 (800 KB) | 585 (577 - 599) | 630 (608 - 645) | 46 (1.08x) |
| 90 (900 KB) | 640 (635 - 647) | 678 (669 - 688) | 38 (1.06x) |
| 100 (1000 KB) | 692 (684 - 711) | 742 (732 - 754) | 50 (1.07x) |
Import maps lie on the critical load path of an application - in theory a large import map might be a performance concern as it would delay module loading.
The large import map variation uses a mapping for every module in the case. So n = 10 corresponds two 20 separate import map entries, n = 20 to 40 etc.
To investigate this, we run on a throttled connection using native import maps in Chrome.
Running on Chrome native, we can see this slowdown very slightly, although it is within the test variance and far more minimal than might have been expected in comparison to the test variation.
| n | Chrome Throttled Import Maps (ms) | Chrome Throttled Individual Import Maps (ms) | Difference (ms, x) |
|---|---|---|---|
| 10 (100 KB) | 149 (143 - 164) | 156 (146 - 167) | 7 (1.04x) |
| 20 (200 KB) | 207 (194 - 210) | 202 (194 - 213) | -5 (0.98x) |
| 30 (300 KB) | 254 (246 - 269) | 265 (251 - 270) | 12 (1.05x) |
| 40 (400 KB) | 311 (304 - 320) | 313 (305 - 327) | 2 (1.01x) |
| 50 (500 KB) | 364 (358 - 415) | 368 (361 - 376) | 4 (1.01x) |
| 60 (600 KB) | 418 (416 - 427) | 422 (418 - 430) | 4 (1.01x) |
| 70 (700 KB) | 475 (469 - 488) | 482 (476 - 498) | 7 (1.01x) |
| 80 (800 KB) | 548 (539 - 554) | 552 (546 - 560) | 4 (1.01x) |
| 90 (900 KB) | 603 (597 - 611) | 609 (605 - 611) | 6 (1.01x) |
| 100 (1000 KB) | 662 (659 - 670) | 671 (667 - 678) | 9 (1.01x) |
To compare the polyfill case, we run the same comparison in Firefox comparing the use of a single path mapping with the individual mappings:
| n | Firefox Throttled Import Maps ES Module Shims (ms) | Firefox Throttled Individual Import Maps ES Module Shims (ms) | Difference (ms, x) |
|---|---|---|---|
| 10 (100 KB) | 253 (241 - 297) | 259 (242 - 297) | 6 (1.02x) |
| 20 (200 KB) | 303 (287 - 346) | 316 (292 - 355) | 13 (1.04x) |
| 30 (300 KB) | 411 (394 - 420) | 417 (398 - 426) | 6 (1.01x) |
| 40 (400 KB) | 409 (390 - 455) | 417 (401 - 462) | 8 (1.02x) |
| 50 (500 KB) | 461 (455 - 506) | 470 (458 - 517) | 9 (1.02x) |
| 60 (600 KB) | 516 (507 - 567) | 529 (514 - 589) | 13 (1.02x) |
| 70 (700 KB) | 573 (555 - 604) | 581 (563 - 629) | 8 (1.01x) |
| 80 (800 KB) | 630 (608 - 645) | 632 (614 - 671) | 2 (1x) |
| 90 (900 KB) | 678 (669 - 688) | 698 (681 - 749) | 20 (1.03x) |
| 100 (1000 KB) | 742 (732 - 754) | 761 (743 - 799) | 18 (1.02x) |
We again see the very slight slowdown of a similar order to native as expected.