Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Monomorphic flow nodes #57977

Merged
merged 17 commits into from
Apr 4, 2024
Merged

Monomorphic flow nodes #57977

merged 17 commits into from
Apr 4, 2024

Conversation

ahejlsberg
Copy link
Member

Experiment to observe the performance effects of monomorphic flow nodes.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Mar 28, 2024
@typescript-bot
Copy link
Collaborator

Looks like you're introducing a change to the public API surface area. If this includes breaking changes, please document them on our wiki's API Breaking Changes page.

Also, please make sure @DanielRosenwasser and @RyanCavanaugh are aware of the changes, just as a heads up.

@ahejlsberg
Copy link
Member Author

@typescript-bot test it

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 28, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top400 ✅ Started ✅ Results
user test this ✅ Started ✅ Results
run dt ✅ Started ✅ Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

Hey @ahejlsberg, the results of running the DT tests are ready.

Everything looks the same!

You can check the log here.

@typescript-bot
Copy link
Collaborator

@ahejlsberg Here are the results of running the user tests comparing main and refs/pull/57977/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,671k (± 0.01%) 296,457k (± 0.01%) +787k (+ 0.27%) 296,436k 296,486k p=0.005 n=6
Parse Time 2.66s (± 0.19%) 2.66s (± 0.24%) ~ 2.65s 2.67s p=0.386 n=6
Bind Time 0.83s (± 0.62%) 0.83s (± 1.60%) ~ 0.82s 0.85s p=0.277 n=6
Check Time 8.24s (± 0.25%) 8.21s (± 0.28%) -0.03s (- 0.38%) 8.18s 8.23s p=0.029 n=6
Emit Time 7.05s (± 0.77%) 7.02s (± 0.25%) ~ 6.99s 7.04s p=0.285 n=6
Total Time 18.79s (± 0.28%) 18.72s (± 0.10%) -0.06s (- 0.35%) 18.69s 18.75s p=0.007 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 193,888k (± 1.02%) 193,596k (± 0.97%) ~ 192,351k 196,132k p=0.378 n=6
Parse Time 1.64s (± 0.51%) 1.65s (± 0.80%) ~ 1.63s 1.67s p=0.280 n=6
Bind Time 0.87s (± 0.86%) 0.88s (± 1.18%) +0.01s (+ 1.53%) 0.87s 0.90s p=0.046 n=6
Check Time 11.24s (± 0.34%) 11.30s (± 0.74%) ~ 11.23s 11.44s p=0.259 n=6
Emit Time 3.15s (± 0.47%) 3.14s (± 0.60%) ~ 3.11s 3.16s p=0.511 n=6
Total Time 16.91s (± 0.25%) 16.98s (± 0.55%) ~ 16.86s 17.12s p=0.126 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,409k (± 0.00%) 348,488k (± 0.01%) +1,079k (+ 0.31%) 348,453k 348,537k p=0.005 n=6
Parse Time 2.97s (± 0.91%) 2.99s (± 0.88%) ~ 2.96s 3.02s p=0.367 n=6
Bind Time 1.11s (± 0.57%) 1.09s (± 0.47%) -0.02s (- 2.10%) 1.08s 1.09s p=0.004 n=6
Check Time 8.28s (± 0.58%) 8.21s (± 0.45%) -0.07s (- 0.85%) 8.16s 8.26s p=0.037 n=6
Emit Time 4.85s (± 0.18%) 4.84s (± 0.40%) ~ 4.82s 4.87s p=0.370 n=6
Total Time 17.21s (± 0.27%) 17.13s (± 0.19%) -0.08s (- 0.47%) 17.10s 17.18s p=0.013 n=6
TFS - node (v18.15.0, x64)
Memory used 302,764k (± 0.01%) 303,890k (± 0.00%) +1,126k (+ 0.37%) 303,876k 303,903k p=0.005 n=6
Parse Time 2.96s (± 0.84%) 2.97s (± 0.63%) ~ 2.94s 2.99s p=0.936 n=6
Bind Time 1.47s (± 0.35%) 1.45s (± 0.95%) -0.02s (- 1.24%) 1.44s 1.48s p=0.039 n=6
Check Time 9.24s (± 0.39%) 9.17s (± 0.59%) ~ 9.11s 9.25s p=0.050 n=6
Emit Time 5.30s (± 0.36%) 5.31s (± 0.74%) ~ 5.25s 5.36s p=0.745 n=6
Total Time 18.98s (± 0.26%) 18.90s (± 0.07%) -0.08s (- 0.43%) 18.88s 18.92s p=0.005 n=6
material-ui - node (v18.15.0, x64)
Memory used 510,062k (± 0.01%) 510,778k (± 0.00%) +716k (+ 0.14%) 510,756k 510,819k p=0.005 n=6
Parse Time 2.65s (± 0.31%) 2.67s (± 0.21%) +0.01s (+ 0.44%) 2.66s 2.67s p=0.025 n=6
Bind Time 0.99s (± 0.99%) 0.98s (± 0.83%) ~ 0.97s 0.99s p=0.282 n=6
Check Time 17.39s (± 0.47%) 17.39s (± 0.48%) ~ 17.29s 17.49s p=0.936 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 21.04s (± 0.42%) 21.03s (± 0.42%) ~ 20.93s 21.14s p=1.000 n=6
mui-docs - node (v18.15.0, x64)
Memory used 1,744,403k (± 0.00%) 1,746,098k (± 0.00%) +1,695k (+ 0.10%) 1,746,075k 1,746,113k p=0.005 n=6
Parse Time 7.78s (± 0.54%) 7.81s (± 0.70%) ~ 7.76s 7.90s p=0.630 n=6
Bind Time 2.83s (± 0.49%) 2.77s (± 1.89%) ~ 2.75s 2.88s p=0.060 n=6
Check Time 67.08s (± 0.58%) 66.67s (± 0.35%) ~ 66.40s 66.95s p=0.091 n=6
Emit Time 0.16s (± 5.21%) 0.15s (± 3.36%) ~ 0.15s 0.16s p=0.523 n=6
Total Time 77.84s (± 0.52%) 77.41s (± 0.28%) ~ 77.15s 77.71s p=0.054 n=6
self-build-src - node (v18.15.0, x64)
Memory used 2,396,325k (± 0.04%) 2,405,727k (± 0.03%) +9,401k (+ 0.39%) 2,405,172k 2,406,649k p=0.005 n=6
Parse Time 7.45s (± 0.67%) 7.50s (± 1.16%) ~ 7.38s 7.60s p=0.261 n=6
Bind Time 2.76s (± 0.71%) 2.77s (± 0.85%) ~ 2.73s 2.80s p=1.000 n=6
Check Time 49.27s (± 0.63%) 49.02s (± 0.37%) ~ 48.82s 49.26s p=0.173 n=6
Emit Time 3.93s (± 1.45%) 3.82s (± 1.95%) -0.11s (- 2.76%) 3.73s 3.91s p=0.045 n=6
Total Time 63.44s (± 0.45%) 63.14s (± 0.30%) ~ 62.86s 63.35s p=0.093 n=6
self-compiler - node (v18.15.0, x64)
Memory used 415,837k (± 0.01%) 416,967k (± 0.02%) +1,130k (+ 0.27%) 416,912k 417,157k p=0.005 n=6
Parse Time 4.10s (± 0.91%) 4.15s (± 0.52%) +0.05s (+ 1.22%) 4.13s 4.18s p=0.030 n=6
Bind Time 1.59s (± 0.93%) 1.61s (± 1.01%) +0.02s (+ 1.57%) 1.59s 1.63s p=0.036 n=6
Check Time 22.52s (± 0.36%) 22.30s (± 0.48%) -0.22s (- 0.96%) 22.13s 22.43s p=0.010 n=6
Emit Time 1.70s (± 1.38%) 1.71s (± 1.75%) ~ 1.67s 1.75s p=0.520 n=6
Total Time 29.90s (± 0.28%) 29.77s (± 0.45%) ~ 29.53s 29.91s p=0.109 n=6
vscode - node (v18.15.0, x64)
Memory used 2,896,268k (± 0.00%) 2,902,703k (± 0.00%) +6,435k (+ 0.22%) 2,902,512k 2,902,831k p=0.005 n=6
Parse Time 12.94s (± 0.27%) 12.93s (± 0.30%) ~ 12.88s 12.98s p=0.518 n=6
Bind Time 4.14s (± 0.13%) 4.12s (± 0.32%) -0.02s (- 0.56%) 4.10s 4.13s p=0.004 n=6
Check Time 72.29s (± 0.45%) 71.74s (± 0.37%) -0.55s (- 0.76%) 71.33s 72.12s p=0.016 n=6
Emit Time 19.40s (± 0.82%) 20.75s (± 9.72%) ~ 19.42s 23.36s p=0.128 n=6
Total Time 108.78s (± 0.33%) 109.54s (± 1.88%) ~ 107.89s 112.21s p=0.689 n=6
webpack - node (v18.15.0, x64)
Memory used 408,792k (± 0.01%) 409,708k (± 0.01%) +916k (+ 0.22%) 409,604k 409,759k p=0.005 n=6
Parse Time 3.24s (± 0.30%) 3.23s (± 0.64%) ~ 3.21s 3.27s p=0.405 n=6
Bind Time 1.40s (± 0.90%) 1.40s (± 0.37%) ~ 1.39s 1.40s p=0.448 n=6
Check Time 14.28s (± 0.22%) 14.20s (± 0.21%) -0.08s (- 0.54%) 14.16s 14.24s p=0.012 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 18.92s (± 0.19%) 18.83s (± 0.19%) -0.09s (- 0.45%) 18.77s 18.87s p=0.008 n=6
xstate - node (v18.15.0, x64)
Memory used 513,414k (± 0.01%) 514,996k (± 0.03%) +1,582k (+ 0.31%) 514,884k 515,314k p=0.005 n=6
Parse Time 3.97s (± 0.46%) 4.04s (± 4.45%) ~ 3.95s 4.41s p=0.627 n=6
Bind Time 1.88s (± 1.40%) 1.88s (± 0.81%) ~ 1.85s 1.89s p=1.000 n=6
Check Time 3.41s (± 0.58%) 3.43s (± 1.41%) ~ 3.38s 3.50s p=0.806 n=6
Emit Time 0.08s (± 6.19%) 0.08s (± 4.99%) ~ 0.08s 0.09s p=0.527 n=6
Total Time 9.35s (± 0.36%) 9.44s (± 1.89%) ~ 9.31s 9.78s p=0.630 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@ahejlsberg Here are the results of running the top 400 repos comparing main and refs/pull/57977/merge:

Everything looks good!

@ahejlsberg
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 28, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,590k (± 0.01%) 296,432k (± 0.01%) +842k (+ 0.28%) 296,406k 296,459k p=0.005 n=6
Parse Time 3.93s (± 0.44%) 3.94s (± 0.35%) ~ 3.92s 3.95s p=0.459 n=6
Bind Time 1.24s (± 1.11%) 1.25s (± 0.88%) ~ 1.24s 1.27s p=0.111 n=6
Check Time 12.01s (± 0.31%) 11.97s (± 0.32%) ~ 11.91s 12.00s p=0.145 n=6
Emit Time 10.45s (± 0.37%) 10.43s (± 0.40%) ~ 10.37s 10.47s p=0.570 n=6
Total Time 27.63s (± 0.18%) 27.58s (± 0.26%) ~ 27.48s 27.67s p=0.260 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 192,742k (± 0.73%) 193,706k (± 0.94%) +965k (+ 0.50%) 192,387k 196,139k p=0.045 n=6
Parse Time 1.34s (± 1.13%) 1.36s (± 0.72%) ~ 1.35s 1.38s p=0.059 n=6
Bind Time 0.73s (± 0.00%) 0.74s (± 0.00%) +0.01s (+ 1.37%) 0.74s 0.74s p=0.001 n=6
Check Time 9.53s (± 0.41%) 9.51s (± 0.35%) ~ 9.46s 9.55s p=0.574 n=6
Emit Time 2.63s (± 0.46%) 2.63s (± 0.37%) ~ 2.62s 2.64s p=0.865 n=6
Total Time 14.23s (± 0.28%) 14.24s (± 0.21%) ~ 14.20s 14.28s p=0.422 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,401k (± 0.01%) 348,455k (± 0.01%) +1,054k (+ 0.30%) 348,420k 348,468k p=0.005 n=6
Parse Time 2.49s (± 0.34%) 2.47s (± 0.85%) ~ 2.43s 2.49s p=0.109 n=6
Bind Time 0.93s (± 0.56%) 0.93s (± 0.55%) ~ 0.93s 0.94s p=0.069 n=6
Check Time 7.02s (± 0.47%) 6.92s (± 0.42%) -0.11s (- 1.52%) 6.89s 6.97s p=0.006 n=6
Emit Time 4.06s (± 0.40%) 4.05s (± 0.51%) ~ 4.03s 4.08s p=0.869 n=6
Total Time 14.49s (± 0.28%) 14.38s (± 0.36%) -0.12s (- 0.80%) 14.29s 14.44s p=0.008 n=6
TFS - node (v18.15.0, x64)
Memory used 302,773k (± 0.01%) 303,921k (± 0.01%) +1,148k (+ 0.38%) 303,886k 303,975k p=0.005 n=6
Parse Time 2.41s (± 0.68%) 2.42s (± 1.21%) ~ 2.37s 2.45s p=0.170 n=6
Bind Time 1.19s (± 0.63%) 1.20s (± 1.33%) ~ 1.18s 1.22s p=0.136 n=6
Check Time 7.54s (± 0.31%) 7.49s (± 0.35%) -0.05s (- 0.73%) 7.44s 7.52s p=0.008 n=6
Emit Time 4.27s (± 0.74%) 4.28s (± 0.54%) ~ 4.25s 4.31s p=0.292 n=6
Total Time 15.41s (± 0.21%) 15.39s (± 0.25%) ~ 15.34s 15.43s p=0.685 n=6
material-ui - node (v18.15.0, x64)
Memory used 510,051k (± 0.00%) 510,814k (± 0.01%) +764k (+ 0.15%) 510,764k 510,858k p=0.005 n=6
Parse Time 2.65s (± 0.56%) 2.66s (± 0.39%) ~ 2.65s 2.68s p=0.101 n=6
Bind Time 0.99s (± 1.11%) 0.98s (± 0.42%) ~ 0.98s 0.99s p=0.090 n=6
Check Time 17.33s (± 0.35%) 17.33s (± 0.55%) ~ 17.21s 17.43s p=0.748 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.97s (± 0.18%) 20.98s (± 0.43%) ~ 20.87s 21.08s p=0.629 n=6
mui-docs - node (v18.15.0, x64)
Memory used 1,744,675k (± 0.00%) 1,746,323k (± 0.00%) +1,648k (+ 0.09%) 1,746,279k 1,746,350k p=0.005 n=6
Parse Time 6.60s (± 0.30%) 6.61s (± 0.40%) ~ 6.56s 6.64s p=0.359 n=6
Bind Time 2.38s (± 0.78%) 2.37s (± 0.56%) ~ 2.36s 2.39s p=0.448 n=6
Check Time 56.67s (± 0.53%) 56.60s (± 0.45%) ~ 56.30s 56.95s p=0.936 n=6
Emit Time 0.13s (± 3.10%) 0.13s (± 3.10%) ~ 0.13s 0.14s p=1.000 n=6
Total Time 65.78s (± 0.43%) 65.71s (± 0.39%) ~ 65.39s 66.07s p=0.873 n=6
self-build-src - node (v18.15.0, x64)
Memory used 2,396,280k (± 0.04%) 2,405,639k (± 0.04%) +9,360k (+ 0.39%) 2,404,293k 2,406,787k p=0.005 n=6
Parse Time 6.00s (± 0.76%) 5.97s (± 0.82%) ~ 5.91s 6.02s p=0.295 n=6
Bind Time 2.26s (± 1.27%) 2.27s (± 0.85%) ~ 2.24s 2.30s p=0.374 n=6
Check Time 39.76s (± 0.18%) 39.48s (± 0.34%) -0.27s (- 0.69%) 39.33s 39.64s p=0.005 n=6
Emit Time 3.12s (± 1.32%) 3.09s (± 2.25%) ~ 3.00s 3.21s p=0.261 n=6
Total Time 51.17s (± 0.24%) 50.84s (± 0.19%) -0.34s (- 0.66%) 50.69s 50.95s p=0.005 n=6
self-compiler - node (v18.15.0, x64)
Memory used 415,838k (± 0.01%) 416,969k (± 0.01%) +1,131k (+ 0.27%) 416,913k 417,010k p=0.005 n=6
Parse Time 3.34s (± 0.73%) 3.33s (± 0.88%) ~ 3.30s 3.38s p=0.463 n=6
Bind Time 1.29s (± 0.91%) 1.33s (± 0.48%) +0.04s (+ 3.23%) 1.32s 1.34s p=0.004 n=6
Check Time 18.23s (± 0.33%) 18.08s (± 0.27%) -0.15s (- 0.80%) 18.02s 18.15s p=0.010 n=6
Emit Time 1.36s (± 1.35%) 1.36s (± 0.62%) ~ 1.34s 1.36s p=0.325 n=6
Total Time 24.22s (± 0.32%) 24.10s (± 0.15%) -0.12s (- 0.50%) 24.05s 24.15s p=0.016 n=6
vscode - node (v18.15.0, x64)
Memory used 2,898,052k (± 0.00%) 2,904,473k (± 0.00%) +6,422k (+ 0.22%) 2,904,364k 2,904,580k p=0.005 n=6
Parse Time 15.96s (± 0.47%) 15.90s (± 0.20%) ~ 15.86s 15.95s p=0.199 n=6
Bind Time 5.01s (± 0.21%) 5.24s (± 0.35%) 🔻+0.23s (+ 4.63%) 5.21s 5.26s p=0.005 n=6
Check Time 87.16s (± 0.28%) 86.69s (± 0.35%) -0.47s (- 0.54%) 86.33s 87.19s p=0.031 n=6
Emit Time 23.63s (± 0.37%) 23.61s (± 0.77%) ~ 23.41s 23.92s p=0.630 n=6
Total Time 131.75s (± 0.18%) 131.44s (± 0.17%) ~ 131.16s 131.80s p=0.054 n=6
webpack - node (v18.15.0, x64)
Memory used 408,980k (± 0.02%) 409,833k (± 0.02%) +854k (+ 0.21%) 409,751k 409,918k p=0.005 n=6
Parse Time 4.79s (± 0.36%) 4.79s (± 0.81%) ~ 4.73s 4.82s p=0.746 n=6
Bind Time 2.08s (± 1.11%) 2.07s (± 1.17%) ~ 2.05s 2.11s p=0.250 n=6
Check Time 20.90s (± 0.54%) 20.77s (± 0.34%) -0.13s (- 0.62%) 20.70s 20.86s p=0.030 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 27.77s (± 0.42%) 27.63s (± 0.27%) -0.14s (- 0.52%) 27.54s 27.72s p=0.045 n=6
xstate - node (v18.15.0, x64)
Memory used 513,420k (± 0.01%) 514,860k (± 0.01%) +1,440k (+ 0.28%) 514,803k 514,919k p=0.005 n=6
Parse Time 3.97s (± 0.53%) 3.97s (± 0.53%) ~ 3.94s 4.00s p=0.872 n=6
Bind Time 1.89s (± 1.60%) 1.88s (± 1.29%) ~ 1.84s 1.90s p=0.570 n=6
Check Time 3.43s (± 0.70%) 3.43s (± 0.68%) ~ 3.39s 3.46s p=0.872 n=6
Emit Time 0.09s (± 5.95%) 0.08s (± 6.19%) ~ 0.08s 0.09s p=0.311 n=6
Total Time 9.38s (± 0.59%) 9.36s (± 0.46%) ~ 9.31s 9.42s p=0.629 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@ahejlsberg
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 28, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@ahejlsberg
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 28, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,619k (± 0.01%) 296,252k (± 0.01%) +634k (+ 0.21%) 296,227k 296,293k p=0.005 n=6
Parse Time 3.21s (± 0.51%) 3.21s (± 0.46%) ~ 3.19s 3.23s p=0.624 n=6
Bind Time 1.00s (± 0.82%) 0.99s (± 1.56%) ~ 0.98s 1.01s p=0.281 n=6
Check Time 9.72s (± 0.31%) 9.71s (± 0.23%) ~ 9.67s 9.73s p=0.629 n=6
Emit Time 8.39s (± 0.35%) 8.38s (± 0.29%) ~ 8.35s 8.41s p=0.464 n=6
Total Time 22.31s (± 0.17%) 22.29s (± 0.12%) ~ 22.25s 22.32s p=0.368 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 192,704k (± 0.75%) 193,843k (± 0.93%) +1,139k (+ 0.59%) 192,630k 196,219k p=0.030 n=6
Parse Time 1.35s (± 0.98%) 1.35s (± 0.56%) ~ 1.34s 1.36s p=0.448 n=6
Bind Time 0.73s (± 0.00%) 0.74s (± 0.00%) +0.01s (+ 1.37%) 0.74s 0.74s p=0.001 n=6
Check Time 9.55s (± 0.71%) 9.63s (± 0.35%) ~ 9.57s 9.66s p=0.090 n=6
Emit Time 2.62s (± 0.56%) 2.63s (± 0.42%) ~ 2.61s 2.64s p=0.186 n=6
Total Time 14.25s (± 0.44%) 14.35s (± 0.17%) +0.10s (+ 0.70%) 14.30s 14.37s p=0.029 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,380k (± 0.01%) 348,418k (± 0.01%) +1,038k (+ 0.30%) 348,395k 348,454k p=0.005 n=6
Parse Time 2.48s (± 0.44%) 2.49s (± 0.16%) ~ 2.48s 2.49s p=0.100 n=6
Bind Time 0.93s (± 0.56%) 0.92s (± 0.00%) -0.01s (- 0.72%) 0.92s 0.92s p=0.025 n=6
Check Time 7.03s (± 0.23%) 6.97s (± 0.36%) -0.05s (- 0.78%) 6.94s 7.00s p=0.007 n=6
Emit Time 4.06s (± 0.25%) 4.05s (± 0.46%) ~ 4.04s 4.09s p=0.324 n=6
Total Time 14.49s (± 0.23%) 14.43s (± 0.18%) -0.06s (- 0.41%) 14.38s 14.45s p=0.005 n=6
TFS - node (v18.15.0, x64)
Memory used 302,776k (± 0.01%) 304,001k (± 0.01%) +1,225k (+ 0.40%) 303,950k 304,073k p=0.005 n=6
Parse Time 2.42s (± 1.00%) 2.42s (± 0.77%) ~ 2.39s 2.44s p=0.413 n=6
Bind Time 1.19s (± 0.63%) 1.19s (± 0.63%) ~ 1.18s 1.20s p=0.487 n=6
Check Time 7.52s (± 0.33%) 7.54s (± 0.43%) ~ 7.49s 7.57s p=0.464 n=6
Emit Time 4.28s (± 0.74%) 4.25s (± 0.69%) ~ 4.20s 4.28s p=0.092 n=6
Total Time 15.41s (± 0.28%) 15.40s (± 0.43%) ~ 15.32s 15.49s p=0.810 n=6
material-ui - node (v18.15.0, x64)
Memory used 510,039k (± 0.01%) 510,574k (± 0.00%) +534k (+ 0.10%) 510,558k 510,598k p=0.005 n=6
Parse Time 2.66s (± 0.31%) 2.67s (± 0.39%) ~ 2.65s 2.68s p=0.546 n=6
Bind Time 1.00s (± 0.41%) 0.98s (± 0.77%) -0.02s (- 2.00%) 0.97s 0.99s p=0.004 n=6
Check Time 17.30s (± 0.38%) 17.31s (± 0.31%) ~ 17.24s 17.38s p=0.872 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.96s (± 0.30%) 20.96s (± 0.26%) ~ 20.90s 21.04s p=0.936 n=6
mui-docs - node (v18.15.0, x64)
Memory used 1,744,624k (± 0.00%) 1,745,945k (± 0.00%) +1,321k (+ 0.08%) 1,745,935k 1,745,963k p=0.005 n=6
Parse Time 7.79s (± 0.42%) 7.81s (± 0.63%) ~ 7.72s 7.87s p=0.261 n=6
Bind Time 2.82s (± 0.41%) 2.78s (± 0.51%) -0.04s (- 1.36%) 2.76s 2.80s p=0.005 n=6
Check Time 66.44s (± 0.49%) 66.72s (± 0.39%) ~ 66.33s 66.97s p=0.128 n=6
Emit Time 0.16s (± 3.29%) 0.16s (± 3.29%) ~ 0.15s 0.16s p=1.000 n=6
Total Time 77.20s (± 0.45%) 77.46s (± 0.33%) ~ 77.08s 77.74s p=0.230 n=6
self-build-src - node (v18.15.0, x64)
Memory used 2,396,152k (± 0.03%) 2,402,666k (± 0.05%) +6,514k (+ 0.27%) 2,401,346k 2,403,954k p=0.005 n=6
Parse Time 4.98s (± 1.01%) 5.02s (± 1.10%) ~ 4.95s 5.11s p=0.297 n=6
Bind Time 1.90s (± 0.77%) 1.93s (± 1.25%) +0.03s (+ 1.67%) 1.91s 1.97s p=0.034 n=6
Check Time 33.87s (± 0.20%) 33.85s (± 0.26%) ~ 33.74s 33.95s p=0.520 n=6
Emit Time 2.68s (± 1.17%) 2.60s (± 2.48%) ~ 2.51s 2.68s p=0.054 n=6
Total Time 43.44s (± 0.23%) 43.42s (± 0.36%) ~ 43.20s 43.57s p=1.000 n=6
self-compiler - node (v18.15.0, x64)
Memory used 415,888k (± 0.02%) 417,323k (± 0.01%) +1,435k (+ 0.35%) 417,296k 417,353k p=0.005 n=6
Parse Time 4.12s (± 0.87%) 4.12s (± 0.52%) ~ 4.09s 4.14s p=1.000 n=6
Bind Time 1.59s (± 0.66%) 1.62s (± 1.03%) +0.02s (+ 1.57%) 1.60s 1.65s p=0.014 n=6
Check Time 22.53s (± 0.20%) 22.37s (± 0.31%) -0.16s (- 0.70%) 22.30s 22.50s p=0.013 n=6
Emit Time 1.68s (± 1.25%) 1.68s (± 1.66%) ~ 1.65s 1.71s p=0.626 n=6
Total Time 29.92s (± 0.21%) 29.80s (± 0.25%) -0.13s (- 0.42%) 29.70s 29.93s p=0.020 n=6
vscode - node (v18.15.0, x64)
Memory used 2,897,392k (± 0.00%) 2,904,135k (± 0.00%) +6,743k (+ 0.23%) 2,903,912k 2,904,250k p=0.005 n=6
Parse Time 15.97s (± 0.27%) 15.94s (± 0.37%) ~ 15.87s 16.03s p=0.297 n=6
Bind Time 5.04s (± 0.46%) 5.02s (± 0.20%) ~ 5.01s 5.03s p=0.118 n=6
Check Time 87.58s (± 0.37%) 87.43s (± 0.43%) ~ 86.94s 88.05s p=0.471 n=6
Emit Time 23.76s (± 0.88%) 24.59s (± 7.68%) ~ 23.68s 28.44s p=0.298 n=6
Total Time 132.35s (± 0.33%) 132.98s (± 1.65%) ~ 131.65s 137.41s p=0.575 n=6
webpack - node (v18.15.0, x64)
Memory used 408,921k (± 0.03%) 409,825k (± 0.01%) +904k (+ 0.22%) 409,756k 409,895k p=0.005 n=6
Parse Time 3.91s (± 0.38%) 3.89s (± 0.52%) ~ 3.88s 3.93s p=0.154 n=6
Bind Time 1.68s (± 1.34%) 1.67s (± 0.70%) ~ 1.65s 1.68s p=0.357 n=6
Check Time 16.85s (± 0.31%) 16.78s (± 0.35%) ~ 16.71s 16.86s p=0.093 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.43s (± 0.24%) 22.34s (± 0.17%) -0.09s (- 0.42%) 22.28s 22.39s p=0.010 n=6
xstate - node (v18.15.0, x64)
Memory used 513,520k (± 0.03%) 514,646k (± 0.02%) +1,126k (+ 0.22%) 514,494k 514,818k p=0.005 n=6
Parse Time 4.89s (± 0.22%) 4.87s (± 0.66%) ~ 4.81s 4.89s p=0.103 n=6
Bind Time 2.31s (± 1.53%) 2.30s (± 0.91%) ~ 2.28s 2.33s p=0.627 n=6
Check Time 4.30s (± 0.50%) 4.29s (± 0.54%) ~ 4.26s 4.32s p=0.869 n=6
Emit Time 0.11s (± 0.00%) 0.11s (± 0.00%) ~ 0.11s 0.11s p=1.000 n=6
Total Time 11.62s (± 0.18%) 11.58s (± 0.49%) ~ 11.49s 11.67s p=0.063 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,684k (± 0.01%) 296,328k (± 0.01%) +644k (+ 0.22%) 296,291k 296,359k p=0.005 n=6
Parse Time 2.67s (± 0.21%) 2.66s (± 0.70%) ~ 2.63s 2.68s p=0.865 n=6
Bind Time 0.84s (± 0.97%) 0.84s (± 1.63%) ~ 0.82s 0.85s p=0.934 n=6
Check Time 8.22s (± 0.30%) 8.25s (± 0.24%) ~ 8.22s 8.27s p=0.073 n=6
Emit Time 7.03s (± 0.44%) 7.04s (± 0.37%) ~ 7.00s 7.07s p=0.747 n=6
Total Time 18.75s (± 0.22%) 18.79s (± 0.21%) ~ 18.71s 18.82s p=0.291 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 192,725k (± 0.80%) 193,742k (± 0.98%) +1,017k (+ 0.53%) 192,269k 196,231k p=0.031 n=6
Parse Time 1.36s (± 1.35%) 1.35s (± 0.60%) ~ 1.35s 1.37s p=0.154 n=6
Bind Time 0.73s (± 0.00%) 0.74s (± 0.00%) +0.01s (+ 1.37%) 0.74s 0.74s p=0.001 n=6
Check Time 9.54s (± 0.27%) 9.58s (± 0.95%) ~ 9.46s 9.67s p=0.378 n=6
Emit Time 2.63s (± 0.24%) 2.71s (± 4.36%) ~ 2.62s 2.89s p=0.157 n=6
Total Time 14.26s (± 0.28%) 14.39s (± 1.25%) ~ 14.19s 14.61s p=0.261 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,413k (± 0.00%) 348,441k (± 0.01%) +1,028k (+ 0.30%) 348,420k 348,474k p=0.005 n=6
Parse Time 3.00s (± 1.27%) 3.00s (± 0.67%) ~ 2.97s 3.02s p=0.686 n=6
Bind Time 1.11s (± 0.37%) 1.10s (± 0.47%) -0.01s (- 0.75%) 1.10s 1.11s p=0.022 n=6
Check Time 8.31s (± 0.40%) 8.23s (± 0.47%) -0.08s (- 0.94%) 8.18s 8.28s p=0.008 n=6
Emit Time 4.85s (± 0.44%) 4.85s (± 0.57%) ~ 4.83s 4.90s p=1.000 n=6
Total Time 17.27s (± 0.30%) 17.18s (± 0.38%) -0.08s (- 0.48%) 17.13s 17.31s p=0.045 n=6
TFS - node (v18.15.0, x64)
Memory used 302,786k (± 0.01%) 303,988k (± 0.01%) +1,202k (+ 0.40%) 303,942k 304,035k p=0.005 n=6
Parse Time 2.40s (± 0.43%) 2.42s (± 0.92%) ~ 2.39s 2.45s p=0.059 n=6
Bind Time 1.19s (± 0.68%) 1.18s (± 0.54%) -0.01s (- 1.12%) 1.17s 1.19s p=0.023 n=6
Check Time 7.50s (± 0.48%) 7.53s (± 0.39%) ~ 7.50s 7.58s p=0.171 n=6
Emit Time 4.26s (± 0.74%) 4.27s (± 0.58%) ~ 4.24s 4.30s p=0.871 n=6
Total Time 15.35s (± 0.40%) 15.39s (± 0.16%) ~ 15.36s 15.42s p=0.102 n=6
material-ui - node (v18.15.0, x64)
Memory used 510,059k (± 0.01%) 510,578k (± 0.00%) +518k (+ 0.10%) 510,565k 510,588k p=0.005 n=6
Parse Time 2.67s (± 0.24%) 2.66s (± 0.50%) ~ 2.64s 2.68s p=0.081 n=6
Bind Time 0.99s (± 0.64%) 0.99s (± 0.76%) ~ 0.98s 1.00s p=0.718 n=6
Check Time 17.34s (± 0.32%) 17.31s (± 0.25%) ~ 17.26s 17.37s p=0.332 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 21.00s (± 0.27%) 20.96s (± 0.20%) ~ 20.91s 21.02s p=0.334 n=6
mui-docs - node (v18.15.0, x64)
Memory used 1,744,643k (± 0.00%) 1,745,919k (± 0.00%) +1,276k (+ 0.07%) 1,745,870k 1,745,962k p=0.005 n=6
Parse Time 6.62s (± 0.24%) 6.59s (± 0.68%) ~ 6.55s 6.67s p=0.086 n=6
Bind Time 2.38s (± 0.58%) 2.33s (± 0.45%) -0.05s (- 2.10%) 2.32s 2.35s p=0.005 n=6
Check Time 56.77s (± 0.48%) 56.85s (± 0.64%) ~ 56.38s 57.27s p=0.810 n=6
Emit Time 0.13s (± 3.87%) 0.13s (± 3.10%) ~ 0.13s 0.14s p=0.595 n=6
Total Time 65.91s (± 0.43%) 65.91s (± 0.57%) ~ 65.40s 66.32s p=0.936 n=6
self-build-src - node (v18.15.0, x64)
Memory used 2,396,215k (± 0.03%) 2,403,535k (± 0.03%) +7,320k (+ 0.31%) 2,402,694k 2,404,514k p=0.005 n=6
Parse Time 5.97s (± 0.77%) 5.98s (± 0.63%) ~ 5.93s 6.02s p=1.000 n=6
Bind Time 2.26s (± 1.45%) 2.27s (± 0.45%) ~ 2.26s 2.29s p=0.106 n=6
Check Time 39.83s (± 0.23%) 39.85s (± 0.28%) ~ 39.74s 40.04s p=0.689 n=6
Emit Time 3.16s (± 2.18%) 3.07s (± 2.42%) ~ 2.98s 3.16s p=0.054 n=6
Total Time 51.23s (± 0.29%) 51.18s (± 0.27%) ~ 51.04s 51.43s p=0.689 n=6
self-compiler - node (v18.15.0, x64)
Memory used 415,831k (± 0.01%) 417,381k (± 0.01%) +1,550k (+ 0.37%) 417,304k 417,464k p=0.005 n=6
Parse Time 4.12s (± 0.85%) 4.12s (± 1.03%) ~ 4.07s 4.18s p=0.872 n=6
Bind Time 1.59s (± 0.86%) 1.63s (± 1.21%) +0.03s (+ 1.99%) 1.60s 1.66s p=0.014 n=6
Check Time 22.46s (± 0.49%) 22.50s (± 0.70%) ~ 22.32s 22.74s p=0.630 n=6
Emit Time 1.70s (± 1.50%) 1.70s (± 1.55%) ~ 1.66s 1.74s p=1.000 n=6
Total Time 29.88s (± 0.43%) 29.94s (± 0.70%) ~ 29.73s 30.28s p=0.687 n=6
vscode - node (v18.15.0, x64)
Memory used 2,898,107k (± 0.00%) 2,904,840k (± 0.00%) +6,733k (+ 0.23%) 2,904,687k 2,904,932k p=0.005 n=6
Parse Time 15.92s (± 0.33%) 15.89s (± 0.43%) ~ 15.82s 15.99s p=0.572 n=6
Bind Time 5.00s (± 0.37%) 4.97s (± 0.55%) ~ 4.94s 5.00s p=0.102 n=6
Check Time 86.73s (± 0.23%) 87.08s (± 0.51%) ~ 86.57s 87.58s p=0.423 n=6
Emit Time 23.62s (± 0.64%) 25.18s (± 9.75%) ~ 23.43s 28.52s p=0.378 n=6
Total Time 131.28s (± 0.21%) 133.13s (± 2.07%) ~ 130.97s 136.67s p=0.423 n=6
webpack - node (v18.15.0, x64)
Memory used 408,885k (± 0.02%) 409,873k (± 0.03%) +988k (+ 0.24%) 409,767k 410,069k p=0.005 n=6
Parse Time 3.88s (± 0.62%) 3.88s (± 0.50%) ~ 3.85s 3.91s p=0.506 n=6
Bind Time 1.68s (± 0.69%) 1.67s (± 1.49%) ~ 1.63s 1.70s p=0.411 n=6
Check Time 16.83s (± 0.28%) 16.78s (± 0.22%) ~ 16.72s 16.82s p=0.170 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.39s (± 0.18%) 22.33s (± 0.22%) -0.06s (- 0.28%) 22.27s 22.42s p=0.044 n=6
xstate - node (v18.15.0, x64)
Memory used 513,486k (± 0.03%) 514,654k (± 0.02%) +1,168k (+ 0.23%) 514,512k 514,840k p=0.005 n=6
Parse Time 3.97s (± 0.59%) 3.98s (± 0.65%) ~ 3.94s 4.02s p=0.809 n=6
Bind Time 1.88s (± 0.97%) 1.88s (± 1.03%) ~ 1.86s 1.91s p=0.745 n=6
Check Time 3.43s (± 0.62%) 3.43s (± 0.70%) ~ 3.39s 3.46s p=0.627 n=6
Emit Time 0.09s (± 9.41%) 0.09s (± 6.44%) ~ 0.08s 0.09s p=0.796 n=6
Total Time 9.38s (± 0.46%) 9.38s (± 0.62%) ~ 9.28s 9.45s p=0.872 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@ahejlsberg
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 31, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@ahejlsberg
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Apr 2, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

Hey @ahejlsberg, the results of running the DT tests are ready.

Everything looks the same!

You can check the log here.

@typescript-bot
Copy link
Collaborator

@ahejlsberg Here are the results of running the user tests comparing main and refs/pull/57977/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,693k (± 0.01%) 295,941k (± 0.01%) +248k (+ 0.08%) 295,876k 295,992k p=0.005 n=6
Parse Time 2.66s (± 0.31%) 2.65s (± 0.48%) ~ 2.63s 2.66s p=0.293 n=6
Bind Time 0.84s (± 0.97%) 0.83s (± 1.60%) ~ 0.82s 0.85s p=0.185 n=6
Check Time 8.26s (± 0.28%) 8.22s (± 0.45%) ~ 8.17s 8.27s p=0.125 n=6
Emit Time 7.05s (± 0.48%) 7.04s (± 0.35%) ~ 7.00s 7.07s p=0.936 n=6
Total Time 18.80s (± 0.23%) 18.74s (± 0.17%) -0.06s (- 0.32%) 18.70s 18.79s p=0.024 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 193,299k (± 0.92%) 193,805k (± 1.00%) ~ 191,975k 195,636k p=0.575 n=6
Parse Time 1.37s (± 0.92%) 1.36s (± 1.23%) ~ 1.34s 1.38s p=0.315 n=6
Bind Time 0.73s (± 0.56%) 0.73s (± 0.71%) ~ 0.72s 0.73s p=0.595 n=6
Check Time 9.55s (± 0.45%) 9.51s (± 0.92%) ~ 9.38s 9.60s p=0.686 n=6
Emit Time 2.63s (± 0.31%) 2.63s (± 0.57%) ~ 2.61s 2.65s p=0.357 n=6
Total Time 14.28s (± 0.33%) 14.22s (± 0.55%) ~ 14.11s 14.30s p=0.228 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,415k (± 0.00%) 347,842k (± 0.01%) +428k (+ 0.12%) 347,807k 347,876k p=0.005 n=6
Parse Time 2.49s (± 0.55%) 2.49s (± 0.55%) ~ 2.47s 2.51s p=1.000 n=6
Bind Time 0.93s (± 0.44%) 0.91s (± 0.60%) -0.03s (- 2.86%) 0.90s 0.91s p=0.003 n=6
Check Time 7.02s (± 0.46%) 6.96s (± 0.31%) -0.06s (- 0.93%) 6.94s 7.00s p=0.007 n=6
Emit Time 4.07s (± 0.63%) 4.08s (± 0.59%) ~ 4.04s 4.11s p=0.466 n=6
Total Time 14.52s (± 0.24%) 14.43s (± 0.21%) -0.09s (- 0.60%) 14.39s 14.48s p=0.006 n=6
TFS - node (v18.15.0, x64)
Memory used 302,779k (± 0.01%) 303,103k (± 0.00%) +324k (+ 0.11%) 303,076k 303,116k p=0.005 n=6
Parse Time 2.41s (± 0.93%) 2.41s (± 0.69%) ~ 2.39s 2.43s p=0.621 n=6
Bind Time 1.21s (± 0.43%) 1.18s (± 0.44%) 🟩-0.04s (- 3.02%) 1.17s 1.18s p=0.004 n=6
Check Time 7.53s (± 0.36%) 7.46s (± 0.60%) -0.06s (- 0.82%) 7.41s 7.53s p=0.024 n=6
Emit Time 4.27s (± 0.55%) 4.25s (± 0.89%) ~ 4.21s 4.31s p=0.685 n=6
Total Time 15.42s (± 0.15%) 15.31s (± 0.32%) -0.11s (- 0.71%) 15.23s 15.36s p=0.005 n=6
material-ui - node (v18.15.0, x64)
Memory used 510,055k (± 0.00%) 510,592k (± 0.01%) +537k (+ 0.11%) 510,533k 510,623k p=0.005 n=6
Parse Time 2.66s (± 0.24%) 2.66s (± 0.19%) ~ 2.66s 2.67s p=0.386 n=6
Bind Time 0.99s (± 0.82%) 0.98s (± 1.67%) ~ 0.97s 1.01s p=0.323 n=6
Check Time 17.32s (± 0.49%) 17.32s (± 0.59%) ~ 17.23s 17.52s p=1.000 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.97s (± 0.38%) 20.98s (± 0.47%) ~ 20.88s 21.16s p=1.000 n=6
mui-docs - node (v18.15.0, x64)
Memory used 1,745,025k (± 0.00%) 1,745,996k (± 0.00%) +970k (+ 0.06%) 1,745,967k 1,746,029k p=0.005 n=6
Parse Time 7.79s (± 0.39%) 7.84s (± 1.65%) ~ 7.73s 8.09s p=0.628 n=6
Bind Time 2.82s (± 0.43%) 2.79s (± 3.13%) ~ 2.73s 2.97s p=0.064 n=6
Check Time 66.96s (± 0.37%) 66.86s (± 0.33%) ~ 66.57s 67.10s p=0.575 n=6
Emit Time 0.16s (± 4.75%) 0.16s (± 3.29%) ~ 0.15s 0.16s p=0.784 n=6
Total Time 77.73s (± 0.31%) 77.65s (± 0.24%) ~ 77.33s 77.80s p=0.376 n=6
self-build-src - node (v18.15.0, x64)
Memory used 2,400,635k (± 0.03%) 2,405,519k (± 0.04%) +4,884k (+ 0.20%) 2,404,363k 2,406,993k p=0.005 n=6
Parse Time 7.44s (± 1.09%) 7.46s (± 1.02%) ~ 7.36s 7.58s p=0.689 n=6
Bind Time 2.78s (± 0.44%) 2.75s (± 0.78%) -0.03s (- 1.20%) 2.73s 2.78s p=0.019 n=6
Check Time 49.37s (± 0.38%) 49.11s (± 0.34%) -0.26s (- 0.52%) 48.83s 49.28s p=0.045 n=6
Emit Time 3.94s (± 1.08%) 3.87s (± 2.81%) ~ 3.74s 3.98s p=0.689 n=6
Total Time 63.55s (± 0.41%) 63.21s (± 0.30%) -0.35s (- 0.54%) 62.97s 63.39s p=0.030 n=6
self-compiler - node (v18.15.0, x64)
Memory used 416,207k (± 0.01%) 416,251k (± 0.01%) ~ 416,210k 416,330k p=0.298 n=6
Parse Time 3.30s (± 1.83%) 3.36s (± 0.86%) ~ 3.30s 3.38s p=0.106 n=6
Bind Time 1.32s (± 4.98%) 1.30s (± 1.35%) ~ 1.29s 1.34s p=0.236 n=6
Check Time 18.20s (± 0.33%) 18.04s (± 0.33%) -0.16s (- 0.89%) 17.95s 18.11s p=0.008 n=6
Emit Time 1.36s (± 1.54%) 1.34s (± 1.02%) ~ 1.33s 1.36s p=0.192 n=6
Total Time 24.18s (± 0.36%) 24.04s (± 0.30%) -0.14s (- 0.58%) 23.92s 24.11s p=0.030 n=6
vscode - node (v18.15.0, x64)
Memory used 2,899,135k (± 0.00%) 2,900,199k (± 0.00%) +1,064k (+ 0.04%) 2,900,069k 2,900,243k p=0.005 n=6
Parse Time 12.94s (± 0.24%) 12.93s (± 0.30%) ~ 12.89s 13.00s p=0.467 n=6
Bind Time 4.16s (± 0.33%) 4.11s (± 0.33%) -0.05s (- 1.20%) 4.09s 4.13s p=0.005 n=6
Check Time 72.17s (± 0.32%) 71.38s (± 0.17%) -0.79s (- 1.10%) 71.15s 71.51s p=0.005 n=6
Emit Time 19.34s (± 0.15%) 19.31s (± 0.47%) ~ 19.16s 19.43s p=0.257 n=6
Total Time 108.60s (± 0.22%) 107.72s (± 0.19%) -0.88s (- 0.81%) 107.34s 107.98s p=0.005 n=6
webpack - node (v18.15.0, x64)
Memory used 408,905k (± 0.03%) 409,171k (± 0.03%) +266k (+ 0.07%) 409,073k 409,388k p=0.008 n=6
Parse Time 3.88s (± 0.28%) 3.88s (± 0.42%) ~ 3.86s 3.90s p=0.498 n=6
Bind Time 1.68s (± 0.97%) 1.67s (± 0.72%) ~ 1.66s 1.69s p=0.288 n=6
Check Time 16.82s (± 0.27%) 16.77s (± 0.26%) ~ 16.73s 16.85s p=0.065 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.39s (± 0.24%) 22.33s (± 0.20%) -0.07s (- 0.31%) 22.26s 22.40s p=0.037 n=6
xstate - node (v18.15.0, x64)
Memory used 513,511k (± 0.02%) 514,638k (± 0.04%) +1,127k (+ 0.22%) 514,470k 514,921k p=0.005 n=6
Parse Time 3.96s (± 0.49%) 3.98s (± 0.25%) ~ 3.97s 3.99s p=0.096 n=6
Bind Time 1.88s (± 1.20%) 1.88s (± 1.05%) ~ 1.86s 1.91s p=0.806 n=6
Check Time 3.45s (± 0.54%) 3.44s (± 0.89%) ~ 3.39s 3.47s p=0.935 n=6
Emit Time 0.08s (± 6.44%) 0.08s (± 6.19%) ~ 0.08s 0.09s p=0.640 n=6
Total Time 9.38s (± 0.42%) 9.39s (± 0.35%) ~ 9.35s 9.44s p=0.521 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,677k (± 0.01%) 295,946k (± 0.01%) +269k (+ 0.09%) 295,918k 295,990k p=0.005 n=6
Parse Time 2.65s (± 0.65%) 2.66s (± 0.15%) ~ 2.66s 2.67s p=0.209 n=6
Bind Time 0.84s (± 1.23%) 0.82s (± 1.25%) ~ 0.81s 0.84s p=0.054 n=6
Check Time 8.26s (± 0.22%) 8.21s (± 0.45%) -0.05s (- 0.63%) 8.15s 8.25s p=0.023 n=6
Emit Time 7.03s (± 0.25%) 7.04s (± 0.29%) ~ 7.00s 7.06s p=0.871 n=6
Total Time 18.78s (± 0.19%) 18.72s (± 0.30%) ~ 18.63s 18.78s p=0.064 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 193,281k (± 0.95%) 192,670k (± 0.75%) ~ 192,044k 195,625k p=0.575 n=6
Parse Time 1.36s (± 1.00%) 1.36s (± 1.45%) ~ 1.33s 1.39s p=0.560 n=6
Bind Time 0.73s (± 0.56%) 0.72s (± 0.71%) ~ 0.72s 0.73s p=0.112 n=6
Check Time 9.54s (± 0.46%) 9.47s (± 0.49%) ~ 9.41s 9.53s p=0.053 n=6
Emit Time 2.63s (± 0.37%) 2.63s (± 0.74%) ~ 2.60s 2.65s p=0.869 n=6
Total Time 14.25s (± 0.26%) 14.19s (± 0.36%) ~ 14.13s 14.27s p=0.065 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,424k (± 0.01%) 347,830k (± 0.00%) +406k (+ 0.12%) 347,811k 347,844k p=0.005 n=6
Parse Time 2.99s (± 1.51%) 2.98s (± 0.75%) ~ 2.94s 3.00s p=0.747 n=6
Bind Time 1.11s (± 0.74%) 1.08s (± 0.50%) -0.02s (- 1.96%) 1.08s 1.09s p=0.004 n=6
Check Time 8.30s (± 0.12%) 8.19s (± 0.24%) -0.11s (- 1.27%) 8.17s 8.22s p=0.004 n=6
Emit Time 4.84s (± 0.25%) 4.85s (± 0.34%) ~ 4.82s 4.87s p=0.283 n=6
Total Time 17.24s (± 0.32%) 17.11s (± 0.18%) -0.13s (- 0.73%) 17.07s 17.16s p=0.005 n=6
TFS - node (v18.15.0, x64)
Memory used 302,774k (± 0.01%) 303,103k (± 0.01%) +329k (+ 0.11%) 303,067k 303,156k p=0.005 n=6
Parse Time 2.39s (± 1.05%) 2.41s (± 1.41%) ~ 2.37s 2.46s p=0.570 n=6
Bind Time 1.19s (± 0.63%) 1.18s (± 0.64%) ~ 1.17s 1.19s p=0.195 n=6
Check Time 7.54s (± 0.37%) 7.49s (± 0.29%) -0.06s (- 0.77%) 7.45s 7.51s p=0.006 n=6
Emit Time 4.27s (± 0.41%) 4.27s (± 0.69%) ~ 4.23s 4.31s p=1.000 n=6
Total Time 15.39s (± 0.34%) 15.34s (± 0.08%) ~ 15.32s 15.35s p=0.056 n=6
material-ui - node (v18.15.0, x64)
Memory used 510,067k (± 0.01%) 510,581k (± 0.00%) +515k (+ 0.10%) 510,548k 510,604k p=0.005 n=6
Parse Time 2.66s (± 0.24%) 2.66s (± 0.44%) ~ 2.65s 2.68s p=1.000 n=6
Bind Time 0.99s (± 1.18%) 0.98s (± 0.77%) ~ 0.97s 0.99s p=0.097 n=6
Check Time 17.31s (± 0.31%) 17.30s (± 0.35%) ~ 17.20s 17.37s p=0.683 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.97s (± 0.30%) 20.95s (± 0.27%) ~ 20.86s 21.01s p=0.422 n=6
mui-docs - node (v18.15.0, x64)
Memory used 1,745,006k (± 0.00%) 1,746,029k (± 0.00%) +1,023k (+ 0.06%) 1,745,939k 1,746,108k p=0.005 n=6
Parse Time 6.60s (± 0.42%) 6.63s (± 0.76%) ~ 6.58s 6.69s p=0.868 n=6
Bind Time 2.38s (± 0.83%) 2.35s (± 0.79%) -0.03s (- 1.19%) 2.33s 2.38s p=0.032 n=6
Check Time 56.52s (± 0.29%) 56.58s (± 0.25%) ~ 56.41s 56.78s p=0.378 n=6
Emit Time 0.13s (± 0.00%) 0.13s (± 3.10%) ~ 0.13s 0.14s p=0.405 n=6
Total Time 65.63s (± 0.23%) 65.69s (± 0.24%) ~ 65.47s 65.86s p=0.575 n=6
self-build-src - node (v18.15.0, x64)
Memory used 2,400,539k (± 0.01%) 2,405,186k (± 0.03%) +4,647k (+ 0.19%) 2,404,040k 2,406,023k p=0.005 n=6
Parse Time 6.04s (± 1.16%) 6.09s (± 1.68%) ~ 6.01s 6.28s p=0.470 n=6
Bind Time 2.25s (± 1.17%) 2.26s (± 1.99%) ~ 2.21s 2.34s p=1.000 n=6
Check Time 39.93s (± 0.44%) 39.54s (± 0.44%) -0.39s (- 0.99%) 39.35s 39.81s p=0.008 n=6
Emit Time 3.15s (± 1.81%) 3.09s (± 3.85%) ~ 2.95s 3.24s p=0.575 n=6
Total Time 51.40s (± 0.27%) 51.00s (± 0.70%) ~ 50.71s 51.62s p=0.093 n=6
self-compiler - node (v18.15.0, x64)
Memory used 416,180k (± 0.01%) 416,246k (± 0.01%) +67k (+ 0.02%) 416,209k 416,277k p=0.045 n=6
Parse Time 3.33s (± 1.24%) 3.32s (± 0.25%) ~ 3.31s 3.33s p=0.870 n=6
Bind Time 1.29s (± 0.49%) 1.31s (± 1.32%) ~ 1.29s 1.33s p=0.059 n=6
Check Time 18.20s (± 0.14%) 17.95s (± 0.40%) -0.25s (- 1.36%) 17.85s 18.04s p=0.005 n=6
Emit Time 1.34s (± 1.12%) 1.35s (± 0.98%) ~ 1.33s 1.37s p=0.677 n=6
Total Time 24.16s (± 0.20%) 23.93s (± 0.33%) -0.23s (- 0.97%) 23.81s 24.03s p=0.005 n=6
vscode - node (v18.15.0, x64)
Memory used 2,899,134k (± 0.00%) 2,900,156k (± 0.00%) +1,021k (+ 0.04%) 2,900,121k 2,900,226k p=0.005 n=6
Parse Time 12.97s (± 0.52%) 12.93s (± 0.22%) ~ 12.88s 12.96s p=0.470 n=6
Bind Time 4.14s (± 0.25%) 4.13s (± 0.48%) ~ 4.11s 4.16s p=0.119 n=6
Check Time 72.20s (± 0.31%) 71.56s (± 0.60%) -0.64s (- 0.89%) 70.95s 72.05s p=0.013 n=6
Emit Time 20.03s (± 7.67%) 19.35s (± 0.47%) ~ 19.24s 19.44s p=0.521 n=6
Total Time 109.34s (± 1.55%) 107.96s (± 0.42%) -1.38s (- 1.26%) 107.42s 108.44s p=0.031 n=6
webpack - node (v18.15.0, x64)
Memory used 408,876k (± 0.01%) 409,180k (± 0.03%) +304k (+ 0.07%) 409,054k 409,375k p=0.005 n=6
Parse Time 3.88s (± 0.78%) 3.88s (± 0.46%) ~ 3.85s 3.90s p=0.517 n=6
Bind Time 1.69s (± 0.48%) 1.67s (± 1.12%) -0.03s (- 1.57%) 1.63s 1.68s p=0.007 n=6
Check Time 16.82s (± 0.29%) 16.75s (± 0.20%) -0.08s (- 0.45%) 16.70s 16.80s p=0.037 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.39s (± 0.07%) 22.30s (± 0.15%) -0.10s (- 0.43%) 22.25s 22.33s p=0.005 n=6
xstate - node (v18.15.0, x64)
Memory used 513,544k (± 0.02%) 514,576k (± 0.02%) +1,032k (+ 0.20%) 514,475k 514,687k p=0.005 n=6
Parse Time 4.91s (± 0.53%) 4.91s (± 0.56%) ~ 4.87s 4.95s p=0.809 n=6
Bind Time 2.32s (± 0.77%) 2.30s (± 0.93%) ~ 2.27s 2.32s p=0.167 n=6
Check Time 4.29s (± 0.82%) 4.31s (± 0.27%) ~ 4.30s 4.33s p=0.935 n=6
Emit Time 0.11s (± 0.00%) 0.11s (± 0.00%) ~ 0.11s 0.11s p=1.000 n=6
Total Time 11.63s (± 0.54%) 11.62s (± 0.29%) ~ 11.58s 11.67s p=0.377 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@ahejlsberg
Copy link
Member Author

Generally positive impact on performance with check time savings of up to 1.5%. Note that this is an API breaking change due to the more uniform layout of flow nodes.

@typescript-bot
Copy link
Collaborator

@ahejlsberg Here are the results of running the top 400 repos comparing main and refs/pull/57977/merge:

Everything looks good!

@jakebailey
Copy link
Member

Note that this is an API breaking change due to the more uniform layout of flow nodes.

To be honest, I'm not sure why any of FlowNode or FlowFlags are public; they are not referenced by anything but themselves. Makes me wonder if we should just be removing these so we can make more changes at will.

But, the API break will probably break the playground's flow node viewer and ts-ast-viewer.com's, but we can probably update both. Realistically the former should have been calling into ts.Debug anyway, and the latter pins specific TS versions, I think.

@jakebailey jakebailey mentioned this pull request Apr 2, 2024
Merged
jakebailey
jakebailey reviewed Apr 2, 2024
View reviewed changes
src/compiler/types.ts
id?: number; // Node id used by flow type cache in checker
id: number; // Node id used by flow type cache in checker
node: unknown; // Node or other data
antecedent: FlowNode | FlowNode[] | undefined;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm kind of surprised that this turned out okay; I would have thought these two types would have different shapes, but I guess they're both pointers...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They're both heap objects so should get the same hidden class.

@jakebailey
Copy link
Member

I opened #58036 which would remove the flow types from the public API, and make this change "non breaking" in a way 😄

@ahejlsberg
Copy link
Member Author

ahejlsberg commented Apr 2, 2024 •
edited
Loading

@jakebailey Maybe we just fold the @internal change into this PR? So we have all the breaks in one place.

@jakebailey
Copy link
Member

I figured it may be better not to, just to make it clear that we intentionally removed them, but I'm not against it if we're good with removing them.

If #58036 is merged first, then your PR doesn't have any breaks (no baseline changes at all).

@DanielRosenwasser DanielRosenwasser mentioned this pull request Apr 2, 2024
Open
@ahejlsberg
Copy link
Member Author

If #58036 is merged first, then your PR doesn't have any breaks (no baseline changes at all).

Fine by me.

@ahejlsberg
Merge branch 'main' into monomorphicFlowNodes
f3dd024 
# Conflicts:
#	src/compiler/checker.ts
@jakebailey
Copy link
Member

Done ☺️

@ahejlsberg
Make createFlowNode internal
423f96d
@ahejlsberg
Merge branch 'main' into monomorphicFlowNodes
8c289a5 
# Conflicts:
#	src/compiler/types.ts
#	tests/baselines/reference/api/typescript.d.ts
@ahejlsberg
Merge branch 'main' into monomorphicFlowNodes
2d58401 
# Conflicts:
#	src/compiler/binder.ts
@ahejlsberg ahejlsberg merged commit 39daa13 into main Apr 4, 2024
24 of 25 checks passed
@ahejlsberg ahejlsberg deleted the monomorphicFlowNodes branch April 4, 2024 23:53
@DanielRosenwasser DanielRosenwasser mentioned this pull request Apr 5, 2024
Open
@jfet97 jfet97 mentioned this pull request Jul 9, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jakebailey jakebailey jakebailey approved these changes

@DanielRosenwasser DanielRosenwasser Awaiting requested review from DanielRosenwasser

@RyanCavanaugh RyanCavanaugh Awaiting requested review from RyanCavanaugh

Assignees

@ahejlsberg ahejlsberg

Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ahejlsberg @typescript-bot @jakebailey