Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(driver-adapters-executor): benchmark GH action #4667

Merged
merged 3 commits into from
Jan 23, 2024

Conversation

jkomyno
Copy link
Contributor

@jkomyno jkomyno commented Jan 23, 2024

This PR closes https://github.com/prisma/team-orm/issues/850.

It:

  • fixes the benchmark Github action
  • adds some missing TypeScript types, which would've been useful during the debugging of the issue
  • skip query params tracking in query-result recording, as they caused trouble with BigInt results, and were not necessary.

Before this PR, we also attempted serialising results with BigInt via superjson, but apparently the serialized strings can't easily be used as hashmap keys (perhpas they're not generated deterministically?)

@jkomyno jkomyno added this to the 5.9.0 milestone Jan 23, 2024
@jkomyno jkomyno marked this pull request as ready for review January 23, 2024 09:14
@jkomyno jkomyno requested a review from a team as a code owner January 23, 2024 09:14
@jkomyno jkomyno requested review from miguelff and Druue and removed request for a team January 23, 2024 09:14
@jkomyno jkomyno self-assigned this Jan 23, 2024
Copy link
Contributor

WASM Size

Engine This PR Base branch Diff
WASM 2.334MiB 2.334MiB 0.000B
WASM (gzip) 907.092KiB 907.092KiB 0.000B

Copy link
Contributor

github-actions bot commented Jan 23, 2024

✅ WASM query-engine performance won't change substantially (1.004x)

Full benchmark report
DATABASE_URL="postgresql://postgres:postgres@localhost:5432/bench?schema=imdb_bench&sslmode=disable" \
node --experimental-wasm-modules query-engine/driver-adapters/executor/dist/bench.mjs
cpu: AMD EPYC 7763 64-Core Processor
runtime: node v18.19.0 (x64-linux)

benchmark                   time (avg)             (min … max)       p75       p99      p995
-------------------------------------------------------------- -----------------------------
• movies.findMany() (all - 25000)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  312.82 ms/iter (309.58 ms … 318.61 ms) 314.35 ms 318.61 ms 318.61 ms
Web Assembly: Latest    398.15 ms/iter (393.85 ms … 410.85 ms) 397.87 ms 410.85 ms 410.85 ms
Web Assembly: Current   394.37 ms/iter (390.27 ms … 402.88 ms) 396.28 ms 402.88 ms 402.88 ms
Node API: Current       229.05 ms/iter (220.88 ms … 237.18 ms) 232.25 ms 237.18 ms 237.18 ms

summary for movies.findMany() (all - 25000)
  Web Assembly: Current
   1.72x slower than Node API: Current
   1.26x slower than Web Assembly: Baseline
   1.01x faster than Web Assembly: Latest

• movies.findMany({ take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   12.63 ms/iter   (11.91 ms … 19.28 ms)  12.29 ms  19.28 ms  19.28 ms
Web Assembly: Latest     15.79 ms/iter    (15.54 ms … 20.7 ms)  15.67 ms   20.7 ms   20.7 ms
Web Assembly: Current    15.79 ms/iter   (15.45 ms … 17.82 ms)  15.83 ms  17.82 ms  17.82 ms
Node API: Current         9.38 ms/iter    (9.14 ms … 10.48 ms)    9.4 ms  10.48 ms  10.48 ms

summary for movies.findMany({ take: 2000 })
  Web Assembly: Current
   1.68x slower than Node API: Current
   1.25x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movies.findMany({ where: {...}, take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline    2.01 ms/iter     (1.84 ms … 3.44 ms)   1.94 ms   3.39 ms   3.43 ms
Web Assembly: Latest      2.52 ms/iter     (2.39 ms … 3.79 ms)   2.49 ms   3.39 ms   3.43 ms
Web Assembly: Current     2.53 ms/iter     (2.42 ms … 4.23 ms)    2.5 ms    4.2 ms   4.22 ms
Node API: Current         1.61 ms/iter     (1.49 ms … 3.26 ms)    1.6 ms   2.29 ms   2.54 ms

summary for movies.findMany({ where: {...}, take: 2000 })
  Web Assembly: Current
   1.58x slower than Node API: Current
   1.26x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline    12.2 ms/iter    (11.83 ms … 13.3 ms)  12.25 ms   13.3 ms   13.3 ms
Web Assembly: Latest     15.81 ms/iter   (15.51 ms … 18.82 ms)  15.75 ms  18.82 ms  18.82 ms
Web Assembly: Current    15.82 ms/iter   (15.45 ms … 17.68 ms)  15.92 ms  17.68 ms  17.68 ms
Node API: Current         9.57 ms/iter     (9.25 ms … 9.96 ms)   9.74 ms   9.96 ms   9.96 ms

summary for movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.65x slower than Node API: Current
   1.3x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline    1.94 ms/iter     (1.83 ms … 6.68 ms)   1.91 ms   3.16 ms   3.21 ms
Web Assembly: Latest      2.46 ms/iter     (2.37 ms … 3.23 ms)   2.46 ms   3.01 ms   3.12 ms
Web Assembly: Current     2.52 ms/iter      (2.4 ms … 4.09 ms)   2.49 ms   3.95 ms   4.06 ms
Node API: Current          1.6 ms/iter     (1.51 ms … 2.65 ms)    1.6 ms    2.1 ms   2.11 ms

summary for movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.58x slower than Node API: Current
   1.3x slower than Web Assembly: Baseline
   1.03x slower than Web Assembly: Latest

• movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   12.13 ms/iter   (11.83 ms … 12.69 ms)  12.19 ms  12.69 ms  12.69 ms
Web Assembly: Latest     15.68 ms/iter   (15.56 ms … 15.91 ms)  15.76 ms  15.91 ms  15.91 ms
Web Assembly: Current    15.73 ms/iter   (15.42 ms … 16.02 ms)  15.84 ms  16.02 ms  16.02 ms
Node API: Current         9.54 ms/iter     (9.09 ms … 11.4 ms)    9.7 ms   11.4 ms   11.4 ms

summary for movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.65x slower than Node API: Current
   1.3x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline    1.92 ms/iter      (1.82 ms … 3.1 ms)   1.91 ms   2.46 ms   2.73 ms
Web Assembly: Latest      2.48 ms/iter     (2.37 ms … 3.94 ms)   2.47 ms   3.79 ms   3.83 ms
Web Assembly: Current     2.47 ms/iter     (2.39 ms … 3.84 ms)   2.47 ms   3.09 ms   3.22 ms
Node API: Current         1.61 ms/iter      (1.5 ms … 2.01 ms)   1.62 ms   1.86 ms   1.99 ms

summary for movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.54x slower than Node API: Current
   1.29x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  932.86 µs/iter   (846.43 µs … 2.68 ms) 919.58 µs   1.54 ms   1.55 ms
Web Assembly: Latest      1.19 ms/iter     (1.13 ms … 1.89 ms)    1.2 ms   1.71 ms   1.76 ms
Web Assembly: Current     1.22 ms/iter     (1.14 ms … 2.06 ms)   1.21 ms   1.86 ms   2.01 ms
Node API: Current       862.36 µs/iter    (777.77 µs … 1.1 ms) 871.89 µs   1.06 ms   1.09 ms

summary for movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
  Web Assembly: Current
   1.41x slower than Node API: Current
   1.31x slower than Web Assembly: Baseline
   1.02x slower than Web Assembly: Latest

• movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  924.49 µs/iter   (882.71 µs … 1.42 ms) 930.08 µs   1.19 ms   1.25 ms
Web Assembly: Latest      1.18 ms/iter     (1.13 ms … 1.89 ms)   1.19 ms   1.55 ms   1.77 ms
Web Assembly: Current     1.19 ms/iter     (1.13 ms … 1.56 ms)    1.2 ms   1.43 ms   1.47 ms
Node API: Current       889.08 µs/iter   (774.97 µs … 2.45 ms) 886.02 µs   1.35 ms   1.47 ms

summary for movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
  Web Assembly: Current
   1.34x slower than Node API: Current
   1.29x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

After changes in e595258

Copy link

codspeed-hq bot commented Jan 23, 2024

CodSpeed Performance Report

Merging #4667 will not alter performance

Comparing fix/skip-params-tracking-in-recording-and-add-types (e595258) with main (6d02932)

Summary

✅ 11 untouched benchmarks

@miguelff miguelff merged commit e086e3a into main Jan 23, 2024
113 of 114 checks passed
@miguelff miguelff deleted the fix/skip-params-tracking-in-recording-and-add-types branch January 23, 2024 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants