Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(schema-engine): disabled implicit transaction for batch statements starting from CockroachDB v22.2 #4632

Merged

Conversation

jkomyno
Copy link
Contributor

@jkomyno jkomyno commented Jan 9, 2024

This PR closes prisma/prisma#20851.

It performs the same logic previously applied to CockroachDB v22.2 only, to any CockroachDB version equal or more recent than v22.2.

It also adds the CockroachDB-specific logic to parse the semantic versions. This implies that, at some point, we could disable the enable_implicit_transaction_for_batch_statements once a new major/minor version is reached.

@jkomyno jkomyno self-assigned this Jan 9, 2024
@jkomyno jkomyno added this to the 5.8.0 milestone Jan 9, 2024
@jkomyno jkomyno marked this pull request as ready for review January 9, 2024 11:58
@jkomyno jkomyno requested a review from a team as a code owner January 9, 2024 11:58
@jkomyno jkomyno requested review from laplab and Weakky and removed request for a team January 9, 2024 11:58
Copy link
Contributor

github-actions bot commented Jan 9, 2024

WASM Size

Engine This PR Base branch Diff
WASM 2.750MiB 2.750MiB 0.000B
WASM (gzip) 1.009MiB 1.009MiB 0.000B

Copy link

codspeed-hq bot commented Jan 9, 2024

CodSpeed Performance Report

Merging #4632 will not alter performance

Comparing feat/cockroachdb-enable-implicit-transaction-for-batch-statements (4d137a7) with main (16ee652)

Summary

✅ 11 untouched benchmarks

Copy link
Contributor

github-actions bot commented Jan 9, 2024

✅ WASM query-engine: no benchmarks have regressed

Full benchmark report
DATABASE_URL="postgresql://postgres:postgres@localhost:5432/bench?schema=imdb_bench&sslmode=disable" \
node --experimental-wasm-modules query-engine/driver-adapters/executor/dist/bench.mjs
cpu: AMD EPYC 7763 64-Core Processor
runtime: node v18.19.0 (x64-linux)

benchmark                   time (avg)             (min … max)       p75       p99      p995
-------------------------------------------------------------- -----------------------------
• movies.findMany() (all - 25000)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   325.9 ms/iter (321.68 ms … 349.12 ms) 324.32 ms 349.12 ms 349.12 ms
Web Assembly: Latest     348.7 ms/iter (344.61 ms … 360.57 ms) 348.22 ms 360.57 ms 360.57 ms
Web Assembly: Current    312.5 ms/iter (308.42 ms … 325.25 ms)    312 ms 325.25 ms 325.25 ms
Node API: Current       234.63 ms/iter (219.35 ms … 256.22 ms) 242.24 ms 256.22 ms 256.22 ms

summary for movies.findMany() (all - 25000)
  Web Assembly: Current
   1.33x slower than Node API: Current
   1.04x faster than Web Assembly: Baseline
   1.12x faster than Web Assembly: Latest

• movies.findMany({ take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   13.14 ms/iter   (12.49 ms … 17.96 ms)   13.1 ms  17.96 ms  17.96 ms
Web Assembly: Latest        14 ms/iter   (13.38 ms … 22.52 ms)  13.73 ms  22.52 ms  22.52 ms
Web Assembly: Current    12.45 ms/iter   (12.24 ms … 15.05 ms)  12.44 ms  15.05 ms  15.05 ms
Node API: Current         9.31 ms/iter    (8.79 ms … 14.44 ms)   9.29 ms  14.44 ms  14.44 ms

summary for movies.findMany({ take: 2000 })
  Web Assembly: Current
   1.34x slower than Node API: Current
   1.06x faster than Web Assembly: Baseline
   1.12x faster than Web Assembly: Latest

• movies.findMany({ where: {...}, take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline    2.34 ms/iter      (2.01 ms … 3.9 ms)   2.29 ms    3.7 ms   3.71 ms
Web Assembly: Latest      2.46 ms/iter     (2.22 ms … 6.86 ms)   2.42 ms   3.61 ms   3.66 ms
Web Assembly: Current     2.13 ms/iter     (1.86 ms … 3.47 ms)   2.04 ms   3.44 ms   3.45 ms
Node API: Current         1.55 ms/iter     (1.44 ms … 1.99 ms)   1.55 ms   1.83 ms   1.92 ms

summary for movies.findMany({ where: {...}, take: 2000 })
  Web Assembly: Current
   1.38x slower than Node API: Current
   1.1x faster than Web Assembly: Baseline
   1.15x faster than Web Assembly: Latest

• movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   13.16 ms/iter    (12.8 ms … 16.64 ms)  13.18 ms  16.64 ms  16.64 ms
Web Assembly: Latest      14.4 ms/iter   (14.07 ms … 15.11 ms)  14.47 ms  15.11 ms  15.11 ms
Web Assembly: Current    12.33 ms/iter    (12.2 ms … 12.78 ms)  12.36 ms  12.78 ms  12.78 ms
Node API: Current         9.86 ms/iter    (8.82 ms … 14.59 ms)   9.87 ms  14.59 ms  14.59 ms

summary for movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.25x slower than Node API: Current
   1.07x faster than Web Assembly: Baseline
   1.17x faster than Web Assembly: Latest

• movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline    2.06 ms/iter     (1.97 ms … 3.32 ms)   2.05 ms   2.76 ms   2.97 ms
Web Assembly: Latest      2.23 ms/iter     (2.15 ms … 3.29 ms)   2.23 ms   2.89 ms      3 ms
Web Assembly: Current     1.96 ms/iter     (1.85 ms … 4.68 ms)   1.94 ms   3.05 ms   3.18 ms
Node API: Current         1.56 ms/iter      (1.47 ms … 1.8 ms)   1.57 ms   1.76 ms   1.79 ms

summary for movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.25x slower than Node API: Current
   1.05x faster than Web Assembly: Baseline
   1.14x faster than Web Assembly: Latest

• movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   13.18 ms/iter   (12.76 ms … 17.68 ms)  13.16 ms  17.68 ms  17.68 ms
Web Assembly: Latest     14.36 ms/iter   (14.04 ms … 14.72 ms)  14.42 ms  14.72 ms  14.72 ms
Web Assembly: Current    12.69 ms/iter   (12.27 ms … 18.04 ms)  12.41 ms  18.04 ms  18.04 ms
Node API: Current         9.36 ms/iter    (8.93 ms … 13.54 ms)   9.41 ms  13.54 ms  13.54 ms

summary for movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.36x slower than Node API: Current
   1.04x faster than Web Assembly: Baseline
   1.13x faster than Web Assembly: Latest

• movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline    2.03 ms/iter     (1.88 ms … 3.14 ms)   2.03 ms    2.7 ms    2.9 ms
Web Assembly: Latest      2.22 ms/iter     (2.14 ms … 3.05 ms)   2.21 ms      3 ms   3.04 ms
Web Assembly: Current     1.92 ms/iter     (1.84 ms … 2.68 ms)   1.91 ms   2.44 ms   2.44 ms
Node API: Current         1.59 ms/iter      (1.5 ms … 1.85 ms)    1.6 ms   1.79 ms    1.8 ms

summary for movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.21x slower than Node API: Current
   1.06x faster than Web Assembly: Baseline
   1.16x faster than Web Assembly: Latest

• movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  988.78 µs/iter   (923.68 µs … 1.59 ms) 984.13 µs   1.45 ms   1.53 ms
Web Assembly: Latest      1.02 ms/iter      (952.58 µs … 2 ms)   1.01 ms   1.51 ms   1.58 ms
Web Assembly: Current   964.88 µs/iter    (895.33 µs … 1.7 ms)  951.9 µs   1.56 ms   1.58 ms
Node API: Current       859.14 µs/iter   (766.46 µs … 1.86 ms) 869.33 µs   1.26 ms   1.69 ms

summary for movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
  Web Assembly: Current
   1.12x slower than Node API: Current
   1.02x faster than Web Assembly: Baseline
   1.06x faster than Web Assembly: Latest

• movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  962.14 µs/iter   (915.22 µs … 1.46 ms) 962.08 µs   1.29 ms   1.35 ms
Web Assembly: Latest    989.29 µs/iter    (936.56 µs … 3.8 ms) 982.63 µs   1.38 ms   1.49 ms
Web Assembly: Current   939.21 µs/iter   (880.55 µs … 1.45 ms) 933.83 µs   1.31 ms   1.41 ms
Node API: Current       847.72 µs/iter    (774.52 µs … 1.2 ms) 864.97 µs   1.01 ms   1.03 ms

summary for movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
  Web Assembly: Current
   1.11x slower than Node API: Current
   1.02x faster than Web Assembly: Baseline
   1.05x faster than Web Assembly: Latest

After changes in 4d137a7

Comment on lines +454 to +518
// This test follows https://github.com/prisma/prisma-engines/pull/4632.
#[test_connector(tags(CockroachDb))]
fn decimal_to_boolean_migrations_work(api: TestApi) {
let dir = api.create_migrations_directory();

let dm1 = r#"
datasource db {
provider = "cockroachdb"
url = env("TEST_DATABASE_URL")
}

model Cat {
id BigInt @id @default(autoincrement())
tag Decimal
}
"#;

api.create_migration("create-cats-decimal", &dm1, &dir)
.send_sync()
.assert_migration_directories_count(1)
.assert_migration("create-cats-decimal", move |migration| {
let expected_script = expect![[r#"
-- CreateTable
CREATE TABLE "Cat" (
"id" INT8 NOT NULL DEFAULT unique_rowid(),
"tag" DECIMAL(65,30) NOT NULL,

CONSTRAINT "Cat_pkey" PRIMARY KEY ("id")
);
"#]];

migration.expect_contents(expected_script)
});

let dm2 = r#"
datasource db {
provider = "cockroachdb"
url = env("TEST_DATABASE_URL")
}

model Cat {
id BigInt @id @default(autoincrement())
tag Boolean
}
"#;

api.create_migration("migrate-cats-boolean", &dm2, &dir)
.send_sync()
.assert_migration_directories_count(2)
.assert_migration("migrate-cats-boolean", move |migration| {
let expected_script = expect![[r#"
/*
Warnings:

- Changed the type of `tag` on the `Cat` table. No cast exists, the column would be dropped and recreated, which cannot be done if there is data, since the column is required.

*/
-- AlterTable
ALTER TABLE "Cat" DROP COLUMN "tag";
ALTER TABLE "Cat" ADD COLUMN "tag" BOOL NOT NULL;
"#]];

migration.expect_contents(expected_script)
});
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to the reviewer: although I expected this test to fail against main when run on CockroachDB 23.1, it doesn't :/

This implies that somewhere in here we're not testing the "bad case for CockroachDB 22.2+" properly.

@Jolg42 Jolg42 modified the milestones: 5.8.0, 5.9.0 Jan 10, 2024
Copy link
Contributor

@Jolg42 Jolg42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

for info, I added a new build in our docker image prismagraphql/cockroachdb-custom for 23.1.13
https://hub.docker.com/r/prismagraphql/cockroachdb-custom/tags

That updated 23.1 as well, which should in theory be 23.1.13 too now.

@Jolg42
Copy link
Contributor

Jolg42 commented Jan 11, 2024

SE: integration tests / cockroach_23_1 (pull_request)
https://github.com/prisma/prisma-engines/actions/runs/7487743713/job/20380785635?pr=4632#step:7:2034

Looks like it's now failing like in
chore(deps): update prismagraphql/cockroachdb-custom:23.1 docker digest to b6cbbf9 #4504
#4504

@jkomyno
Copy link
Contributor Author

jkomyno commented Jan 11, 2024

SE: integration tests / cockroach_23_1 (pull_request) https://github.com/prisma/prisma-engines/actions/runs/7487743713/job/20380785635?pr=4632#step:7:2034

Looks like it's now failing like in chore(deps): update prismagraphql/cockroachdb-custom:23.1 docker digest to b6cbbf9 #4504 #4504

I see the failing tests having a maxValue attribute in sequence as a test diff.
That comes out of https://github.com/prisma/prisma-engines/blob/0de06ae93d1d4ec6268874052d9c303e539e72be/schema-engine/connectors/sql-schema-connector/src/introspection/rendering/defaults.rs.

It's currently unclear to me why updating a patch version of CockroachDB would result in such a disruptive change.

@jkomyno
Copy link
Contributor Author

jkomyno commented Jan 11, 2024

I will revert b027610 for now, and modify + review #4504 as part of Tech Debt Friday.

@Jolg42
Copy link
Contributor

Jolg42 commented Jan 11, 2024

Indeed, a revert sounds good as we can tackle this separately.

@jkomyno
Copy link
Contributor Author

jkomyno commented Jan 11, 2024

(Seeing as you're already assigned to #4504, @Jolg42, we can also pair together on it!)

@jkomyno jkomyno merged commit 79f6f09 into main Jan 26, 2024
138 checks passed
@jkomyno jkomyno deleted the feat/cockroachdb-enable-implicit-transaction-for-batch-statements branch January 26, 2024 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ERROR: column "..." being dropped, try again later when applying migrations with CRDB 23.1
2 participants