Skip to content
This repository was archived by the owner on Oct 18, 2023. It is now read-only.

Unexpected disk usage amplification when DB created from a seed file #583

Open
avinassh opened this issue Aug 7, 2023 · 0 comments
Open

Comments

@avinassh
Copy link
Member

avinassh commented Aug 7, 2023

For databases created from seed files, I have noticed higher disk usage than expected. The disk space usage is 4x for primary and 3x for replicas. (the usage peaked at 4.5x and 3.5x respectively)

My seed file is 1.7GB. The disk space used in primary was 6.8GB (while peak at 8.2GB).

To recreate, you may run following commands to generate a test database:

$ sqlite3 big.db

PRAGMA journal_mode=WAL;

CREATE TABLE test (
  id INTEGER PRIMARY KEY NOT NULL, 
  x REAL NOT NULL,
  y REAL NOT NULL,
  z REAL NOT NULL
);

INSERT INTO test
    WITH RECURSIVE
      cnt( id, x, y, z) AS (
      VALUES(1 , random(), random(), random()) UNION ALL 
      SELECT id+1,random(),random(), random() FROM cnt WHERE ID<50000000)
    select * from cnt;

related: This slack thread has some internal details tested on Turso cloud

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant