Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix callgrind action again #718

Merged
merged 1 commit into from
Jan 18, 2024
Merged

Conversation

kazimuth
Copy link
Contributor

Description of Changes

Implement the change @cloutiertyler suggested.

@kazimuth
Copy link
Contributor Author

callgrind please

Copy link

github-actions bot commented Jan 12, 2024

Benchmark results # Callgrind Benchmark Report

These benchmarks were run using callgrind,
an instruction-level profiler. They allow comparisons between sqlite (sqlite), SpacetimeDB running through a module (stdb_module),
and the underlying SpacetimeDB data storage engine (stdb_raw).

Of these, sqlite is the least noisy, stdb_raw is intermediate, and stdb_module is the noisiest. This is due to uncontrollable nondeterminism,
such as thread ordering and filesystem interactions.

Note: there are currently a minimal number of stdb_module benchmarks, pending a fix for an unfortunate interaction between callgrind and sled.
Their benchmark numbers are correct, just expensive to collect.

Legend:

  • preload: number of rows pre-loaded into the database.
  • count: number of rows touched by a transaction.
  • index types:
    • unique: a single, unique index on the id column.
    • non_unique: non-unique, no indexes.
    • btrees: non-unique, separate btree index on every column. (Not a multi-column index!)
  • schemas:
    • person(id: u32, name: String, age: u64) is a small row containing a string.
    • location(id: u32, x: u64, y: u64) is a small row containing some ints.

filter

db data_type index_strategy count preload table_name new instructions old instructions delta instructions
sqlite u64 non_unique 1 100 person 43162 43162 +0.00%
sqlite u64 non_unique 4 100 person 44512 44512 +0.00%
sqlite u64 non_unique 16 100 person 49920 49920 +0.00%
sqlite u64 non_unique 64 100 person 71572 71572 +0.00%
sqlite u64 btrees 1 100 person 10658 10658 +0.00%
sqlite u64 btrees 4 100 person 14247 14247 +0.00%
sqlite u64 btrees 16 100 person 28865 28868 -0.01%
sqlite u64 btrees 64 100 person 86848 86848 +0.00%
sqlite string btrees 1 100 person 11406 11393 +0.11%
sqlite string btrees 4 100 person 15233 15246 -0.09%
sqlite string btrees 16 100 person 30514 30514 +0.00%
sqlite string btrees 64 100 person 91662 91662 +0.00%
stdb_raw u64 non_unique 1 100 person 155445 155445 +0.00%
stdb_raw u64 non_unique 4 100 person 157725 157725 +0.00%
stdb_raw u64 non_unique 16 100 person 166845 166845 +0.00%
stdb_raw u64 non_unique 64 100 person 203239 203239 +0.00%
stdb_raw u64 btrees 1 100 person 19357 19357 +0.00%
stdb_raw u64 btrees 4 100 person 26562 26562 +0.00%
stdb_raw u64 btrees 16 100 person 55905 55905 +0.00%
stdb_raw u64 btrees 64 100 person 170829 170786 +0.03%
stdb_raw string non_unique 1 100 person 177431 177431 +0.00%
stdb_raw string non_unique 4 100 person 179716 179716 +0.00%
stdb_raw string non_unique 16 100 person 189006 189006 +0.00%
stdb_raw string non_unique 64 100 person 225718 225718 +0.00%
stdb_raw string btrees 1 100 person 20410 20404 +0.03%
stdb_raw string btrees 4 100 person 28026 28030 -0.01%
stdb_raw string btrees 16 100 person 56545 56552 -0.01%
stdb_raw string btrees 64 100 person 171610 171616 -0.00%

insert bulk

db index_strategy count preload table_name new instructions old instructions delta instructions
sqlite unique 1 50 person 16782 16785 -0.02%
sqlite unique 25 50 person 90749 90749 +0.00%
sqlite unique 50 50 person 168405 168405 +0.00%
sqlite unique 100 50 person 324174 324174 +0.00%
stdb_raw unique 1 50 person 65621 65618 +0.00%
stdb_raw unique 25 50 person 1295124 1295140 -0.00%
stdb_raw unique 50 50 person 2466695 2473195 -0.26%
stdb_raw unique 100 50 person 4927179 4927654 -0.01%

iterate

db index_strategy count table_name new instructions old instructions delta instructions
sqlite unique 1 person 8565 8565 +0.00%
sqlite unique 25 person 24361 24361 +0.00%
sqlite unique 50 person 40825 40825 +0.00%
sqlite unique 100 person 73748 73748 +0.00%
stdb_raw unique 1 person 12925 12925 +0.00%
stdb_raw unique 25 person 39253 39253 +0.00%
stdb_raw unique 50 person 66692 66692 +0.00%
stdb_raw unique 100 person 121556 121556 +0.00%

serialize_product_value

count format new instructions old instructions delta instructions
50 json 28829 28829 +0.00%
50 bsatn 15989 15989 +0.00%
100 json 56344 56344 +0.00%
100 bsatn 28806 28806 +0.00%

empty transaction

db new instructions old instructions delta instructions
sqlite 3672 3672 +0.00%
stdb_raw 7646 7646 +0.00%
stdb_module 31769 32372 -1.86%

Copy link
Contributor

@cloutiertyler cloutiertyler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delicious

@kazimuth kazimuth added this pull request to the merge queue Jan 18, 2024
Merged via the queue into master with commit bcffb5d Jan 18, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants