[Storage] Open db in parallel when storage sharding is enabled. #15504

grao1991 · 2024-12-05T01:06:30Z

Description

How Has This Been Tested?

Key Areas to Review

Type of Change

Which Components or Systems Does This Change Impact?

Checklist

I have read and followed the CONTRIBUTING doc
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I identified and added all stakeholders and component owners affected by this change as reviewers
I tested both happy and unhappy path of the functionality
I have made corresponding changes to the documentation

trunk-io · 2024-12-05T01:06:34Z

⏱️ 5h 43m total CI duration on this PR

Slowest 15 Jobs	Cumulative Duration	Recent Runs
execution-performance / single-node-performance	1h 35m	🟥 🟩 🟥 🟩 🟩
test-target-determinator	22m	🟩 🟩 🟩 🟩 🟩
execution-performance / test-target-determinator	22m	🟩 🟩 🟩 🟩 🟩
check	18m	🟩 🟩 🟩 🟩 🟩
forge-e2e-test / forge	13m	🟥
rust-move-tests	13m	🟩
rust-move-tests	13m	🟩
rust-move-tests	13m	🟩
rust-move-tests	13m	🟩
rust-cargo-deny	13m	🟩 🟩 🟩 🟩 🟩 (+2 more)
rust-move-tests	13m	🟩
rust-images / rust-all	12m	🟩
forge-compat-test / forge	10m	🟥
forge-framework-upgrade-test / forge	9m	🟥
forge-framework-upgrade-test / forge	9m	🟥

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

storage/aptosdb/src/state_kv_db.rs

msmouse · 2024-12-05T01:46:45Z

storage/aptosdb/src/state_kv_db.rs

-            readonly,
-            sharding,
-        )
+        Self::open(db_paths, rocksdb_configs.state_kv_db_config, readonly)


let's rename to Self::open_sharded() or something?

msmouse · 2024-12-05T01:47:44Z

storage/aptosdb/src/ledger_db/mod.rs

+        let mut transaction_db = None;
+        let mut transaction_info_db = None;
+        let mut write_set_db = None;
+        THREAD_MANAGER.get_non_exe_cpu_pool().scope(|s| {


seems no particular reason to use this pool?

No. Just need some threads here.

msmouse · 2024-12-05T02:16:27Z

storage/aptosdb/src/db_options.rs

-    } else {
-        state_kv_db_column_families()
-    };
+pub(super) fn gen_state_kv_cfds(rocksdb_config: &RocksdbConfig) -> Vec<ColumnFamilyDescriptor> {


probably gen_state_kv_shard_cfds()

github-actions · 2024-12-05T07:34:44Z

✅ Forge suite `compat` success on `c2969f0b0ecd826aa1516a386aed59496a247958` ==> `067a47dbbfc151e4cedfce1480de1f759da80cd5`

Compatibility test results for c2969f0b0ecd826aa1516a386aed59496a247958 ==> 067a47dbbfc151e4cedfce1480de1f759da80cd5 (PR)
1. Check liveness of validators at old version: c2969f0b0ecd826aa1516a386aed59496a247958
compatibility::simple-validator-upgrade::liveness-check : committed: 17136.63 txn/s, latency: 1986.89 ms, (p50: 2100 ms, p70: 2100, p90: 2200 ms, p99: 2400 ms), latency samples: 550440
2. Upgrading first Validator to new version: 067a47dbbfc151e4cedfce1480de1f759da80cd5
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 7445.36 txn/s, latency: 3874.36 ms, (p50: 4300 ms, p70: 4500, p90: 4600 ms, p99: 4700 ms), latency samples: 141300
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 7510.75 txn/s, latency: 4347.09 ms, (p50: 4700 ms, p70: 4700, p90: 4900 ms, p99: 5300 ms), latency samples: 248560
3. Upgrading rest of first batch to new version: 067a47dbbfc151e4cedfce1480de1f759da80cd5
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 7799.87 txn/s, latency: 3653.86 ms, (p50: 3900 ms, p70: 4200, p90: 4700 ms, p99: 4900 ms), latency samples: 144920
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 8091.71 txn/s, latency: 4042.49 ms, (p50: 4300 ms, p70: 4400, p90: 4700 ms, p99: 5200 ms), latency samples: 264340
4. upgrading second batch to new version: 067a47dbbfc151e4cedfce1480de1f759da80cd5
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 10856.74 txn/s, latency: 2549.75 ms, (p50: 2500 ms, p70: 3100, p90: 3400 ms, p99: 3600 ms), latency samples: 191240
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 11831.23 txn/s, latency: 2699.94 ms, (p50: 2700 ms, p70: 3100, p90: 3200 ms, p99: 3500 ms), latency samples: 381300
5. check swarm health
Compatibility test for c2969f0b0ecd826aa1516a386aed59496a247958 ==> 067a47dbbfc151e4cedfce1480de1f759da80cd5 passed
Test Ok

github-actions · 2024-12-05T07:35:21Z

✅ Forge suite `realistic_env_max_load` success on `067a47dbbfc151e4cedfce1480de1f759da80cd5`

two traffics test: inner traffic : committed: 14890.96 txn/s, latency: 2666.38 ms, (p50: 2700 ms, p70: 2700, p90: 2900 ms, p99: 3000 ms), latency samples: 5661840
two traffics test : committed: 100.03 txn/s, latency: 1406.55 ms, (p50: 1400 ms, p70: 1400, p90: 1500 ms, p99: 2100 ms), latency samples: 1740
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 1.518, avg: 1.464", "ConsensusProposalToOrdered: max: 0.319, avg: 0.289", "ConsensusOrderedToCommit: max: 0.368, avg: 0.359", "ConsensusProposalToCommit: max: 0.655, avg: 0.649"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.57s no progress at version 15704 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.69s no progress at version 2448364 (avg 0.69s) [limit 16].
Test Ok

github-actions · 2024-12-05T07:37:12Z

✅ Forge suite `framework_upgrade` success on `c2969f0b0ecd826aa1516a386aed59496a247958` ==> `067a47dbbfc151e4cedfce1480de1f759da80cd5`

Compatibility test results for c2969f0b0ecd826aa1516a386aed59496a247958 ==> 067a47dbbfc151e4cedfce1480de1f759da80cd5 (PR)
Upgrade the nodes to version: 067a47dbbfc151e4cedfce1480de1f759da80cd5
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1468.28 txn/s, submitted: 1471.05 txn/s, failed submission: 2.77 txn/s, expired: 2.77 txn/s, latency: 2096.11 ms, (p50: 2100 ms, p70: 2400, p90: 2800 ms, p99: 4500 ms), latency samples: 127180
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1438.92 txn/s, submitted: 1442.88 txn/s, failed submission: 3.96 txn/s, expired: 3.96 txn/s, latency: 2084.63 ms, (p50: 2100 ms, p70: 2300, p90: 2600 ms, p99: 3500 ms), latency samples: 130880
5. check swarm health
Compatibility test for c2969f0b0ecd826aa1516a386aed59496a247958 ==> 067a47dbbfc151e4cedfce1480de1f759da80cd5 passed
Upgrade the remaining nodes to version: 067a47dbbfc151e4cedfce1480de1f759da80cd5
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1400.90 txn/s, submitted: 1403.97 txn/s, failed submission: 3.07 txn/s, expired: 3.07 txn/s, latency: 2333.86 ms, (p50: 2400 ms, p70: 2400, p90: 3000 ms, p99: 4500 ms), latency samples: 118740
Test Ok

grao1991 requested review from msmouse and lightmark as code owners December 5, 2024 01:06

grao1991 requested a review from areshand December 5, 2024 01:06

graphite-app bot reviewed Dec 5, 2024

View reviewed changes

storage/aptosdb/src/state_kv_db.rs Outdated Show resolved Hide resolved

grao1991 force-pushed the grao_open_db branch from 8d01152 to 4d76468 Compare December 5, 2024 01:10

grao1991 enabled auto-merge (squash) December 5, 2024 01:12

msmouse reviewed Dec 5, 2024

View reviewed changes

grao1991 force-pushed the grao_open_db branch from 4d76468 to fbf050c Compare December 5, 2024 01:53

grao1991 requested a review from msmouse December 5, 2024 01:54

msmouse approved these changes Dec 5, 2024

View reviewed changes