Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

config: adjust shard cache sizes #10409

Merged
merged 2 commits into from
Jan 12, 2024

Conversation

pugachAG
Copy link
Contributor

@pugachAG pugachAG commented Jan 11, 2024

  • Do not apply default size increase introduced in feat: Change default shard trie cache size to 500MB #10373 to view_trie_cache since view calls are not latency-sensitive
  • Opt out shard 1 from the increase since it only contains aurora account and based on the current metrics has very low cache miss rate even with a size of 50MB
  • Configure shard 3 override of 3GB to also apply after resharding

@pugachAG pugachAG requested a review from wacban January 11, 2024 21:14
@pugachAG pugachAG requested a review from a team as a code owner January 11, 2024 21:14
Copy link

codecov bot commented Jan 11, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (0a98fbd) 72.00% compared to head (61de47b) 71.90%.
Report is 5 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #10409      +/-   ##
==========================================
- Coverage   72.00%   71.90%   -0.11%     
==========================================
  Files         718      719       +1     
  Lines      144544   144914     +370     
  Branches   144544   144914     +370     
==========================================
+ Hits       104080   104199     +119     
- Misses      35705    35942     +237     
- Partials     4759     4773      +14     
Flag Coverage Δ
backward-compatibility 0.08% <0.00%> (-0.01%) ⬇️
db-migration 0.08% <0.00%> (-0.01%) ⬇️
genesis-check 1.26% <100.00%> (+<0.01%) ⬆️
integration-tests 36.70% <100.00%> (-0.04%) ⬇️
linux 71.41% <100.00%> (-0.16%) ⬇️
linux-nightly 71.48% <100.00%> (-0.10%) ⬇️
macos 55.23% <100.00%> (-0.09%) ⬇️
pytests 1.48% <100.00%> (+<0.01%) ⬆️
sanity-checks 1.27% <100.00%> (+<0.01%) ⬆️
unittests 68.14% <100.00%> (-0.11%) ⬇️
upgradability 0.13% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@wacban wacban left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks a lot for looking into it!

@@ -240,17 +240,22 @@ impl Default for StoreConfig {
block_size: bytesize::ByteSize::kib(16),

trie_cache: TrieCacheConfig {
default_max_bytes: DEFAULT_SHARD_CACHE_TOTAL_SIZE_LIMIT,
default_max_bytes: 500_000_000,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not for this pr, but bytesize is a nice type safe helper for dealing with bytes

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

core/store/src/config.rs Show resolved Hide resolved
Comment on lines 251 to 253
(ShardUId { version: 1, shard_id: 3 }, 3_000_000_000),
// After resharding "token.sweat" account moves to shard 4
(ShardUId { version: 2, shard_id: 4 }, 3_000_000_000),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

// In simple nightshade the heavy contract "token.sweat" is in shard 3
(ShardUId { version: 1, shard_id: 3 }, 3_000_000_000),
// In simple nightshade v2 the heavy contract "token.sweat" is in shard 4
(ShardUId { version: 2, shard_id: 4 }, 3_000_000_000),

Comment on lines +256 to +257
(ShardUId { version: 1, shard_id: 1 }, 50_000_000),
(ShardUId { version: 2, shard_id: 1 }, 50_000_000),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it the default now? Does it need to be set explicitly?

Copy link
Contributor Author

@pugachAG pugachAG Jan 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nope, I've changed the default via default_max_bytes field above to 500MB

@pugachAG pugachAG added this pull request to the merge queue Jan 12, 2024
Merged via the queue into near:master with commit 38afdd5 Jan 12, 2024
18 of 20 checks passed
@pugachAG pugachAG deleted the fix-shard-cache-after-resharding branch January 12, 2024 11:04
@pugachAG pugachAG added the A-storage Area: storage and databases label Jan 12, 2024
github-merge-queue bot pushed a commit that referenced this pull request Jan 12, 2024
As suggested in
#10409 (comment) to
keep it consistent with the rest of `StoreConfig`.
posvyatokum pushed a commit that referenced this pull request Jan 22, 2024
* Do not apply default size increase introduced in #10373 to
`view_trie_cache` since view calls are not latency-sensitive
* Opt out shard 1 from the increase since it only contains aurora
account and based on the current metrics has very low cache miss rate
even with a size of 50MB
* Configure shard 3 override of 3GB to also apply after resharding
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-storage Area: storage and databases
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants