BP-44: Ledger storage metrics enhancement #2862
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dlg99
approved these changes
Oct 28, 2021
|
rerun failure checks |
1 similar comment
|
rerun failure checks |
Vanlightly
force-pushed
the
bp-44-ledger-storage
branch
2 times, most recently
from
56b169d to
d20790a
Compare
Various enhancements related to DbLedgerStorage: - All DbLedgerStorage stats now report their ledgerDir. - Read cache hit/misses now differentiated from write cache. - Read/write cache hit/misses now counters, not OpStatsLoggers. The OpStatsLoggers are relatively expensive and the counter values are most important here. - Stats where thread info useful are now thread-scoped. In general the focus is on time based metrics for operations carried out by the read/write thread pools. - Time spent counters on sub-operations for reads (entry log, locations index, readahead). - Flush time started moved to after lock to avoid 200% time utilization calculations. Can reach 200% because one thread is busy flusing and another busy waiting on the lock. - Time spent counters on sub-operations for flushes (entry log, locations index, ledgers index). - DbStorage thread reports time utilization. - SyncThread reports time utilization. Some new gauges that report configuration values which are useful for utilization calculations in dashboards/alerts: - Write cache max size. - Number of ledger dirs. - Readahead batch size.
Vanlightly
force-pushed
the
bp-44-ledger-storage
branch
from
d20790a to
9dcda91
Compare
Ghatage
pushed a commit
to sijie/bookkeeper
that referenced
this pull request
Jul 12, 2024
* Ledger storage metrics enhancement Various enhancements related to DbLedgerStorage: - All DbLedgerStorage stats now report their ledgerDir. - Read cache hit/misses now differentiated from write cache. - Read/write cache hit/misses now counters, not OpStatsLoggers. The OpStatsLoggers are relatively expensive and the counter values are most important here. - Stats where thread info useful are now thread-scoped. In general the focus is on time based metrics for operations carried out by the read/write thread pools. - Time spent counters on sub-operations for reads (entry log, locations index, readahead). - Flush time started moved to after lock to avoid 200% time utilization calculations. Can reach 200% because one thread is busy flusing and another busy waiting on the lock. - Time spent counters on sub-operations for flushes (entry log, locations index, ledgers index). - DbStorage thread reports time utilization. - SyncThread reports time utilization. Some new gauges that report configuration values which are useful for utilization calculations in dashboards/alerts: - Write cache max size. - Number of ledger dirs. - Readahead batch size. * Fix checkstyle
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
See BP-44 for full motivation
Changes
Various enhancements related to DbLedgerStorage:
The OpStatsLoggers are relatively expensive and the counter
values are most important here.
general the focus is on time based metrics for operations
carried out by the read/write thread pools.
locations index, readahead).
utilization calculations. Can reach 200% because one thread
is busy flushing and another busy waiting on the lock.
locations index, ledgers index).
Some new gauges that report configuration values which are useful
for utilization calculations in dashboards/alerts:
Master Issue: #2834