This repository has been archived by the owner on Jan 13, 2025. It is now read-only.
Report new received crds signatures and their respective origins to metrics #32504
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 7 commits
July 14, 2023 14:11
added 8 commits
July 14, 2023 15:25
Codecov Report
@@ Coverage Diff @@
## master #32504 +/- ##
=======================================
Coverage 82.0% 82.0%
=======================================
Files 780 780
Lines 210789 210798 +9
=======================================
+ Hits 172931 172950 +19
+ Misses 37858 37848 -10 |
behzadnouri
reviewed
Jul 17, 2023
behzadnouri
reviewed
Jul 18, 2023
added 4 commits
July 18, 2023 10:30
behzadnouri
reviewed
Jul 19, 2023
added 2 commits
July 19, 2023 13:31
gregcusack
commented
Jul 19, 2023
behzadnouri
previously approved these changes
Jul 19, 2023
behzadnouri
approved these changes
Jul 19, 2023
|
testnet node confirmed the ~1 message report per second with |
gregcusack
added
the
automerge
was this 1 message per minute or per second? |
ahhh sorry! this is 1 message PER MINUTE. |
behzadnouri
approved these changes
Jul 20, 2023
mergify
bot
removed
the
automerge
|
automerge label removed due to a CI failure |
mergify bot
pushed a commit
that referenced
this pull request
Aug 1, 2023
…etrics (#32504) * screwed up old branch and syncing with upstream branch * add fixed size ring buff instead of variable sized vecdeque for reporting signatures * modify difficulty to take in n 0 bits and check if signature ending ends in n 0 bits * update to only push every 18 trailing zero bits. and clean up * report origin with signature. and set trailing 0s to 8 for local testing * change back to 18 trailing zeros and rm unused imports * run cargo rmt * run ./scripts/cargo-for-all-lock-files.sh tree * allow integer arithmetic for bit comparison * rm unused lifetime * rm duplicate entry? * re implement ring buf * put ringbuf in sorted order * ringbuf in cargo.toml now in sorted order * rm ring buf, refactor, fix trailing zero bug * fix bug in trailing zeros. was comparing wrong ones * fix needless range loop bug * fix bug * change trailing zero checking to build in methods and only report first 8 bytes of signature and origin pubkey * report full origin string and first 8 bytes of signature * set SIGNATURE_SAMPLE_TRAILING_ZEROS back to 18 * forgot to run cargo tree * avoid panic and change working * rm bs58 * pass in Option<String> into datapoint_info * shorten metric names (cherry picked from commit 80f7082)
gregcusack
pushed a commit
that referenced
this pull request
Aug 1, 2023
…etrics (#32504) * screwed up old branch and syncing with upstream branch * add fixed size ring buff instead of variable sized vecdeque for reporting signatures * modify difficulty to take in n 0 bits and check if signature ending ends in n 0 bits * update to only push every 18 trailing zero bits. and clean up * report origin with signature. and set trailing 0s to 8 for local testing * change back to 18 trailing zeros and rm unused imports * run cargo rmt * run ./scripts/cargo-for-all-lock-files.sh tree * allow integer arithmetic for bit comparison * rm unused lifetime * rm duplicate entry? * re implement ring buf * put ringbuf in sorted order * ringbuf in cargo.toml now in sorted order * rm ring buf, refactor, fix trailing zero bug * fix bug in trailing zeros. was comparing wrong ones * fix needless range loop bug * fix bug * change trailing zero checking to build in methods and only report first 8 bytes of signature and origin pubkey * report full origin string and first 8 bytes of signature * set SIGNATURE_SAMPLE_TRAILING_ZEROS back to 18 * forgot to run cargo tree * avoid panic and change working * rm bs58 * pass in Option<String> into datapoint_info * shorten metric names (cherry picked from commit 80f7082)
gregcusack
pushed a commit
that referenced
this pull request
Aug 1, 2023
…ns to metrics (backport of #32504) (#32674) Report new received crds signatures and their respective origins to metrics (#32504) * screwed up old branch and syncing with upstream branch * add fixed size ring buff instead of variable sized vecdeque for reporting signatures * modify difficulty to take in n 0 bits and check if signature ending ends in n 0 bits * update to only push every 18 trailing zero bits. and clean up * report origin with signature. and set trailing 0s to 8 for local testing * change back to 18 trailing zeros and rm unused imports * run cargo rmt * run ./scripts/cargo-for-all-lock-files.sh tree * allow integer arithmetic for bit comparison * rm unused lifetime * rm duplicate entry? * re implement ring buf * put ringbuf in sorted order * ringbuf in cargo.toml now in sorted order * rm ring buf, refactor, fix trailing zero bug * fix bug in trailing zeros. was comparing wrong ones * fix needless range loop bug * fix bug * change trailing zero checking to build in methods and only report first 8 bytes of signature and origin pubkey * report full origin string and first 8 bytes of signature * set SIGNATURE_SAMPLE_TRAILING_ZEROS back to 18 * forgot to run cargo tree * avoid panic and change working * rm bs58 * pass in Option<String> into datapoint_info * shorten metric names (cherry picked from commit 80f7082) Co-authored-by: Greg Cusack <greg.cusack@solana.com>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
There is currently no way to track how gossip messages propagate throughout the network.
We have no way to answer the question: for a specific message and origin, how many validators receive the message?
Summary of Changes
Report any new crds values' signature and origin to metrics
Reduce event rate by only reporting signatures with
ntrailing zeros, wherenis 1818 trailing zeros gives us a difficulty level that results in approximately 1-2 new signatures reported every minute per validator
Approximate number of new push messages received per validator:
log2(250k) = ~18
data reported as:
series:
cluster_info_crds_stats_message_signatures_receiveddata:
crds_originandcrds_signatureFixes #