Upgrade confirmation height table to include cemented frontier #2481
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
wezrule
added
documentation
cryptocode
reviewed
Jan 15, 2020
cryptocode
previously approved these changes
Jan 15, 2020
guilhermelawless
previously approved these changes
Jan 15, 2020
wezrule
dismissed stale reviews from guilhermelawless and cryptocode
via
ed2a1e4
guilhermelawless
previously approved these changes
Jan 15, 2020
guilhermelawless
previously approved these changes
Jan 15, 2020
cryptocode
previously approved these changes
Jan 15, 2020
wezrule
dismissed stale reviews from cryptocode and guilhermelawless
via
90d5168
cryptocode
approved these changes
Jan 16, 2020
wezrule
added a commit
to wezrule/raiblocks
that referenced
this pull request
Jan 22, 2020
…urrency#2481) * Upgrade confirmation height table to include cemented frontier * Stein review comments * Add //clang-format block around lambda * Update comment (thanks Gui) * Upgrade to v17 instead * Update comments
wezrule
added a commit
to wezrule/raiblocks
that referenced
this pull request
Jan 22, 2020
…urrency#2481) * Upgrade confirmation height table to include cemented frontier * Stein review comments * Add //clang-format block around lambda * Update comment (thanks Gui) * Upgrade to v17 instead * Update comments
wezrule
added a commit
that referenced
this pull request
Jan 24, 2020
* Node telemetry * Add genesis block * Don't checkif at the end of the stream with UDP to support compatibility with different message versions * Add single request and optional blocking * Formatting * Use nano locks * Fix clang build * Out of date comments * Update some names of classes/functions/variables * Fix for disconnected peer channels & some cleanup * Remove unnecessary lock requirement in active_transactions::lock. (#2475) * Remove unnecessarry lock requirement in active_transactions::lock. This also removes some code-smell around passing in a bool to determine if the lock should actually be acquired. * Using a timeout counter in line with other test idioms. * Confirmation solicitor revamp (#2472) * Optional minimum version when querying representatives from crawler * Revamping confirmation_solicitor to mimick previous active_transactions behavior * Use a time-based approach to throttle confirmation requests and block flooding * Addressing Wesley review * Remove unusued node.hpp include (thanks wes) * Simplify logic by using unordered_map::operator[] which calls the default constructor if not found * Split solicitor add into broadcast+add and bring back the logic to active_transactions This brings back rate-limitting logic and modifying election variables to active_transactions only. Timings are also slightly adjusted: - Only 2 requests required before starting to flood blocks - Timings for test network * Rename flag * Only broadcast OR request confirmation in the same loop for the same election * Enclose lambda in clang-format off * Beta network reset #2 (#2476) * Delete rep_weights_beta.bin * New genesis * Change header_magic_number to one never used before * use v1.1 for actions-aws-cli (#2486) * The start of CLI tests (#2403) * The start of CLI tests * Add needed header file after merge * Serg review comments * Websocket bootstrap subscription (#2471) * Bootstrap attempt ID * Websocket bootstrap subscription * Change processed blocks factor for requeued pulls (#2437) From `processed blocks / 10000` to `processed blocks / 4096` for better processing of largest chains in case of failures * Upgrade confirmation height table to include cemented frontier (#2481) * Upgrade confirmation height table to include cemented frontier * Stein review comments * Add //clang-format block around lambda * Update comment (thanks Gui) * Upgrade to v17 instead * Update comments * Request aggregator (#2485) * Request aggregator Adds a class that runs in a new thread to pool confirmation requests by endpoint. This allows a reduction of bandwidth, vote generation, and moves some vote generation out of the I/O threads. * Use a constant for confirm_ack_hashes_max * Small code simplification * Use const transaction * Disable clang-format for lambdas * Add missing deadlines and update deadline before poll block * Use a scoped lock_guard pattern and initialize start in-class * Misc. fixes and documentation * use clang-format-8 (#2490) There is some funkiness with clang-format-9 and lambdas that are not honored * Formatting fix so clangformat 8 applies cleanly (#2491) * Support multiple work peers in the same host (#2477) * Correct check for peers when creating work * Support multiple work peers in the same address * Send cancels despite errors to account for non-conforming implementations * Add tests using a fake work peer, acting as good, malicious or slow * Comment * Formatting * Fix multiple response error in RPC * Formatting * Add extra error handling when specifying port and not address with the RPC (and vice versa) * Formatting * Serg comments * Formatting * Mask not needed * Missed file check in for mask removal * Fix test failures * Gui suggestions * Formatting * Fix assert with test on Windows and some cleanup * Lower number of nodes in node_telemetry.many_nodes when using sanitizers Co-authored-by: clemahieu <clemahieu@gmail.com> Co-authored-by: Guilherme Lawless <guilherme.lawless@gmail.com> Co-authored-by: Russel Waters <vaelstrom@gmail.com> Co-authored-by: Sergey Kroshnin <sergiysw@gmail.com> Co-authored-by: cryptocode <stein@nano.org>
zhyatt
added
rpc
zhyatt
removed
the
blocker
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a preparation step for a confirmation height redesign I am doing. I'm modifying the confirmation height table to also include the hash at that height (cemented frontier). Currently the confirmation height processor starts from a hash in a chain and works downwards until it hits the first cemented block, because there's no other way to reach it. Then it starts traversing the bottom and cementing those blocks. This works fine when storing the receive blocks as we iterate, but this can lead to a lot of memory being used if long chains are traversed and it doesn't scale. It also doesn't play nicely with a bounded memory algorithm, because we would be wasting a lot of read IO traversing accounts many times from the top. I want to start from the bottom (cemented frontier) and work upwards, this limits waste and also cements the bottom most blocks quicker. To do this I need to know what the block hash is at that height though, which is what I am now storing.
Updated some conf heights tests which used multiple nodes, to just use 1 which makes debugging simpler.
Upgrade, this was from a v14 ledger (Edit: Outdated as we now use v17) (I left the later portion of the logs from v14 -> v15 at the top):