Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(pruner, storage): prune receipts & save checkpoints to database #3733

Merged
merged 22 commits into from
Jul 24, 2023

Conversation

shekhirin
Copy link
Collaborator

@shekhirin shekhirin commented Jul 12, 2023

Resolves #3697

Adds pruning of receipts the the Pruner which is activated during the live sync. Pruning of receipts is done in one go, and the progress is printed every 10k receipts. After pruning is done, writes checkpoints to the database.

@shekhirin shekhirin added C-enhancement New feature or request A-db Related to the database A-pruning Related to pruning or full node labels Jul 12, 2023
@codecov
Copy link

codecov bot commented Jul 12, 2023

Codecov Report

Merging #3733 (12582e4) into main (cb0947d) will increase coverage by 0.02%.
The diff coverage is 74.50%.

Impacted file tree graph

Impacted Files Coverage Δ
bin/reth/src/debug_cmd/merkle.rs 1.14% <0.00%> (ø)
bin/reth/src/node/mod.rs 9.94% <0.00%> (-0.12%) ⬇️
bin/reth/src/stage/dump/merkle.rs 0.00% <0.00%> (ø)
crates/config/src/config.rs 64.65% <0.00%> (ø)
crates/primitives/src/lib.rs 100.00% <ø> (ø)
crates/primitives/src/prune/checkpoint.rs 100.00% <ø> (ø)
crates/prune/src/error.rs 0.00% <ø> (ø)
crates/storage/db/src/tables/mod.rs 53.65% <ø> (ø)
...tes/storage/provider/src/providers/database/mod.rs 30.88% <0.00%> (-0.35%) ⬇️
crates/storage/provider/src/providers/mod.rs 15.68% <0.00%> (-0.11%) ⬇️
... and 10 more

... and 12 files with indirect coverage changes

Flag Coverage Δ
integration-tests 15.57% <0.00%> (-0.05%) ⬇️
unit-tests 64.44% <74.50%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
reth binary 26.44% <0.00%> (-0.04%) ⬇️
blockchain tree 83.02% <ø> (ø)
pipeline 89.62% <100.00%> (+0.02%) ⬆️
storage (db) 74.24% <61.11%> (-0.03%) ⬇️
trie 94.65% <ø> (ø)
txpool 46.00% <ø> (-0.57%) ⬇️
networking 77.72% <ø> (+0.04%) ⬆️
rpc 58.22% <ø> (-0.01%) ⬇️
consensus 65.32% <100.00%> (+0.07%) ⬆️
revm 33.68% <ø> (ø)
payload builder 6.61% <ø> (ø)
primitives 88.09% <36.36%> (+0.05%) ⬆️

@shekhirin shekhirin marked this pull request as ready for review July 17, 2023 06:51
@shekhirin shekhirin requested a review from gakonst as a code owner July 17, 2023 06:51
Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks very clean and was easy to follow.

tests also look great

storage_history
/// Returns block up to which pruning needs to be done, inclusive, according to the provided
/// prune mode and tip.
pub fn prune_to_block(&self, mode: &PruneMode, tip: BlockNumber) -> BlockNumber {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe prune_target_block?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

follow-up PR will change the name

@joshieDo joshieDo enabled auto-merge July 19, 2023 16:24
@shekhirin shekhirin disabled auto-merge July 23, 2023 22:01
@shekhirin shekhirin enabled auto-merge July 24, 2023 15:26
@shekhirin shekhirin added this pull request to the merge queue Jul 24, 2023
@shekhirin shekhirin removed this pull request from the merge queue due to a manual request Jul 24, 2023
@shekhirin shekhirin enabled auto-merge July 24, 2023 16:15
@shekhirin shekhirin added this pull request to the merge queue Jul 24, 2023
Merged via the queue into main with commit 1ca7f3a Jul 24, 2023
29 checks passed
@shekhirin shekhirin deleted the alexey/pruner-receipts branch July 24, 2023 16:50
merklefruit pushed a commit to anton-rs/op-reth that referenced this pull request Jul 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-db Related to the database A-pruning Related to pruning or full node C-enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Respect Receipts prune part in the pruner
3 participants