Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: refactor generation of snapshots from the cli #5464

Merged
merged 14 commits into from
Nov 24, 2023
Merged

Conversation

joshieDo
Copy link
Collaborator

  • Refactors the generation of snapshots from the cli (dedup some code)
  • Add parallelism if desired
  • Add some stats that get printed at the end of the generation

@joshieDo joshieDo added A-cli Related to the reth CLI A-static-files Related to static files labels Nov 16, 2023
Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

smol nit,

pending @shekhirin

bin/reth/src/db/snapshots/mod.rs Outdated Show resolved Hide resolved
Copy link
Collaborator

@shekhirin shekhirin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, only nits

/// statistics about various aspects of each snapshot, such as filters size,
/// offset index size, offset list size, and loading time.
fn stats(&self, snapshots: Vec<impl AsRef<Path>>) -> eyre::Result<()> {
let mb = 1024.0 * 1024.0;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: const

Comment on lines +243 to +244
avg_duration.as_millis() as f64,
avg_duration.as_micros() as f64
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we just use the Debug impl for Duration here?

crates/storage/nippy-jar/src/filter/mod.rs Outdated Show resolved Hide resolved
@joshieDo joshieDo added this pull request to the merge queue Nov 24, 2023
Merged via the queue into main with commit af88851 Nov 24, 2023
28 checks passed
@joshieDo joshieDo deleted the joshie/gen-snapshots branch November 24, 2023 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-cli Related to the reth CLI A-static-files Related to static files
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants