Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/FlushWAL on flush. #7642

Merged
merged 10 commits into from
Oct 24, 2024
Merged

Feature/FlushWAL on flush. #7642

merged 10 commits into from
Oct 24, 2024

Conversation

asdacap
Copy link
Contributor

@asdacap asdacap commented Oct 22, 2024

  • Attempts to make writes more reliable.
  • Calls FlushWAl on Flush(). This ensure that WAL is written to storage, which normally does not happen as it only is written to OS cache until the OS decided to do so.
    • This is the default rockdb behaviour and means that a crash of the program is recoverable but not a crash of the whole system.
  • This change ensure that the writes is recoverable in the event of a system crash. Or something unexpected happens in the OS.
  • Also add flush (but only WAL as memtable flush can run in background safely) to persist commit set in triestore. This ensure no funny business going on with missing writes. This adds about 5-10 percent to the pruning time unfortunately.
  • Also change write buffer recommendation to increase write buffer size instead of write buffer count. This tend to improve pruning time stability a bit and probably improve the time it take to get back up to speed after memory pruning at expense of memory. But I guess if you increase pruning cache, you have memory to spare anyway.
  • Also add logs on which state root is persisted on persist commit set.
  • Also when commit set of unexpected number happened, write info log. This generally means a reorg, or something strange is happening if its not a reorg.

Types of changes

What types of changes does your code introduce?

  • Optimization

Testing

Requires testing

  • Yes
  • No

If yes, did you write tests?

  • Yes
  • No

Notes on testing

  • Mainnet catch up.
  • Mainnet sync
  • Mainnet archive

@asdacap asdacap merged commit a262636 into master Oct 24, 2024
75 checks passed
@asdacap asdacap deleted the feature/more-log branch October 24, 2024 02:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants