feat: allow syncing op-mainnet with only state and without importing blocks/receipts
#10850
Conversation
joshieDo
added
C-enhancement
joshieDo
changed the title
op-mainnet with only state and without importing blocks/receipts
|
will be great for testing |
Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
There was a problem hiding this comment.
We need to edit rpc for this too, any queries made for blocks in the ovm range, need to return an error. Alternatively, we need to add docs explaining that rpc responses may return undefined behaviour. Needs to include an exhaustive list of the rpc endpoints that are effected, and how, so that no one bases their scraping on this. Maybe logging a warning "don't use rpc for scraping" if op mainnet is started with dummy blocks, is sufficient.
|
i feel like that's overblown. an empty response should suffice imo. this is happening for txs/receips, but need to add that for the block then, so it doesnt return an invalid block scrapers that own the node are aware of the lack of ovm historical data |
joshieDo
requested review from
rakita,
Rjected,
onbjerg,
mattsse,
fgimenez and
gakonst
as code owners
| assert_eq!( | ||
| sf_provider.latest_writer(segment)?.user_header().block_end(), | ||
| Some(BEDROCK_HEADER.number - 1) | ||
| ); |
There was a problem hiding this comment.
this should include a message because will panic
| /// * Headers: It will push an empty block. | ||
| /// * Transactions: It will not push any tx, only increments the end block range. | ||
| /// * Receipts: It will not push any receipt, only increments the end block range. | ||
| fn append_dummy_chain(sf_provider: &StaticFileProvider) -> Result<(), eyre::Error> { |
There was a problem hiding this comment.
can we make this accept the target as argument, could be useful in the future
| let receipts_writer = sf_provider.latest_writer(StaticFileSegment::Receipts)?; | ||
| let (tx, rx) = std::sync::mpsc::channel(); | ||
|
|
||
| rayon::scope(|s| { |
There was a problem hiding this comment.
this should be fine because this is the only thing we're running, even if this does a lot of IO.
still would prefer regular thread for this.
Running optimism mainnet is an hard endeavour since on top of state (14G uncompressed / 4GB compressed), one needs to export (from op-geth) and import blocks (103G / 43G) and receipts (173G / 71G). The process of acquiring this historical data is also quite troublesome.
On top of that, these blocks cannot even be executed, so they are there only for historical/RPC purposes.
Proposing adding
op-reth init-state --chain optimism --without-ovm world_trie.jsonlwhich just imports state and readies the chain for syncing.It will generate a dummy chain (with no transactions) up the last OVM block header. It then, appends the real Bedrock block. Takes around 90seconds and 14G which come from inserting >100_000_000 empty headers.