Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(tree): use in-memory data first to query total difficulty #11382

Merged
merged 5 commits into from
Oct 1, 2024

Conversation

fgimenez
Copy link
Member

@fgimenez fgimenez commented Oct 1, 2024

Flips the order of data retrieval in header_td_by_number, first in-memory, then db.

Additionally, to prevent races with the persistence task in cases of reorgs, when the required block is found in memory it uses the last finalized block to get the total difficulty, if none is found we will use the last block number found in memory (in this case we are still exposed to reorgs)

ref #10941

@fgimenez fgimenez added C-bug An unexpected or incorrect behavior A-blockchain-tree Related to sidechains, reorgs and pending blocks C-hivetest Used for labelling automated issues relating to hive test failures labels Oct 1, 2024
.get_finalized_num_hash()
.unwrap_or_else(|| BlockNumHash::new(0, B256::default()));
self.database.header_td_by_number(last_finalized_num_hash.number)
} else if let Some(td) = self.database.header_td_by_number(number)? {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think, if its not in memory we can just return whatever self.database method returns

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we won't ever hit that if we hardcoded the post merge ttd

but we can't use BlockNumHash::new(0, B256::default()) because here because this would return the genesis ttd.
this would only be reachable in custom setups, but the unwrap_or_else should use the last_block_number() instead, even if there's a race condition, but this only exists until there is a finalized block, so imo this is fine

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think, if its not in memory we can just return whatever self.database method returns

good point, done here 36b11f7

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but we can't use BlockNumHash::new(0, B256::default()) because here because this would return the genesis ttd.
this would only be reachable in custom setups, but the unwrap_or_else should use the last_block_number() instead, even if there's a race condition, but this only exists until there is a finalized block, so imo this is fine

yep, if get_finalized_num_hash returns None and we try to get last_block_number the test is flaky again, should we do that instead?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the flaky test is fine, we know why it's flaky and why this isn't an issue.
also total difficulty will be removed from rpc so this case will eventually go away

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense done here 418c48e ptal

not adding the hive test to expected failures bc it will be randomly passing/failing

.get_finalized_num_hash()
.unwrap_or_else(|| BlockNumHash::new(0, B256::default()));
self.database.header_td_by_number(last_finalized_num_hash.number)
} else if let Some(td) = self.database.header_td_by_number(number)? {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we won't ever hit that if we hardcoded the post merge ttd

but we can't use BlockNumHash::new(0, B256::default()) because here because this would return the genesis ttd.
this would only be reachable in custom setups, but the unwrap_or_else should use the last_block_number() instead, even if there's a race condition, but this only exists until there is a finalized block, so imo this is fine

@fgimenez fgimenez removed the C-hivetest Used for labelling automated issues relating to hive test failures label Oct 1, 2024
Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

// Otherwise, return what we have on disk for the input block
number
};
self.database.header_td_by_number(number)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah only updating the number is even better

@mattsse mattsse enabled auto-merge October 1, 2024 18:13
@mattsse mattsse added this pull request to the merge queue Oct 1, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 1, 2024
@mattsse mattsse added this pull request to the merge queue Oct 1, 2024
Merged via the queue into main with commit b9341a7 Oct 1, 2024
35 checks passed
@mattsse mattsse deleted the fgimenez/fix-hive-reorg-test branch October 1, 2024 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-blockchain-tree Related to sidechains, reorgs and pending blocks C-bug An unexpected or incorrect behavior
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants