Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modify the algorithm of updateFinalizedBlockRoots #13486

Merged
merged 13 commits into from
Mar 21, 2024

Conversation

rkapka
Copy link
Contributor

@rkapka rkapka commented Jan 18, 2024

What type of PR is this?

Other

What does this PR do? Why is it needed?

The motivation for this PR is the following note from IsFinalizedBlock:

// Note: beacon blocks from the latest finalized epoch return true, whether or not they are
// considered canonical in the "head view" of the beacon node.

This means that an orphaned block can be viewed as finalized, as was reported in #13408. To fix this, I redesigned the algorithm so that it traverses parent->child block relationships in the blockParentRootIndicesBucket bucket to find the path between the last finalized checkpoint and the current finalized checkpoint.

TestStore_OrphanedBlockIsNotFinalized is the test covering the main improvement. I completely removed TestStore_IsFinalized_ForkEdgeCase because it tested that a finalized checkpoint is reverted.

The new algorithm shaves off a staggering 10ms from epoch processing time 😆

Which issues(s) does this PR fix?

Fixes #13408

@rkapka rkapka requested a review from a team as a code owner January 18, 2024 17:24
@rkapka rkapka force-pushed the finalized-blocks-audit branch 2 times, most recently from 6c69629 to 12bd6f5 Compare January 18, 2024 18:16
@@ -374,63 +374,63 @@ func TestLoadBlocks_BadStart(t *testing.T) {
// \- B7
func tree1(t *testing.T, beaconDB db.Database, genesisRoot []byte) ([][32]byte, []*ethpb.SignedBeaconBlock, error) {
b0 := util.NewBeaconBlock()
b0.Block.Slot = 0
b0.Block.Slot = 1
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incrementing slot everywhere because genesis block has slot 0

require.NoError(t, err)
require.Equal(t, 10, len(filteredBlocks))
require.Equal(t, 7, len(filteredBlocks))
Copy link
Contributor Author

@rkapka rkapka Jan 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting example to demonstrate the purpose of the new algorithm. The tree constructed in tree1 has orphaned blocks, but they were expected to be finalized.

@rkapka rkapka force-pushed the finalized-blocks-audit branch from 0cc22fe to 38579c4 Compare January 24, 2024 13:56
@prestonvanloon prestonvanloon self-requested a review January 29, 2024 15:08
# Conflicts:
#	beacon-chain/db/kv/finalized_block_roots.go
if bytes.Equal(r, checkpointRoot) {
return true, [][]byte{r}
}
children := lookupValuesForIndices(ctx, map[string][]byte{string(blockParentRootIndicesBucket): r}, tx)[0]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will panic if lookupValuesForIndices returns an empty slice, which is what happens if you send a genesis root here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a condition for it although it is not strictly necessary. Since the passed in map contains one key, the resulting slice will have one item.

@@ -297,3 +232,50 @@ func (s *Store) FinalizedChildBlock(ctx context.Context, blockRoot [32]byte) (in
tracing.AnnotateError(span, err)
return blk, err
}

func pathToFinalizedCheckpoint(ctx context.Context, roots [][]byte, checkpointRoot []byte, tx *bolt.Tx) (bool, [][]byte) {
if len(roots) == 1 && roots[0] == nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if an empty slice is passed for roots we should also exit early

if i == len(finalized)-1 {
// We don't know the finalized child of the new finalized checkpoint.
// It will be filled out in the next function call.
container = &ethpb.FinalizedBlockRootContainer{}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This completely misses adding in ParentRoot for the container. In fact it appears this PR has completely removed it.

container = &ethpb.FinalizedBlockRootContainer{
ParentRoot: finalized[i-1],
}
} else if i == 0 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be better to use a switch statement ? If not handling i in a progressive order is reads better

if i == 0 {
// do something
else if i == len(finalized) -1 {
// do something
} else {
// ...
}

@prestonvanloon prestonvanloon added this pull request to the merge queue Mar 21, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Mar 21, 2024
@prestonvanloon prestonvanloon added this pull request to the merge queue Mar 21, 2024
Merged via the queue into develop with commit 32fb183 Mar 21, 2024
16 of 17 checks passed
@prestonvanloon prestonvanloon deleted the finalized-blocks-audit branch March 21, 2024 21:50
kasey added a commit that referenced this pull request Mar 29, 2024
@kasey kasey mentioned this pull request Mar 29, 2024
github-merge-queue bot pushed a commit that referenced this pull request Mar 29, 2024
* Revert "Modify the algorithm of `updateFinalizedBlockRoots` (#13486)"

This reverts commit 32fb183.

* migration to fix index corruption from pr 13486

* bail as soon as we see 10 epochs without the bug

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Error on finalized flag for /eth/v2/beacon/blocks/{block_id}
3 participants