Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Verkle Implementation: Build out Trie Processing #3430

Merged
merged 58 commits into from
Jun 21, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
0b03444
add verkle crypto to trie
acolytec3 May 10, 2024
f468ce2
Merge remote-tracking branch 'origin/master' into verkle-trie-build-out
acolytec3 May 13, 2024
c1ce5eb
More spam
acolytec3 May 13, 2024
a317698
Merge remote-tracking branch 'origin/master' into verkle-trie-build-out
acolytec3 May 15, 2024
1670155
Update leafNode.create [no ci]
acolytec3 May 15, 2024
55d644f
Merge remote-tracking branch 'origin/master' into verkle-trie-build-o…
acolytec3 May 16, 2024
974d1ae
small removal [no ci]
acolytec3 May 16, 2024
570f8c4
Merge remote-tracking branch 'origin/master' into verkle-trie-build-o…
acolytec3 May 17, 2024
430d801
Update trie.get to check existence first
acolytec3 May 20, 2024
fa34dde
Update `put` when inserting a new leaf node [no ci]
acolytec3 May 20, 2024
b2806cf
more halfway stuff [no ci]
acolytec3 May 21, 2024
f9e019e
Remove unnecessary commit [no ci]
acolytec3 May 31, 2024
74676b0
update verkle crypto dep [no ci]
acolytec3 Jun 3, 2024
a267be1
Add helper to create leaf values as 16 bytes
acolytec3 Jun 3, 2024
e3ab994
Changes to support c1 and c2 [no ci]
acolytec3 Jun 3, 2024
6e7e91e
update new leaf node commitment correctly [no ci]
acolytec3 Jun 4, 2024
4a716a7
Merge remote-tracking branch 'origin/master' into verkle-trie-build-o…
acolytec3 Jun 4, 2024
a2ed90e
Changes needed to make `put` work [no ci]
acolytec3 Jun 6, 2024
1902ebc
Begin fixing internal node implementation [no ci]
acolytec3 Jun 6, 2024
541240c
move verkleCrypto to verkleNode constructor opts
acolytec3 Jun 7, 2024
76da96c
address feedback
acolytec3 Jun 7, 2024
52d50a8
WIP [no ci]
acolytec3 Jun 7, 2024
86dd9e9
Finish naive findpath implementation
acolytec3 Jun 10, 2024
189b112
Update internal node layout
acolytec3 Jun 11, 2024
60219ae
WIP [no ci]
acolytec3 Jun 11, 2024
bbd0e96
Partial implementation of put [no ci]
acolytec3 Jun 11, 2024
2301d2c
update verkle crypto [no ci]
acolytec3 Jun 12, 2024
dcdd778
Update this.root [no ci]
acolytec3 Jun 12, 2024
cf33476
Clean up db opt typing [no ci]
acolytec3 Jun 12, 2024
b9285e2
API clean/comments [no ci]
acolytec3 Jun 12, 2024
1dcc64d
fix logic bug for nonexistent path [no ci]
acolytec3 Jun 12, 2024
f42ac23
Describe logic for updating tree along path [no ci]
acolytec3 Jun 12, 2024
e4e257a
Update `put` to use `saveStack` [no ci]
acolytec3 Jun 13, 2024
1b1706d
WIP [no ci]
acolytec3 Jun 13, 2024
96f99aa
revise leafNode.create API [no ci]
acolytec3 Jun 13, 2024
e4bcb66
more updates to put/get [no ci]
acolytec3 Jun 13, 2024
27144c0
More wip [no ci]
acolytec3 Jun 13, 2024
7f4ead6
Fix bug in internalNode deserialization [no ci]
acolytec3 Jun 14, 2024
abc442d
Add more comments [no ci]
acolytec3 Jun 14, 2024
eeb2ec8
remove duplicative function [no ci]
acolytec3 Jun 14, 2024
e7e81bf
more wip [no ci]
acolytec3 Jun 14, 2024
991f4d2
Add code to produce a 2 layer tree [no ci]
acolytec3 Jun 15, 2024
58e5298
wip [no ci]
acolytec3 Jun 17, 2024
06350d0
Add some initial debug logs [no ci]
acolytec3 Jun 17, 2024
df22e18
More progress [no ci]
acolytec3 Jun 17, 2024
69267f7
more half-working fixes [no ci]
acolytec3 Jun 18, 2024
911c5b1
Merge remote-tracking branch 'origin/master' into verkle-trie-build-out
acolytec3 Jun 19, 2024
a38cc80
Fix typing issues and remove walk controller
acolytec3 Jun 19, 2024
ce343ae
Remove walk controller export [no ci]
acolytec3 Jun 19, 2024
9c7b214
Add new test to demonstrate putting values in the trie [no ci]
acolytec3 Jun 19, 2024
ae09bdb
Add comment
acolytec3 Jun 19, 2024
05bfebd
Remove obsolete references
acolytec3 Jun 19, 2024
0b4904c
lint
acolytec3 Jun 19, 2024
cd65a9a
Remove references to depth and unused API components
acolytec3 Jun 20, 2024
1ca879e
Merge branch 'master' into verkle-trie-build-out
acolytec3 Jun 20, 2024
3ab0700
Update packages/verkle/src/node/internalNode.ts
acolytec3 Jun 21, 2024
720ab09
Address feedback
acolytec3 Jun 21, 2024
b47fa49
fix tests
acolytec3 Jun 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions packages/verkle/src/node/baseVerkleNode.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,9 @@ import type { VerkleCrypto } from 'verkle-cryptography-wasm'

export abstract class BaseVerkleNode<T extends VerkleNodeType> implements VerkleNodeInterface {
public commitment: Uint8Array
public depth: number
protected verkleCrypto: VerkleCrypto
constructor(options: VerkleNodeOptions[T]) {
this.commitment = options.commitment
this.depth = options.depth
this.verkleCrypto = options.verkleCrypto
}

Expand Down
9 changes: 6 additions & 3 deletions packages/verkle/src/node/internalNode.ts
Original file line number Diff line number Diff line change
Expand Up @@ -58,13 +58,16 @@ export class InternalNode extends BaseVerkleNode<VerkleNodeType.Internal> {
const children = childrenCommitments.map((commitment, idx) => {
return { commitment, path: childrenPaths[idx] }
})
return new InternalNode({ commitment, depth, verkleCrypto, children })
return new InternalNode({ commitment, verkleCrypto, children })
}

static create(depth: number, verkleCrypto: VerkleCrypto): InternalNode {
/**
* Generates a new Internal node with default commitment
*/

acolytec3 marked this conversation as resolved.
Show resolved Hide resolved
static create(verkleCrypto: VerkleCrypto): InternalNode {
const node = new InternalNode({
commitment: verkleCrypto.zeroCommitment,
depth,
verkleCrypto,
})

Expand Down
7 changes: 1 addition & 6 deletions packages/verkle/src/node/leafNode.ts
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,6 @@ export class LeafNode extends BaseVerkleNode<VerkleNodeType.Leaf> {
return new LeafNode({
stem,
values,
depth,
commitment,
c1,
c2,
Expand All @@ -112,7 +111,7 @@ export class LeafNode extends BaseVerkleNode<VerkleNodeType.Leaf> {
const c2 = rawNode[4]
const values = rawNode.slice(5, rawNode.length)

return new LeafNode({ depth, stem, values, c1, c2, commitment, verkleCrypto })
return new LeafNode({ stem, values, c1, c2, commitment, verkleCrypto })
}

// Retrieve the value at the provided index from the values array
Expand Down Expand Up @@ -181,8 +180,4 @@ export class LeafNode extends BaseVerkleNode<VerkleNodeType.Leaf> {
...this.values,
]
}

setDepth(depth: number): void {
this.depth = depth
}
}
8 changes: 1 addition & 7 deletions packages/verkle/src/node/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,24 +19,18 @@ export interface TypedVerkleNode {
export type VerkleNode = TypedVerkleNode[VerkleNodeType]

export interface VerkleNodeInterface {
hash(): any
hash(): Uint8Array
serialize(): Uint8Array
}

interface BaseVerkleNodeOptions {
// Value of the commitment
commitment: Uint8Array
depth: number
verkleCrypto: VerkleCrypto
}

interface VerkleInternalNodeOptions extends BaseVerkleNodeOptions {
// Children nodes of this internal node.
children?: ChildNode[]

// Values of the child commitments before the tree is modified by inserts.
// This is useful because the delta of the child commitments can be used to efficiently update the node's commitment
copyOnWrite?: Record<string, Uint8Array>
}
interface VerkleLeafNodeOptions extends BaseVerkleNodeOptions {
stem: Uint8Array
Expand Down
450 changes: 192 additions & 258 deletions packages/verkle/src/verkleTree.ts

Large diffs are not rendered by default.

12 changes: 8 additions & 4 deletions packages/verkle/test/crypto.spec.ts
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
import { Address, bytesToHex, hexToBytes, randomBytes } from '@ethereumjs/util'
import {
Address,
type VerkleCrypto,
type VerkleExecutionWitness,
bytesToHex,
hexToBytes,
randomBytes,
} from '@ethereumjs/util'
import { loadVerkleCrypto } from 'verkle-cryptography-wasm'
import { assert, beforeAll, describe, it } from 'vitest'

import * as verkleBlockJSON from '../../statemanager/test/testdata/verkleKaustinen6Block72.json'
import { getStem, verifyProof } from '../src/index.js'

import type { VerkleCrypto } from '../src/index.js'
import type { VerkleExecutionWitness } from '@ethereumjs/block'

describe('Verkle cryptographic helpers', () => {
let verkle: VerkleCrypto
beforeAll(async () => {
Expand Down
18 changes: 4 additions & 14 deletions packages/verkle/test/internalNode.spec.ts
Original file line number Diff line number Diff line change
@@ -1,28 +1,21 @@
import { equalsBytes, randomBytes } from '@ethereumjs/util'
import { type VerkleCrypto, equalsBytes, randomBytes } from '@ethereumjs/util'
import { loadVerkleCrypto } from 'verkle-cryptography-wasm'
import { assert, beforeAll, describe, it } from 'vitest'

import { NODE_WIDTH, VerkleNodeType, decodeNode } from '../src/node/index.js'
import { InternalNode } from '../src/node/internalNode.js'

import type { VerkleCrypto } from '../src/types.js'

describe('verkle node - internal', () => {
let verkleCrypto: VerkleCrypto
beforeAll(async () => {
verkleCrypto = await loadVerkleCrypto()
})
it('constructor should create an internal node', async () => {
const commitment = randomBytes(32)
const depth = 2
const node = new InternalNode({ commitment, depth, verkleCrypto })
const node = new InternalNode({ commitment, verkleCrypto })

assert.equal(node.type, VerkleNodeType.Internal, 'type should be set')
assert.ok(
equalsBytes(node.commitment as unknown as Uint8Array, commitment),
'commitment should be set'
)
assert.equal(node.depth, depth, 'depth should be set')
assert.ok(equalsBytes(node.commitment, commitment), 'commitment should be set')

// Children nodes should all default to null.
assert.equal(node.children.length, NODE_WIDTH, 'number of children should equal verkle width')
Expand All @@ -33,16 +26,14 @@ describe('verkle node - internal', () => {
})

it('create method should create an internal node', async () => {
const depth = 3
const node = InternalNode.create(depth, verkleCrypto)
const node = InternalNode.create(verkleCrypto)

assert.equal(node.type, VerkleNodeType.Internal, 'type should be set')
assert.deepEqual(
node.commitment,
verkleCrypto.zeroCommitment,
'commitment should be set to point identity'
)
assert.equal(node.depth, depth, 'depth should be set')

// Children nodes should all default to null.
assert.equal(node.children.length, NODE_WIDTH, 'number of children should equal verkle width')
Expand All @@ -61,7 +52,6 @@ describe('verkle node - internal', () => {
const node = new InternalNode({
children,
verkleCrypto,
depth: 0,
commitment: verkleCrypto.zeroCommitment,
})
const serialized = node.serialize()
Expand Down
8 changes: 1 addition & 7 deletions packages/verkle/test/leafNode.spec.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,10 @@
import { equalsBytes, randomBytes } from '@ethereumjs/util'
import { type VerkleCrypto, equalsBytes, randomBytes } from '@ethereumjs/util'
import { loadVerkleCrypto } from 'verkle-cryptography-wasm'
import { assert, beforeAll, describe, it } from 'vitest'

import { VerkleNodeType } from '../src/node/index.js'
import { LeafNode } from '../src/node/leafNode.js'

import type { VerkleCrypto } from '../src/types.js'

describe('verkle node - leaf', () => {
let verkleCrypto = undefined as never as VerkleCrypto
beforeAll(async () => {
Expand All @@ -18,13 +16,10 @@ describe('verkle node - leaf', () => {
const c2 = randomBytes(64)
const stem = randomBytes(32)
const values = new Array<Uint8Array>(256).fill(randomBytes(32))

const depth = 2
const node = new LeafNode({
c1,
c2,
commitment,
depth,
stem,
values,
verkleCrypto,
Expand All @@ -42,7 +37,6 @@ describe('verkle node - leaf', () => {
values.every((value, index) => equalsBytes(value, node.values[index])),
'values should be set'
)
assert.equal(node.depth, depth, 'depth should be set')
})

it('create method should create an leaf node', async () => {
Expand Down
7 changes: 1 addition & 6 deletions packages/verkle/test/verkle.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -195,13 +195,8 @@ describe('findPath validation', () => {
leafNode2.setValue(hexToBytes(keys[2])[31], hexToBytes(values[2]))
putStack.push([stem2, leafNode2])

// Update the depth of leafNode1
// Note: We wouldn't need to do this if we didn't track depth as a property of each node
leafNode1.depth = 2
putStack.push([stem1, leafNode1])

// Create new internal node
const internalNode1 = InternalNode.create(1, verkleCrypto)
const internalNode1 = InternalNode.create(verkleCrypto)

// Compute the portion of stem1 and stem2 that match
// Note: We subtract 1 since we are using 0-indexed arrays
Expand Down
Loading