Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shrink Trienodes #7915

Merged
merged 13 commits into from
Dec 18, 2024
Merged

Shrink Trienodes #7915

merged 13 commits into from
Dec 18, 2024

Conversation

benaadams
Copy link
Member

@benaadams benaadams commented Dec 15, 2024

Changes

  • TrieNode _storageRoot only needs to exist on Leaf saving 8 bytes
  • LastSeen can be combined with the other flags saving 8 bytes
  • _data can be converted to fields/inline array saving 24 bytes; and removing one object per TrieNode (are a lot of object in trie and GC time is a function of object count)

Before

Type layout for 'TrieNode'
Size: 48 bytes. Paddings: 1 bytes (%2 of empty space)
|==============================================================|
| Object Header (8 bytes)                                      |
|--------------------------------------------------------------|
| Method Table Ptr (8 bytes)                                   |
|==============================================================|
|   0-7: TrieNode _storageRoot (8 bytes)                       |
|--------------------------------------------------------------|
|  8-15: RlpFactory _rlp (8 bytes)                             |
|--------------------------------------------------------------|
| 16-23: Object[] _data (8 bytes)                              |
|--------------------------------------------------------------|
| 24-31: Hash256 <Keccak>k__BackingField (8 bytes)             |
|--------------------------------------------------------------|
| 32-39: Int64 <LastSeen>k__BackingField (8 bytes)             |
|--------------------------------------------------------------|
| 40-43: Int32 _isDirty (4 bytes)                              |
|--------------------------------------------------------------|
|    44: Boolean <IsBoundaryProofNode>k__BackingField (1 byte) |
|--------------------------------------------------------------|
|    45: Boolean <IsPersisted>k__BackingField (1 byte)         |
|--------------------------------------------------------------|
|    46: NodeType <NodeType>k__BackingField (1 byte)           |
| |==============================|                             |
| |     0: Byte value__ (1 byte) |                             |
| |==============================|                             |
|--------------------------------------------------------------|
|    47: padding (1 byte)                                      |
|==============================================================|

After

Type layout for 'TrieNode'
Size: 32 bytes. Paddings: 0 bytes (%0 of empty space)
|==================================================|
| Object Header (8 bytes)                          |
|--------------------------------------------------|
| Method Table Ptr (8 bytes)                       |
|==================================================|
|   0-7: RlpFactory _rlp (8 bytes)                 |
|--------------------------------------------------|
|  8-15: INodeData _nodeData (8 bytes)             |
|--------------------------------------------------|
| 16-23: Hash256 <Keccak>k__BackingField (8 bytes) |
|--------------------------------------------------|
| 24-31: Int64 _blockAndFlags (8 bytes)            |
|==================================================|


Type layout for 'BranchData'
Size: 128 bytes. Paddings: 120 bytes (%93 of empty space)
|==========================================|
| Object Header (8 bytes)                  |
|------------------------------------------|
| Method Table Ptr (8 bytes)               |
|==========================================|
| 0-127: BranchArray _branches (128 bytes) |
| |===================================|    |
| |   0-7: Object _element0 (8 bytes) |    |
| |-----------------------------------|    |
| | 8-127: padding (120 bytes)        |    |
| |===================================|    |
|==========================================|


Type layout for 'ExtensionData'
Size: 16 bytes. Paddings: 0 bytes (%0 of empty space)
|================================|
| Object Header (8 bytes)        |
|--------------------------------|
| Method Table Ptr (8 bytes)     |
|================================|
|   0-7: Byte[] _key (8 bytes)   |
|--------------------------------|
|  8-15: Object _value (8 bytes) |
|================================|


Type layout for 'LeafData'
Size: 32 bytes. Paddings: 4 bytes (%12 of empty space)
|========================================================|
| Object Header (8 bytes)                                |
|--------------------------------------------------------|
| Method Table Ptr (8 bytes)                             |
|========================================================|
|   0-7: Byte[] <Key>k__BackingField (8 bytes)           |
|--------------------------------------------------------|
|  8-15: TrieNode <StorageRoot>k__BackingField (8 bytes) |
|--------------------------------------------------------|
| 16-31: CappedArray`1 _value (16 bytes)                 |
| |================================|                     |
| |   0-7: Byte[] _array (8 bytes) |                     |
| |--------------------------------|                     |
| |  8-11: Int32 _length (4 bytes) |                     |
| |--------------------------------|                     |
| | 12-15: padding (4 bytes)       |                     |
| |================================|                     |
|========================================================|

Types of changes

What types of changes does your code introduce?

  • Optimization
  • Refactoring

Copy link
Member

@LukaszRozmej LukaszRozmej left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we change Keccak from Hash256? to ValueHash256??


private readonly CappedArray<byte> _value;

public byte[] Key { get; set; }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inline array?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is a bunch of follow up work that can be done; but probably good as now

@benaadams
Copy link
Member Author

Should we change Keccak from Hash256? to ValueHash256??

Problematic since the entries in the level above are TrieNode or Hash256; so either the inline array in branches goes from 128 bytes to 512 bytes, or we box the ValueHash256 so it can go in an object reference, but then might as well have just kept it as a Hash256?

@benaadams benaadams marked this pull request as ready for review December 17, 2024 06:22
Copy link
Contributor

@asdacap asdacap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@benaadams benaadams merged commit 7252943 into master Dec 18, 2024
79 checks passed
@benaadams benaadams deleted the trienode branch December 18, 2024 09:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants