Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IMPROVED] Add node48 to stree #5585

Merged
merged 1 commit into from
Jun 24, 2024
Merged

[IMPROVED] Add node48 to stree #5585

merged 1 commit into from
Jun 24, 2024

Conversation

neilalexander
Copy link
Member

A node256 is nearly 4KB in memory whereas a node48 is closer to 1KB, so there is a potential for anywhere up to 75% memory savings in some subject spaces.

Implemented as per the ART paper, although with the caveat that our pointers are actually 16 bytes (as interface{}s) rather than 8 bytes as the paper assumes:

Node48: As the number of entries in a node increases, searching the key array becomes expensive. Therefore, nodes with more than 16 pointers do not store the keys explicitly. Instead, a 256-element array is used, which can be indexed with key bytes directly. If a node has between 17 and 48 child pointers, this array stores indexes into a second array which contains up to 48 pointers. This indirection saves space in comparison to 256 pointers of 8 bytes, because the indexes only require 6 bits (we use 1 byte for simplicity).

Signed-off-by: Neil Twigg neil@nats.io

@neilalexander neilalexander requested a review from a team as a code owner June 24, 2024 11:56
A `node256` is nearly 4KB in memory whereas a `node48` is closer to 1KB.

Signed-off-by: Neil Twigg <neil@nats.io>
Copy link
Member

@derekcollison derekcollison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

type node48 struct {
child [48]node
meta
key [256]byte
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting approach, most of the complexity for a node48 comes from the key handling, this avoids that for a slight increase per node size but with still good savings from child array.

@derekcollison derekcollison merged commit 9ac2523 into main Jun 24, 2024
3 checks passed
@derekcollison derekcollison deleted the neil/node48 branch June 24, 2024 16:20
@wallyqs wallyqs changed the title Add node48 to stree [IMPROVED] Add node48 to stree Jun 25, 2024
wallyqs added a commit that referenced this pull request Jun 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants