Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce index size by using the key's array index instead of the key name #405

Closed
krisk opened this issue Apr 25, 2020 · 0 comments
Closed
Assignees
Labels
Milestone

Comments

@krisk
Copy link
Owner

krisk commented Apr 25, 2020

Description

We can reduce the index size by referencing the array index of the key, as opposed to the key text itself when constructing the $ entry of each record.

Describe the solution you'd like

Currently, the index stores keys of every entry:

[
  {
    i: 0,
    '$': {
      'title': { v: "Old Man's War", n: 0.5773502691896258 },
      'author.firstName': { v: 'John', n: 1 },
      'author.tags.name': [
        { v: 'Tag 1', i: 2, n: 0.7071067811865475 },
        { v: 'Tag 2', i: 1, n: 0.7071067811865475 },
        { v: 'Tag 3', i: 0, n: 0.7071067811865475 }
      ]
    }
  }
  /* rest of items */
]

Above, "title", "author.firstName", and "'author.tags.name" are going to be repeated for n number of entries. However, we can minimize the index size by simply using the key index:

{
  keys: [ 'title', 'author.firstName', 'author.tags.name' ],
  list: [
    {
      i: 0,
      '$': {
        '0': { v: "Old Man's War", n: 0.5773502691896258 },
        '1': { v: 'John', n: 1 },
        '2': [
          { v: 'Tag 1', i: 2, n: 0.7071067811865475 },
          { v: 'Tag 2', i: 1, n: 0.7071067811865475 },
          { v: 'Tag 3', i: 0, n: 0.7071067811865475 }
        ]
      }
    }
    /* rest of items */
  ]
}

With the above, we saved a considerable amount of space.

@krisk krisk added the feature label Apr 25, 2020
@krisk krisk added this to the v5.3.0 milestone Apr 25, 2020
@krisk krisk self-assigned this Apr 25, 2020
krisk added a commit that referenced this issue Apr 25, 2020
@krisk krisk changed the title Reduce index size Reduce index size by using the key's array index instead of the key name Apr 25, 2020
@krisk krisk closed this as completed Apr 28, 2020
@krisk krisk modified the milestones: v5.3.0, v6.0.0 May 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant