Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write VLong in opposite order for better outputs sharing in the FST #12620

Closed
gf2121 opened this issue Oct 4, 2023 · 1 comment · Fixed by #12631
Closed

Write VLong in opposite order for better outputs sharing in the FST #12620

gf2121 opened this issue Oct 4, 2023 · 1 comment · Fixed by #12631

Comments

@gf2121
Copy link
Contributor

gf2121 commented Oct 4, 2023

Description

We also should really explore the TODO above to write vLong in opposite byte order -- this might save quite a bit of storage in the FST since outputs would share more prefixes. Again, separate issue 😀

From #12604 (comment)

Related to this TODO

@mikemccand
Copy link
Member

This might be needle moving on the size of the FSTs created by block tree for the terms index, since it encodes long as vLong in its output. We should only try this "reverse vLong" in that context (private to FST encoding) ... e.g. it would not likely help for FST Outputs<Long> since the encoded output is a true long value not a byte[] leading with an encoded vLong and most of the written long values in that case will already be smallish.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants