Set ReadAdvice#NORMAL on files that have a forward-only access pattern.
#13450
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ess pattern. This introduces a new API that allows directories to optimize access to `IndexInput`s that have a forward-only access pattern by reading ahead of the current position. It would be applicable to: - Postings lists, - Doc values, - Norms, - Points. Relates apache#13179
9 tasks
rmuir
reviewed
Jun 4, 2024
| } | ||
| // TODO: Is there a hint we can give to the OS to let it optimize for our forward-only access | ||
| // pattern in the given range? | ||
| } |
There was a problem hiding this comment.
or even better, MADV_NORMAL for the range, which is less aggressive.
There was a problem hiding this comment.
I thought you had concerns about MADV_NORMAL but I like it better too, I updated the PR to use ReadAdvice.NORMAL instead.
This reverts commit 8375caf.
…only access pattern." This reverts commit 8a5cb8f.
…ern. This applies to files where performing readahead could help: - Doc values data (`.dvd`) - Norms data (`.nvd`) - Docs and freqs in postings lists (`.doc`) - Points data (`.kdd`) Other files (KNN vectors, stored fields, term vectors) keep using a `RANDOM` advice.
rmuir
approved these changes
Jun 5, 2024
jpountz
changed the title
IndexInput about slices that have a forward-only access pattern.ReadAdvice#NORMAL on files that have a forward-only access pattern.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This applies to files where performing readahead could help:
.dvd).nvd).doc).kdd)Other files (KNN vectors, stored fields, term vectors) keep using a
RANDOMadvice.