-
Notifications
You must be signed in to change notification settings - Fork 458
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix issues with prefetch ring buffer resize #9847
Conversation
5625 tests run: 5389 passed, 0 failed, 236 skipped (full report)Code coverage* (full report)
* collected from Rust tests only The comment gets automatically updated with the latest test results
85bc9b9 at 2024-11-23T09:35:02.523Z :recycle: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This patch looks correct to me. See my comment on the "Change prefetch buffer overflow criteria" part though.
readahead_buffer_resize() is called from the GUC's assign hook. If readahead_buffer_resize() throws an error for any reason (OOM, network error etc.), that could be a problem. The GUC is marked as PGC_USERSET, so its value might need to be changed e.g on transaction abort, or by RESET ALL, and it would be unpleasant if that would fail. But that's not new in this PR.
In general it would feel less error-prone if we'd just throw away all prefetched state, disconnect all connections, and start from scratch.
Looks correct. AFAICS prefetch_set_unused() worked too, though. This is just an optimization to avoid unnecessary work on the old prefetching queue that we are about to throw away anyway, right? Maybe add a comment on that. |
I was not 100% sure about use of |
## Problem See https://neondb.slack.com/archives/C04DGM6SMTM/p1732110190129479 We observe the following error in the logs ``` [XX000] ERROR: [NEON_SMGR] [shard 3] Incorrect prefetch read: status=1 response=0x7fafef335138 my=128 receive=128 ``` most likely caused by changing `neon.readahead_buffer_size` ## Summary of changes 1. Copy shard state 2. Do not use prefetch_set_unused in readahead_buffer_resize 3. Change prefetch buffer overflow criteria --------- Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
Problem
See https://neondb.slack.com/archives/C04DGM6SMTM/p1732110190129479
We observe the following error in the logs
most likely caused by changing
neon.readahead_buffer_size
Summary of changes