Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pageserver: fix vectored read image layer skip #9015

Closed
wants to merge 3 commits into from

Conversation

VladLazar
Copy link
Contributor

Problem

When vectored read reaches a delta layer with an image inside of it,
it might have to read different key ranges at different LSN ranges.

A              B
+--------------+ l1
|              |      ^ LSN
|    ===== l2  |      |
|    C E D     |      |
+--------------+ l3   +-----> Key

Let's assume that we are reading the [A, E) key range and the current search LSN (or cont LSN) for this range is greater than l1. We will perform a ranged search to see which layers to visit next. It will yield two results (both pointing to the same layer):

  • delta layer [A, B): key_range=[A, C) lsn_floor=l3
  • delta layer [A, B): key_range=[C, E) lsn_floor=l2

Now as Christian pointed out above, the fringe makes the incorrect assumption that all reads within a layer will be at the same LSN. Note how we don't use the lsn_range argument if the layer was already in the fringe. Hence, we use the first LSN range for both key ranges. This skips the image layer for the [C, E) range.

Summary of changes

  • Add a unit test which reproduces the issue
  • Add inefficient fix - TODO: this should be replaced with a solution that only visits the layer once

Fixes #9012

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

Copy link

github-actions bot commented Sep 16, 2024

4968 tests run: 4804 passed, 0 failed, 164 skipped (full report)


Flaky tests (3)

Postgres 17

Postgres 16

Postgres 14

  • test_ondemand_wal_download_in_replication_slot_funcs: release-arm64

Code coverage* (full report)

  • functions: 31.8% (7419 of 23299 functions)
  • lines: 49.9% (59681 of 119500 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
e02334c at 2024-09-16T19:39:55.740Z :recycle:

@VladLazar
Copy link
Contributor Author

Closing in favour of #9026

@VladLazar VladLazar closed this Sep 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Vectored get image layer skipping
2 participants