Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pageserver: simple fix for vectored read image layer skip #9026

Merged
merged 4 commits into from
Sep 19, 2024

Conversation

VladLazar
Copy link
Contributor

Problem

Different keyspaces may require different floor LSNs in vectored
delta layer visits. This patch adds support for such cases.

Summary of changes

Different keyspaces wishing to read the same layer might
require different stop lsns (or lsn floor). The start LSN
of the read (or the lsn ceil) will always be the same.

With this observation, we fix skipping of image layers by
indexing the fringe by layer id plus lsn floor.

This is very simple, but means that we can visit delta layers twice
in certain cases. Still, I think it's very unlikely for any extra merging
to have taken place in this case, so perhaps it makes sense to go
with the simpler patch.

Fixes #9012
Alternative to #9025

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

Different keyspaces wishing to read the same layer might
require different stop lsns (or lsn floor). The start LSN
of the read (or the lsn ceil) will always be the same.

With this observation, we fix skipping of image layers by
indexing the fringe by layer id plus lsn floor.
Copy link

github-actions bot commented Sep 17, 2024

5072 tests run: 4908 passed, 0 failed, 164 skipped (full report)


Flaky tests (11)

Postgres 17

Postgres 16

Postgres 15

Postgres 14

Code coverage* (full report)

  • functions: 31.8% (7422 of 23309 functions)
  • lines: 49.8% (59697 of 119797 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
60451ca at 2024-09-19T12:58:58.061Z :recycle:

@VladLazar VladLazar marked this pull request as ready for review September 19, 2024 09:46
@VladLazar VladLazar requested a review from a team as a code owner September 19, 2024 09:46
@VladLazar VladLazar requested a review from arssher September 19, 2024 09:46
@VladLazar VladLazar removed the request for review from arssher September 19, 2024 09:47
@VladLazar VladLazar merged commit 21eeafa into main Sep 19, 2024
80 checks passed
@VladLazar VladLazar deleted the vlad/fix-vec-read-img-layer-skip-take-1 branch September 19, 2024 13:51
skyzh added a commit that referenced this pull request Nov 12, 2024
I had an impression that gc-compaction didn't test the case where the
first record of the key history is will_init because of there are some
code path that will panic in this case. Luckily it got fixed in
#9026 so we can now implement
such tests.

Part of #9114

## Summary of changes

* Randomly changed some images into will_init neon wal record
* Split `test_simple_bottom_most_compaction_deltas` into two test cases,
one of them has the bottom layer as delta layer with will_init flags,
while the other is the original one with image layers.

---------

Signed-off-by: Alex Chi Z <chi@neon.tech>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Vectored get image layer skipping
2 participants