Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Epic: pageserver support for sharding phase 1 #5507

Closed
9 of 15 tasks
jcsp opened this issue Oct 9, 2023 · 1 comment
Closed
9 of 15 tasks

Epic: pageserver support for sharding phase 1 #5507

jcsp opened this issue Oct 9, 2023 · 1 comment
Assignees
@jcsp
Labels
c/storage/pageserver Component: storage: pageserver t/Epic Issue type: Epic t/feature Issue type: feature, for new features or requests

Comments

@jcsp
Copy link
Collaborator

jcsp commented Oct 9, 2023 •
edited
Loading

This epic describes the pageserver changes required to enable sharding phase 1.

Phase 1 does not include splitting: tenants will be created with a set number of shards.

Tasks

Part 1: establish APIs

Milestone: control plane code that expects sharding-aware APIs can talk to the pageserver, and compute will work when using a ShardMap to talk to multiple pageservers.

Tasks
Preview Give feedback

Part 2: make sharding work

Milestone: the shards within a tenant will store different data: we can demonstrate hosting a 1TiB tenant where each pageserver only holds 1/nth data.

Tasks
Preview Give feedback
  1. pageserver: shard-aware metadata #5924
    c/storage/pageserver t/feature
    jcsp
  2. pageservers: shard-aware pageservice #6026
    c/storage/pageserver t/feature
    jcsp
  3. pageserver: shard-aware WAL ingest #6025
    c/storage/pageserver t/feature
    jcsp

Part 3: Loose ends

Outstanding loose ends items moved into #6288

Tasks
Preview Give feedback
  2. pageserver: update metrics for shard IDs #5953
    c/storage/pageserver t/feature
  6. scrubber: update for sharding #5929
    c/storage/pageserver t/feature
    jcsp
  8. pageserver: cleaner error handling if client sends sharded request to wrong pageserver #6038
    a/tech_debt c/storage/pageserver
    jcsp
  9. pageserver: clean up sharding helper names #6072
    a/tech_debt c/storage/pageserver
    jcsp
@jcsp jcsp added t/feature Issue type: feature, for new features or requests c/storage/pageserver Component: storage: pageserver t/Epic Issue type: Epic labels Oct 9, 2023
@jcsp jcsp mentioned this issue Oct 9, 2023
Closed
3 tasks
@problame problame mentioned this issue Nov 22, 2023
Closed
This was referenced Nov 28, 2023
Merged
Merged
jcsp added a commit that referenced this issue Dec 5, 2023
@jcsp
pageserver: add Key->Shard mapping logic & use it in page service (#5980
61fe9d3 
)

## Problem

When a pageserver receives a page service request identified by
TenantId, it must decide which `Tenant` object to route it to.

As in earlier PRs, this stuff is all a no-op for tenants with a single
shard: calls to `is_key_local` always return true without doing any
hashing on a single-shard ShardIdentity.

Closes: #6026

## Summary of changes

- Carry immutable `ShardIdentity` objects in Tenant and Timeline. These
provide the information that Tenants/Timelines need to figure out which
shard is responsible for which Key.
- Augment `get_active_tenant_with_timeout` to take a `ShardSelector`
specifying how the shard should be resolved for this tenant. This mode
depends on the kind of request (e.g. basebackups always go to shard
zero).
- In `handle_get_page_at_lsn_request`, handle the case where the
Timeline we looked up at connection time is not the correct shard for
the page being requested. This can happen whenever one node holds
multiple shards for the same tenant. This is currently written as a
"slow path" with the optimistic expectation that usually we'll run with
one shard per pageserver, and the Timeline resolved at connection time
will be the one serving page requests. There is scope for optimization
here later, to avoid doing the full shard lookup for each page.
- Omit consumption metrics from nonzero shards: only the 0th shard is
responsible for tracing accurate relation sizes.

Note to reviewers:
- Testing of these changes is happening separately on the
`jcsp/sharding-pt1` branch, where we have hacked neon_local etc needed
to run a test_pg_regress.
- The main caveat to this implementation is that page service
connections still look up one Timeline when the connection is opened,
before they know which pages are going to be read. If there is one shard
per pageserver then this will always also be the Timeline that serves
page requests. However, if multiple shards are on one pageserver then
get page requests will incur the cost of looking up the correct Timeline
on each getpage request. We may look to improve this in future with a
"sticky" timeline per connection handler so that subsequent requests for
the same Timeline don't have to look up again, and/or by having postgres
pass a shard hint when connecting. This is tracked in the "Loose ends"
section of #5507
@jcsp
Copy link
Collaborator Author

jcsp commented Jan 8, 2024

Loose ends moved into #6288

@jcsp jcsp closed this as completed Jan 8, 2024
@jcsp jcsp mentioned this issue Mar 11, 2024
Closed
@problame problame mentioned this issue Apr 30, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jcsp jcsp

Labels
c/storage/pageserver Component: storage: pageserver t/Epic Issue type: Epic t/feature Issue type: feature, for new features or requests
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jcsp