Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Simplify pageserver initialization & configuration #7736

Closed
8 tasks done
VladLazar opened this issue May 13, 2024 · 5 comments · Fixed by #7766
Closed
8 tasks done

Simplify pageserver initialization & configuration #7736

VladLazar opened this issue May 13, 2024 · 5 comments · Fixed by #7766
Assignees
@problame @VladLazar
Labels
a/deploy Area: related to deployment a/tech_debt Area: related to tech debt

Comments

@VladLazar
Copy link
Contributor

VladLazar commented May 13, 2024 •
edited by problame
Loading

Overall plan: https://www.notion.so/neondatabase/Rollout-Plan-simplified-pageserver-initialization-f935ae02b225444e8a41130b7d34e4ea?pvs=4

Tasks
Preview Give feedback
  2. pageserver: use identity file as node id authority and remove init command and config-override flags #7766
    problame
  8. pageserver: fail if id is present in pageserver.toml #8489
@VladLazar VladLazar added t/bug Issue Type: Bug a/tech_debt Area: related to tech debt and removed t/bug Issue Type: Bug labels May 13, 2024
@VladLazar VladLazar self-assigned this May 13, 2024
@VladLazar VladLazar mentioned this issue May 15, 2024
Merged
5 tasks
@VladLazar
Copy link
Contributor Author

Last week (ending 2024-05-17):

Pageserver and ansible changes are good to go, but cplane usage of --config-override is blocking them.
The cplane changes have to merge first. Once that's done apply the plan from here:
https://www.notion.so/neondatabase/Rollout-Plan-simplified-pageserver-initialization-f935ae02b225444e8a41130b7d34e4ea

Once Christian and/or Vlad are back from holiday:

@VladLazar
Copy link
Contributor Author

2024-06-12:

  • Discussed the removal --config-override from cplane repo with Alexey. The control plane will pick it up.
  • We can resume work on our side once that's happened.

@VladLazar
Copy link
Contributor Author

2024-07-01:

  • Waiting for control plane to remove uses of --config-override flag - should be done early this week
  • Rebase everything and merge

@VladLazar
Copy link
Contributor Author

2024-07-08:

problame added a commit that referenced this issue Jul 23, 2024
@VladLazar @problame
pageserver: use identity file as node id authority and remove init co…
a1272b6 
…mmand and config-override flags (#7766)

Ansible will soon write the node id to `identity.toml` in the work dir
for new pageservers. On the pageserver side, we read the node id from
the identity file if it is present and use that as the source of truth.
If the identity file is missing, cannot be read, or does not
deserialise, start-up is aborted.
 
This PR also removes the `--init` mode and the `--config-override` flag
from the `pageserver` binary.
The neon_local is already not using these flags anymore.

Ansible still uses them until the linked change is merged & deployed,
so, this PR has to land simultaneously or after the Ansible change due
to that.

Related Ansible change: neondatabase/infra#1322
Cplane change to remove config-override usages:
neondatabase/cloud#13417
Closes: #7736
Overall plan:
https://www.notion.so/neondatabase/Rollout-Plan-simplified-pageserver-initialization-f935ae02b225444e8a41130b7d34e4ea?pvs=4

Co-authored-by: Christian Schwarz <christian@neon.tech>
@problame problame reopened this Jul 24, 2024
problame added a commit that referenced this issue Jul 24, 2024
@problame
pageserver: fail if id is present in pageserver.toml
ec79f89 
`identity.toml` is the authoritative place for `id` as of
#7766

Overall plan: Overall plan: https://www.notion.so/neondatabase/Rollout-Plan-simplified-pageserver-initialization-f935ae02b225444e8a41130b7d34e4ea?pvs=4

refs #7736
@problame problame mentioned this issue Jul 24, 2024
Merged
5 tasks
@problame problame added the a/deploy Area: related to deployment label Jul 24, 2024
@problame problame self-assigned this Jul 26, 2024
problame added a commit that referenced this issue Jul 29, 2024
@problame
pageserver: fail if id is present in pageserver.toml (#8489)
d09dad0 
Overall plan:
https://www.notion.so/neondatabase/Rollout-Plan-simplified-pageserver-initialization-f935ae02b225444e8a41130b7d34e4ea?pvs=4

---

`identity.toml` is the authoritative place for `id` as of
#7766

refs #7736
koivunej pushed a commit that referenced this issue Jul 29, 2024
@problame @koivunej
pageserver: fail if id is present in pageserver.toml (#8489)
e0a5bb1 
Overall plan:
https://www.notion.so/neondatabase/Rollout-Plan-simplified-pageserver-initialization-f935ae02b225444e8a41130b7d34e4ea?pvs=4

---

`identity.toml` is the authoritative place for `id` as of
#7766

refs #7736
@problame
Copy link
Contributor

This epic is complete as of this week's storage deploy. https://neondb.slack.com/archives/C03H1K0PGKH/p1722347077872989?thread_ts=1722244979.635389&cid=C03H1K0PGKH

@problame problame mentioned this issue Aug 12, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@problame problame

@VladLazar VladLazar

Labels
a/deploy Area: related to deployment a/tech_debt Area: related to tech debt
Projects
None yet
Milestone
No milestone
2 participants
@problame @VladLazar