Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug fix: Remote restores should only be eligible on uninitialized clusters #249

Merged
merged 1 commit into from
Jul 26, 2024

Conversation

davissp14
Copy link
Contributor

Problem
After a cluster is restored, the restore.lock is set on the primary to prevent the restore from re-triggering on reboot. However, the restore.lock is not set on the replicas so when they are rebooted the restore process will kick in and blow away the repmgr database...

If you hit this issue, you will see a log on your replica saying: Restoring from a hot standby., with the following logs:

2024-07-26T01:48:22Z app[8d9511aee26008] ord [info]postgres | 2024-07-26 01:48:22.876 UTC [14421] FATAL:  database "repmgr" does not exist
2024-07-26T01:48:22Z app[8d9511aee26008] ord [info]failed post-init: failed to register new standby: failed to register standby: exit status 1. Retrying...

Solution
The restore process is now being properly restricted to uninitialized clusters. If the cluster has already been initialized, meaning the cluster's primary has been established, the restore process will be ignored.

@davissp14 davissp14 changed the title Remote restores should only be eligible on uninitialized clusters Bug fix: Remote restores should only be eligible on uninitialized clusters Jul 26, 2024
@davissp14 davissp14 merged commit 8f5e645 into master Jul 26, 2024
6 of 7 checks passed
@davissp14 davissp14 deleted the restore-bug-fix branch July 26, 2024 02:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant