|
| 1 | +--- |
| 2 | +layout: docs |
| 3 | +page_title: Vault HA upgrades without Autopilot Upgrade Automation (Pre 1.11) |
| 4 | +description: |- |
| 5 | + Upgrade instructions for Vault HA Pre 1.11 or Vault without autopilot upgrade automation being enabled. Be sure to read the Upgrading-Vault Guides as well. |
| 6 | +--- |
| 7 | + |
| 8 | +# Vault HA upgrades without Autopilot Upgrade Automation (Pre 1.11) |
| 9 | + |
| 10 | +This is our recommended upgrade procedure if **one** of the following applies: |
| 11 | + |
| 12 | +- Running Vault version earlier than 1.11 |
| 13 | +- Opt-out the [Autopilot automated upgrade](/vault/docs/concepts/integrated-storage/autopilot#automated-upgrade) features with Vault 1.11 or later |
| 14 | +- Running Vault with external storage backend such as Consul |
| 15 | + |
| 16 | +You should consider how to apply the steps described in this document to your |
| 17 | +particular setup since HA setups can differ on whether a load balancer is in |
| 18 | +use, what addresses clients are being given to connect to Vault (standby + |
| 19 | +leader, leader-only, or discovered via service discovery), etc. |
| 20 | + |
| 21 | +If you are running on Vault 1.11+ with Integrated Storage and wish to enable the |
| 22 | +Autopilot upgrade automation features, read to the [automated |
| 23 | +upgrades](/vault/docs/concepts/integrated-storage/autopilot#automated-upgrades) |
| 24 | +documentation for details and the [Automate Upgrades with Vault |
| 25 | +Enterprise](/vault/tutorials/raft/raft-upgrade-automation) tutorial for |
| 26 | +additional guidance. |
| 27 | + |
| 28 | + |
| 29 | +## HA Installations |
| 30 | + |
| 31 | +Regardless of the method you use, do not fail over from a newer version of Vault |
| 32 | +to an older version. Our suggested procedure is designed to prevent this. |
| 33 | + |
| 34 | +Please note that Vault does not support true zero-downtime upgrades, but with |
| 35 | +proper upgrade procedure the downtime should be very short (a few hundred |
| 36 | +milliseconds to a second depending on how the speed of access to the storage |
| 37 | +backend). |
| 38 | + |
| 39 | +<Warning title="Important"> |
| 40 | + |
| 41 | +If you are currently running on Vault 1.11+ with Integrated Storage and have |
| 42 | +chosen to opt-out the Autopilot automated upgrade features, please disable the |
| 43 | +default automated upgrade migrations feature of the Vault. To disable this |
| 44 | +feature, follow the [Automate Upgrades with Vault Enterprise Autopilot |
| 45 | +configuration](/vault/tutorials/raft/raft-upgrade-automation#autopilot-configuration) |
| 46 | +tutorial for more details. Without disabling this feature, you may run into Lost |
| 47 | +Quorum issue as described in the [Quorum lost while upgrading the vault from |
| 48 | +1.11.0 to later version of |
| 49 | +it](https://support.hashicorp.com/hc/en-us/articles/7122445204755-Quorum-lost-while-upgrading-the-vault-from-1-11-0-to-later-version-of-it) |
| 50 | +article. |
| 51 | + |
| 52 | +</Warning> |
| 53 | + |
| 54 | +Perform these steps on each standby: |
| 55 | + |
| 56 | +1. Properly shut down Vault on the standby node via `SIGINT` or `SIGTERM` |
| 57 | +2. Replace the Vault binary with the new version; ensure that `mlock()` |
| 58 | + capability is added to the new binary with |
| 59 | + [setcap](/vault/docs/configuration#disable_mlock) |
| 60 | +3. Start the standby node |
| 61 | +4. Unseal the standby node |
| 62 | +5. Verify `vault status` shows correct Version and HA Mode is `standby` |
| 63 | +6. Review the node's logs to ensure successful startup and unseal |
| 64 | + |
| 65 | +At this point all standby nodes are upgraded and ready to take over. The |
| 66 | +upgrade will not complete until one of the upgraded standby nodes takes over |
| 67 | +active duty. |
| 68 | + |
| 69 | +To complete the cluster upgrade: |
| 70 | + |
| 71 | +1. Properly shut down the remaining (active) node |
| 72 | + |
| 73 | + <Note> |
| 74 | + |
| 75 | + It is important that you shut the node down properly. |
| 76 | + This will perform a step-down and release the HA lock, allowing a standby |
| 77 | + node to take over with a very short delay. |
| 78 | + If you kill Vault without letting it release the lock, a standby node will |
| 79 | + not be able to take over until the lock's timeout period has expired. This |
| 80 | + is backend-specific but could be ten seconds or more. |
| 81 | + |
| 82 | + </Note> |
| 83 | + |
| 84 | +2. Replace the Vault binary with the new version; ensure that `mlock()` |
| 85 | + capability is added to the new binary with |
| 86 | + [setcap](/vault/docs/configuration#disable_mlock) |
| 87 | +3. Start the node |
| 88 | +4. Unseal the node |
| 89 | +5. Verify `vault status` shows correct Version and HA Mode is `standby` |
| 90 | +6. Review the node's logs to ensure successful startup and unseal |
| 91 | + |
| 92 | +Internal upgrade tasks will happen after one of the upgraded standby nodes |
| 93 | +takes over active duty. |
| 94 | + |
| 95 | +Be sure to also read and follow any instructions in the version-specific |
| 96 | +upgrade notes. |
| 97 | + |
| 98 | +## Enterprise Replication Installations |
| 99 | + |
| 100 | +<Note> |
| 101 | + |
| 102 | +Prior to any upgrade, be sure to also read and follow any instructions in the |
| 103 | +version-specific upgrade notes which are found in the navigation menu for this |
| 104 | +documentation. |
| 105 | + |
| 106 | +</Note> |
| 107 | + |
| 108 | +Upgrading Vault Enterprise clusters which participate in [Enterprise |
| 109 | +Replication](/vault/docs/enterprise/replication) requires the following basic |
| 110 | +order of operations: |
| 111 | + |
| 112 | +- **Upgrade the replication secondary instances first** using appropriate |
| 113 | + guidance from the previous sections depending on whether each secondary |
| 114 | + instance is non-HA or HA |
| 115 | +- Verify functionality of each secondary instance after upgrading |
| 116 | +- When satisfied with functionality of upgraded secondary instances, upgrade |
| 117 | + the primary instance |
| 118 | + |
| 119 | +<Note> |
| 120 | + |
| 121 | +It is not safe to replicate from a newer version of Vault to an older version. |
| 122 | +When upgrading replicated clusters, ensure that upstream clusters are always on |
| 123 | +older versions of Vault than downstream clusters. |
| 124 | + |
| 125 | +</Note> |
| 126 | + |
| 127 | +Here is an example of upgrading four Vault replicated Vault clusters: |
| 128 | + |
| 129 | + |
| 130 | + |
| 131 | +In the above scenario, the ideal upgrade procedure would be as follows, |
| 132 | +verifying functionality after each cluster upgrade. |
| 133 | + |
| 134 | +1. Upgrade Clusters B and D, using the HA upgrade process above. These clusters |
| 135 | + have no downstream clusters, so they should be upgraded first, but the |
| 136 | + ordering of B vs D does not matter. |
| 137 | +2. Upgrade Cluster C, which now has an upgraded downstream cluster (Cluster D). |
| 138 | + Because Cluster C is a cluster, it should also use the HA upgrade process. |
| 139 | +3. Finally, upgrade Cluster A. All clusters downstream of A will already be |
| 140 | + upgraded. It should be upgraded last, as it is a Performance Primary and a DR |
| 141 | + Primary. |
0 commit comments