From d411609f07801125b609d7d641c50f9bbb4910c8 Mon Sep 17 00:00:00 2001 From: Gaurav Bafna Date: Mon, 10 Jun 2024 11:32:19 +0530 Subject: [PATCH 1/6] Mark docrep to remote migration as GA and modify settings names Signed-off-by: Gaurav Bafna --- .../remote-store/migrating-to-remote.md | 42 ++++++++++--------- 1 file changed, 22 insertions(+), 20 deletions(-) diff --git a/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md b/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md index 65ae48735b..961ebb2f26 100644 --- a/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md +++ b/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md @@ -8,9 +8,6 @@ grand_parent: Availability and recovery # Migrating to remote-backed storage -This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/OpenSearch/issues/7986). -{: .warning} - Introduced 2.14 {: .label .label-purple } @@ -24,9 +21,9 @@ Rolling upgrades, sometimes referred to as *node replacement upgrades*, can be p Review [Upgrading OpenSearch]({{site.url}}{{site.baseurl}}/upgrade-opensearch/index/) for recommendations about backing up your configuration files and creating a snapshot of the cluster state and indexes before you make any changes to your OpenSearch cluster. -Before migrating to remote-backed storage, upgrade to OpenSearch 2.14. +Before migrating to remote-backed storage, upgrade to OpenSearch 2.15. -As of OpenSearch 2.14, OpenSearch nodes cannot be migrated back to document replication. If you need to revert the migration, then you will need to perform a fresh installation of OpenSearch and restore the cluster from a snapshot. Take a snapshot and store it in a remote repository before beginning the upgrade procedure. +As of OpenSearch 2.15, OpenSearch nodes cannot be migrated back to document replication. If you need to revert the migration, then you will need to perform a fresh installation of OpenSearch and restore the cluster from a snapshot. Take a snapshot and store it in a remote repository before beginning the upgrade procedure. {: .important} ## Performing the upgrade @@ -105,14 +102,14 @@ As of OpenSearch 2.14, OpenSearch nodes cannot be migrated back to document repl } ``` -1. Set the `remote_store.compatibility_mode` setting to `mixed` to allow remote-backed storage nodes to join the cluster. Then set `migration.direction` to `remote_store`, which allocates new indexes to remote-backed data nodes. The following example updates the aforementioned setting using the Cluster Settings API: +1. Set the `cluster.remote_store.compatibility_mode` setting to `mixed` to allow remote-backed storage nodes to join the cluster. Then set `migration.direction` to `remote_store`, which allocates new indexes to remote-backed data nodes. The following example updates the aforementioned setting using the Cluster Settings API: ```json PUT "/_cluster/settings?pretty" { "persistent": { - "remote_store.compatibility_mode": "mixed", - "migration.direction" : "remote_store" + "cluster.remote_store.compatibility_mode": "mixed", + "cluster.migration.direction" : "remote_store" } } ``` @@ -121,13 +118,15 @@ As of OpenSearch 2.14, OpenSearch nodes cannot be migrated back to document repl ```json { "acknowledged" : true, - "persistent" : { - "remote_store" : { - "compatibility_mode" : "mixed", - "migration.direction" : "remote_store" + "persistent" : { + "cluster" : { + "remote_store" : { + "compatibility_mode" : "mixed", + "migration.direction" : "remote_store" + } }, "transient" : { } - } + } } ``` @@ -233,8 +232,8 @@ As of OpenSearch 2.14, OpenSearch nodes cannot be migrated back to document repl PUT "/_cluster/settings?pretty" { "persistent": { - "remote_store.compatibility_mode": strict, - "migration.direction" : none + "cluster.remote_store.compatibility_mode": null, + "cluster.migration.direction" : null } } ``` @@ -243,7 +242,10 @@ As of OpenSearch 2.14, OpenSearch nodes cannot be migrated back to document repl ```json { "acknowledged" : true, - "persistent" : { strict }, + "persistent" : { + "cluster.remote_store.compatibility_mode": null, + "cluster.migration.direction" : null + }, "transient" : { } } ``` @@ -255,8 +257,8 @@ The migration to the remote store is now complete. Use the following cluster settings to enable migration to a remote-backed cluster. -| Field | Data type | Description | -| :--- |:--- |:---| -| `remote_store.compatibility_mode` | String | When set to `strict`, only allows the creation of either non-remote or remote nodes, depending upon the initial cluster type. When set to `mixed`, allows both remote and non-remote nodes to join the cluster. Default is `strict`. | -| `migration.direction` | String | Creates new shards only on remote-backed storage nodes. Default is `None`. | +| Field | Data type | Description | +|:------------------------------------------|:--- |:---| +| `cluster.remote_store.compatibility_mode` | String | When set to `strict`, only allows the creation of either non-remote or remote nodes, depending upon the initial cluster type. When set to `mixed`, allows both remote and non-remote nodes to join the cluster. Default is `strict`. | +| `cluster.migration.direction` | String | Creates new shards only on remote-backed storage nodes. Default is `None`. | From 337a53b87253a31b4e2d742f6f937b68cf75077b Mon Sep 17 00:00:00 2001 From: Gaurav Bafna <85113518+gbbafna@users.noreply.github.com> Date: Mon, 10 Jun 2024 15:29:58 +0530 Subject: [PATCH 2/6] Update _tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md Co-authored-by: Bhumika Saini Signed-off-by: Gaurav Bafna <85113518+gbbafna@users.noreply.github.com> --- .../remote-store/migrating-to-remote.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md b/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md index 961ebb2f26..ef9de8d975 100644 --- a/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md +++ b/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md @@ -102,7 +102,7 @@ As of OpenSearch 2.15, OpenSearch nodes cannot be migrated back to document repl } ``` -1. Set the `cluster.remote_store.compatibility_mode` setting to `mixed` to allow remote-backed storage nodes to join the cluster. Then set `migration.direction` to `remote_store`, which allocates new indexes to remote-backed data nodes. The following example updates the aforementioned setting using the Cluster Settings API: +1. Set the `cluster.remote_store.compatibility_mode` setting to `mixed` to allow remote-backed storage nodes to join the cluster. Then set `cluster.migration.direction` to `remote_store`, which allocates new indexes to remote-backed data nodes. The following example updates the aforementioned setting using the Cluster Settings API: ```json PUT "/_cluster/settings?pretty" From 957da2860c8aff76433c3143b3659c231c7da0e6 Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Mon, 17 Jun 2024 10:49:46 -0600 Subject: [PATCH 3/6] Update migrating-to-remote.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi --- .../remote-store/migrating-to-remote.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md b/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md index ef9de8d975..e6edd4d53b 100644 --- a/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md +++ b/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md @@ -8,7 +8,7 @@ grand_parent: Availability and recovery # Migrating to remote-backed storage -Introduced 2.14 +Introduced 2.15 {: .label .label-purple } Remote-backed storage offers a new way to protect against data loss by automatically creating backups of all index transactions and sending them to remote storage. To use this feature, [segment replication]({{site.url}}{{site.baseurl}}/opensearch/segment-replication/) must be enabled. @@ -21,9 +21,9 @@ Rolling upgrades, sometimes referred to as *node replacement upgrades*, can be p Review [Upgrading OpenSearch]({{site.url}}{{site.baseurl}}/upgrade-opensearch/index/) for recommendations about backing up your configuration files and creating a snapshot of the cluster state and indexes before you make any changes to your OpenSearch cluster. -Before migrating to remote-backed storage, upgrade to OpenSearch 2.15. +Before migrating to remote-backed storage, upgrade to OpenSearch 2.15 or later. -As of OpenSearch 2.15, OpenSearch nodes cannot be migrated back to document replication. If you need to revert the migration, then you will need to perform a fresh installation of OpenSearch and restore the cluster from a snapshot. Take a snapshot and store it in a remote repository before beginning the upgrade procedure. +Before upgrading to OpenSearch 2.15, take a cluster snapshot and store it remotely. OpenSearch 2.15 nodes cannot revert to document replication. If migration needs to be undone, perform a fresh OpenSearch installation and restore from the remote snapshot. Securely storing the snapshot remotely allows retrieving and restoring it if issues arise during migration. {: .important} ## Performing the upgrade From 01b7d0a79f49fbf13e0d2f8f238198864d7813bf Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Tue, 18 Jun 2024 15:45:37 -0600 Subject: [PATCH 4/6] Update _tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi --- .../remote-store/migrating-to-remote.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md b/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md index e6edd4d53b..9388d52c12 100644 --- a/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md +++ b/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md @@ -23,7 +23,7 @@ Review [Upgrading OpenSearch]({{site.url}}{{site.baseurl}}/upgrade-opensearch/in Before migrating to remote-backed storage, upgrade to OpenSearch 2.15 or later. -Before upgrading to OpenSearch 2.15, take a cluster snapshot and store it remotely. OpenSearch 2.15 nodes cannot revert to document replication. If migration needs to be undone, perform a fresh OpenSearch installation and restore from the remote snapshot. Securely storing the snapshot remotely allows retrieving and restoring it if issues arise during migration. +Before upgrading to OpenSearch 2.15, take a cluster snapshot and store it remotely. OpenSearch 2.15 nodes cannot revert to document replication. If a migration needs to be undone, perform a fresh OpenSearch installation and restore from the remote snapshot. Storing the snapshot remotely allows you to retrieve and restore it if issues arise during migration. {: .important} ## Performing the upgrade From 779e037fe919fef4f79bb8afd763a5250310f10e Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Tue, 18 Jun 2024 15:45:54 -0600 Subject: [PATCH 5/6] Update _tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi --- .../remote-store/migrating-to-remote.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md b/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md index 9388d52c12..f1ac0b6c22 100644 --- a/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md +++ b/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md @@ -259,6 +259,6 @@ Use the following cluster settings to enable migration to a remote-backed cluste | Field | Data type | Description | |:------------------------------------------|:--- |:---| -| `cluster.remote_store.compatibility_mode` | String | When set to `strict`, only allows the creation of either non-remote or remote nodes, depending upon the initial cluster type. When set to `mixed`, allows both remote and non-remote nodes to join the cluster. Default is `strict`. | +| `cluster.remote_store.compatibility_mode` | String | When set to `strict`, only allows the creation of either non-remote or remote nodes, depending on the initial cluster type. When set to `mixed`, allows both remote and non-remote nodes to join the cluster. Default is `strict`. | | `cluster.migration.direction` | String | Creates new shards only on remote-backed storage nodes. Default is `None`. | From 4d4721f540e0d98613abfa5a29d1cf19a6c4bb65 Mon Sep 17 00:00:00 2001 From: Melissa Vagi Date: Tue, 18 Jun 2024 15:46:01 -0600 Subject: [PATCH 6/6] Update _tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi --- .../remote-store/migrating-to-remote.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md b/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md index f1ac0b6c22..d97150812e 100644 --- a/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md +++ b/_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md @@ -260,5 +260,5 @@ Use the following cluster settings to enable migration to a remote-backed cluste | Field | Data type | Description | |:------------------------------------------|:--- |:---| | `cluster.remote_store.compatibility_mode` | String | When set to `strict`, only allows the creation of either non-remote or remote nodes, depending on the initial cluster type. When set to `mixed`, allows both remote and non-remote nodes to join the cluster. Default is `strict`. | -| `cluster.migration.direction` | String | Creates new shards only on remote-backed storage nodes. Default is `None`. | +| `cluster.migration.direction` | String | Creates new shards only on remote-backed storage nodes. Default is `None`. |