Add docs on searchable snaps costs (#77607) (#77745)

* Add docs on searchable snaps costs

Adds a note on why searchable snapshots is cheaper, including warnings
that it might be more expensive too.

* Split into sections

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

* data -> the shard contents

* More wording tweaks

* Apply suggestions from code review

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

docs/reference/ilm/actions/ilm-searchable-snapshot.asciidoc

@@ -17,11 +17,9 @@ the frozen phase, the action mounts a <<partially-mounted,partially mounted
 index>> to the frozen tier. In other phases, the action mounts a
 <<fully-mounted,fully mounted index>> to the corresponding data tier.
 
-IMPORTANT: If the `searchable_snapshot` action is used in the hot phase the
-subsequent phases cannot include the `shrink`, `forcemerge`, or `freeze`
-actions.
+If the `searchable_snapshot` action is used in the hot phase the subsequent
+phases cannot include the `shrink`, `forcemerge`, or `freeze` actions.
 
-[NOTE]
 This action cannot be performed on a data stream's write index. Attempts to do
 so will fail. To convert the index to a searchable snapshot, first
 <<manually-roll-over-a-data-stream,manually roll over>> the data stream. This
@@ -31,6 +29,12 @@ Using a policy that makes use of the <<ilm-rollover, rollover>> action
 in the hot phase will avoid this situation and the need for a manual rollover for future
 managed indices.
 
+IMPORTANT: Mounting and relocating the shards of {search-snap} indices involves
+copying the shard contents from the snapshot repository. This may incur
+different costs from the copying between nodes that happens with regular
+indices. These costs are typically lower but, in some environments, may be
+higher. See <<searchable-snapshots-costs>> for more details.
+
 By default, this snapshot is deleted by the <<ilm-delete, delete action>> in the delete phase.
 To keep the snapshot, set `delete_searchable_snapshot` to `false` in the delete action.
 

docs/reference/searchable-snapshots/index.asciidoc

@@ -200,6 +200,51 @@ node that does not have the <<data-frozen-node,`data_frozen`>> role, it will be
 is set to `0b`. Additionally, nodes with a shared  cache can only have a single
 <<path-settings,data path>>.
 
+[discrete]
+[[searchable-snapshots-costs]]
+=== Reduce costs with {search-snaps}
+
+In most cases, {search-snaps} reduce the costs of running a cluster by removing
+the need for replica shards  and for shard data to be copied between
+nodes. However, if it's particularly expensive to retrieve data from a snapshot
+repository in your environment, {search-snaps} may be more costly than
+regular indices. Ensure that the cost structure of your operating environment is
+compatible with {search-snaps} before using them.
+
+[discrete]
+[[searchable-snapshots-costs-replicas]]
+==== Replica costs
+
+For resiliency, a regular index requires multiple redundant copies of each shard
+across multiple nodes. If a node fails, {es} uses the redundancy to rebuild any
+lost shard copies. A {search-snap} index doesn't require replicas. If a node
+containing a {search-snap} index fails, {es} can rebuild the lost shard copy
+from the snapshot repository.
+
+Without replicas, rarely-accessed {search-snap} indices require far fewer
+resources. A cold data tier that contains replica-free fully-mounted
+{search-snap} indices requires half the nodes and disk space of a tier
+containing the same data in regular indices. The frozen tier, which contains
+only partially-mounted {search-snap} indices, requires even fewer resources.
+
+[discrete]
+[[snapshot-retrieval-costs]]
+==== Data transfer costs
+
+When a shard of a regular index is moved between nodes, its contents are copied
+from another node in your cluster. In many environments, the costs of moving data
+between nodes are significant, especially if running in a Cloud environment with
+nodes in different zones. In contrast, when mounting a {search-snap} index or
+moving one of its shards, the data is always copied from the snapshot repository.
+This is typically much cheaper.
+
+WARNING: Most cloud providers charge significant fees for data transferred
+between regions and for data transferred out of their platforms. You should only
+mount snapshots into a cluster that is in the same region as the snapshot
+repository. If you wish to search data across multiple regions, configure
+multiple clusters and use <<modules-cross-cluster-search,{ccs}>> or
+<<xpack-ccr,{ccr}>> instead of {search-snaps}.
+
 [discrete]
 [[back-up-restore-searchable-snapshots]]
 === Back up and restore {search-snaps}