[RW Separation] Add routing preference to route requests only to search replicas.

[mch2]

Description

This adds SEARCH_REPLICA routing preference and defaults to this preference for indices that have them.

Related Issues

Resolves #15562

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for 72312a1: TIMEOUT

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for bb19dc2: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 72312a1: TIMEOUT

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for fced109: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❕ Gradle check result for aac9334: UNSTABLE

• TEST FAILURES:

      1 org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=snapshot.status/10_basic/Get snapshot status}

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov]

Codecov Report

Attention: Patch coverage is 84.21053% with 3 lines in your changes missing coverage. Please review.

Project coverage is 71.90%. Comparing base (e087272) to head (268535a).
Report is 9 commits behind head on main.

Files with missing lines	Patch %	Lines
...g/opensearch/cluster/routing/OperationRouting.java	66.66%	0 Missing and 2 partials ⚠️
...search/cluster/routing/IndexShardRoutingTable.java	90.90%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #15563      +/-   ##
============================================
- Coverage     72.00%   71.90%   -0.11%     
+ Complexity    64149    64123      -26     
============================================
  Files          5269     5269              
  Lines        299834   299878      +44     
  Branches      43330    43344      +14     
============================================
- Hits         215895   215623     -272     
- Misses        66217    66447     +230     
- Partials      17722    17808      +86     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[mch2]

removed changelog entry here to reduce all the backport / gc pain, will include 1 entry for all the separation prs and link to the design issue.

[github-actions]

✅ Gradle check result for 268535a: SUCCESS