Change successfulSearchShardIndices to Set<Index>

[dzane17]

Description

Change successfulSearchShardIndices to Set<Index>

Passing the entire Index object allows greater flexibility over index String names. successfulSearchShardIndices is used in the Query Insights plugin to determine query field types.

Related Issues

RFC opensearch-project/query-insights#69
PR opensearch-project/query-insights#130

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

✅ Gradle check result for 9a5dac6: SUCCESS

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.03%. Comparing base (a81b868) to head (058fb78).
Report is 7 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #16110      +/-   ##
============================================
+ Coverage     71.94%   72.03%   +0.08%     
- Complexity    64612    64729     +117     
============================================
  Files          5298     5301       +3     
  Lines        301952   302355     +403     
  Branches      43627    43681      +54     
============================================
+ Hits         217247   217805     +558     
+ Misses        66884    66704     -180     
- Partials      17821    17846      +25     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

✅ Gradle check result for 3ce3c96: SUCCESS

[reta]

@dzane17 could you please provide more context behind this change: why do you need to expose the IndicesService to plugins? thank you

[reta]

@cwperks - Please correct me if I am wrong, but onIndexModule can only be leveraged for listening shard level search events, not request level. We need the search request listener for what we are trying to achieve here! cc: @reta

Got it, thanks @jainankitk , so here is the way we could make it work, which is a bit cumbersome but solves the dependency problem, the QueryInsightsService in injected as the service class:

    @Override
    public Collection<Class<? extends LifecycleComponent>> getGuiceServiceClasses() {
        return List.of(QueryInsightsService.class);
    }

Since QueryInsightsListener is created before QueryInsightsService (and it is injectable), we could inject QueryInsightsListener right there and initialize it with the indicesService or QueryInsightsService (fe using setter) since none of this services are needed during QueryInsightsListener initialization:

@Inject
    public QueryInsightsService(
        final ClusterSettings clusterSettings,
        final ThreadPool threadPool,
        final Client client,
        final MetricsRegistry metricsRegistry,
        final NamedXContentRegistry namedXContentRegistry,
        final IndicesService indicesService,
        final QueryInsightsListener listener
    ) {
  }

[dzane17]

Thanks all for the input. Since indicesService does not contain mappings for ALL indices in the cluster, we have no choice but to retrieve mappings from cluster state. This involves some clunky map parsing so we will introduce a small cache in Query-Insights to limit the frequency of this field->fieldType lookup. ClusterService is already present in the plugin createComponents method so I have reduced this PR to remaining structural changes needed in core.

[github-actions]

❌ Gradle check result for a489774: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 058fb78: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[jainankitk]

Seems due to caching changes:

[Test Result](https://build.ci.opensearch.org/job/gradle-check/49029/testReport/) (1 failure / ±0)

    [org.opensearch.indices.IndicesRequestCacheCleanupIT.testDynamicStalenessThresholdUpdate](https://build.ci.opensearch.org/job/gradle-check/49029/testReport/junit/org.opensearch.indices/IndicesRequestCacheCleanupIT/testDynamicStalenessThresholdUpdate/)

@sgup432 - Can you take a look at this?

[github-actions]

✅ Gradle check result for 058fb78: SUCCESS

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-16110-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 5279d21f8c85e78a91159af6e17e52789355f2a4
# Push it to GitHub
git push --set-upstream origin backport/backport-16110-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-16110-to-2.x.