Adds cluster setting to configure index state management jitter

[downsrob]

Signed-off-by: Clay Downs downsrob@amazon.com

Description of changes: Adds cluster setting to allow users to enable and configure the jitter setting for index state management jobs. The purpose of the jitter is to add a randomized delay to the start time of ISM jobs to make sure that all indices do not become active at the same time. The jitter is stored as a double and represents the maximum percentage of the job interval to add to the job start time. The default jitter is 0.6, and the cluster setting resides under plugins.index_state_management.jitter.

The job scheduler has another configurable cluster setting, jitter_limit, which represents the maximum jitter which may be applied to the job. The jitter cluster setting may be higher than this limit, but jitter times will be capped to the jitter_limit percentage of the job interval. The job scheduler jitter_limit default is 0.6.

At this time, modifying the jitter cluster setting will only apply the changed jitter to newly created jobs.

CheckList:

• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov-commenter]

Codecov Report

Merging #153 (3aea0a5) into main (dafd3bb) will decrease coverage by 0.06%.
The diff coverage is 92.85%.

@@             Coverage Diff              @@
##               main     #153      +/-   ##
============================================
- Coverage     75.88%   75.82%   -0.07%     
+ Complexity     1922     1916       -6     
============================================
  Files           260      260              
  Lines         10992    11014      +22     
  Branches       1720     1721       +1     
============================================
+ Hits           8341     8351      +10     
- Misses         1699     1705       +6     
- Partials        952      958       +6     

Impacted Files	Coverage Δ
...t/indexstatemanagement/model/ManagedIndexConfig.kt	78.09% <75.00%> (-0.70%)	⬇️
...pensearch/indexmanagement/IndexManagementPlugin.kt	92.68% <100.00%> (+0.03%)	⬆️
...nt/indexstatemanagement/ManagedIndexCoordinator.kt	72.82% <100.00%> (+0.38%)	⬆️
...exstatemanagement/settings/ManagedIndexSettings.kt	99.10% <100.00%> (+0.05%)	⬆️
...sport/action/addpolicy/TransportAddPolicyAction.kt	59.09% <100.00%> (+0.81%)	⬆️
...ent/indexstatemanagement/util/ManagedIndexUtils.kt	79.31% <100.00%> (+0.08%)	⬆️
...arch/indexmanagement/rollup/RollupSearchService.kt	57.40% <0.00%> (-3.71%)	⬇️
...rch/indexmanagement/rollup/model/RollupMetadata.kt	82.73% <0.00%> (-1.79%)	⬇️
...ensearch/indexmanagement/rollup/model/ISMRollup.kt	92.98% <0.00%> (-1.76%)	⬇️
...earch/indexmanagement/transform/model/Transform.kt	86.00% <0.00%> (-1.50%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dafd3bb...3aea0a5. Read the comment docs.

[thalurur]

minor: Do you think it makes sense to set this setting at index_management level so it can be applied to other features of index management in future if needed?

[downsrob]

I think that if a jitter setting is added to the other features of index management then it would make sense to have the setting feature-scoped by adding it as a setting at the transforms or rollups level, as I don't think a user's setting of jitter for index management would directly relate to how that user might want to configure a potential rollup jitter.