Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport/backport 15508 to 2.x #15646

Closed

Conversation

sumitasr
Copy link
Member

@sumitasr sumitasr commented Sep 4, 2024

Description

[Describe what this change achieves]

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

reta and others added 30 commits June 27, 2024 14:02
…outside of the OpenSearch core (opensearch-project#14575) (opensearch-project#14579)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
(cherry picked from commit 391dee2)
…project#14566)

* fix file cache initialization



* changelog



* add test



---------


(cherry picked from commit 2be25bb)

Signed-off-by: panguixin <panguixin@bytedance.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…ck (opensearch-project#14187) (opensearch-project#14565)

* Moving query recompute out of write lock



* [Tiered Caching] Moving query recomputation logic outside of write lock



* Adding java doc for the completable map



* Changes to call future handler only once per key



* Fixing spotless check



* Added changelog



* Addressing comments



* Fixing gradle fail



* Addressing comments to refactor unit test



* minor UT refactor



---------

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com>
Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com>
…tly (opensearch-project#14550) (opensearch-project#14589)

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently



* Addressing comment



---------




(cherry picked from commit 729276f)

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com>
…-project#14594)

(cherry picked from commit 8ad199d)

Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…ject#14427) (opensearch-project#14593)

* Add unittests for RemoteClusterStateAttributesManager


(cherry picked from commit a34270d)

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
opensearch-project#14552) (opensearch-project#14600)

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>



* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>



* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>



* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>



* Fix issue 14519:Fix spotless issues. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>



* Fix issue 14519:update changelog



---------




(cherry picked from commit c71fd4a)

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Daniel Widdis <widdis@gmail.com>
…h-project#14554) (opensearch-project#14595)

(cherry picked from commit 8e493f3)

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…#14597) (opensearch-project#14598)

(cherry picked from commit 5c8623f)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…ository-azure (opensearch-project#14610) (opensearch-project#14616)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...



* Updating SHAs



* Update changelog



---------




(cherry picked from commit 8904557)

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
…4564) (opensearch-project#14617)

* Fix testInvalidInput flakiness



* Addressed andrross's comment



* rerun security check



---------



(cherry picked from commit f9512db)

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
…oject#14627)

(cherry picked from commit f1f4f89)

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…f recursion. (opensearch-project#14200) (opensearch-project#14629)

* Refactor FilterPath parse function (opensearch-project#12067)


* Implement unit tests for FilterPathTests (opensearch-project#12067)


* Write warn log if Filter is empty; Add comments (opensearch-project#12067)


* Add changelog



* Remove unnecessary log statement



* Remove unused logger



* Spotless apply



* Remove incorrect changelog



---------



(cherry picked from commit 0742453)

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
… RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513) (opensearch-project#14639)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings


(cherry picked from commit 501a702)

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
opensearch-project#14635) (opensearch-project#14661)

* Allow system index warning



* Add to CHANGELOG



* Address code review comments



---------


(cherry picked from commit f14b5c8)

Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
… in /modules/lang-mustache (opensearch-project#14672) (opensearch-project#14676)

* Bump com.github.spullara.mustache.java:compiler

Bumps [com.github.spullara.mustache.java:compiler](https://github.com/spullara/mustache.java) from 0.9.13 to 0.9.14.
- [Commits](spullara/mustache.java@mustache.java-0.9.13...mustache.java-0.9.14)

---
updated-dependencies:
- dependency-name: com.github.spullara.mustache.java:compiler
  dependency-type: direct:production
  update-type: version-update:semver-patch
...



* Updating SHAs



* Update changelog



---------




(cherry picked from commit ed1d852)

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
…sitory-azure (opensearch-project#14673) (opensearch-project#14678)

* Bump net.minidev:accessors-smart in /plugins/repository-azure

Bumps [net.minidev:accessors-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/netplex/json-smart-v2/releases)
- [Commits](netplex/json-smart-v2@2.5.0...2.5.1)

---
updated-dependencies:
- dependency-name: net.minidev:accessors-smart
  dependency-type: direct:production
  update-type: version-update:semver-patch
...



* Updating SHAs



* Update changelog



---------




(cherry picked from commit 41fa085)

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
… values and index_prefixes (opensearch-project#10959) (opensearch-project#14686)

* Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes


* Add more test



* modify change log



* Fix test failure



* Change the indexAnalyzer used by prefix field



* Skip old version for yaml test



* Optimize some code



* Fix test failure



* Modify yaml test description



* Remove the name parameter for setAnalyzer()



---------


(cherry picked from commit 2e63913)

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…nsearch-project#14415) (opensearch-project#14692)

* Create new extension point in SystemIndexPlugin for a single plugin to get registered system indices



* Add to CHANGELOG



* WIP on system indices from IndexNameExpressionResolver



* Add test in IndexNameExpressionResolverTests



* Remove changes in SystemIndexPlugin



* Add method in IndexNameExpressionResolver to get matching system indices



* Show how resolver can be chained to get system indices



* Fix forbiddenApis check



* Update CHANGELOG



* Make SystemIndices internal



* Remove unneeded changes



* Fix CI failures



* Fix precommit errors



* Use Regex instead of WildcardMatcher



* Address code review feedback



* Allow caller to pass index expressions



* Create SystemIndexRegistry



* Update CHANGELOG



* Remove singleton limitation



* Add javadoc



* Add @experimentalapi annotation



---------


(cherry picked from commit bf56227)

Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…dsBalancer to filter remote search shard from relocation decision (opensearch-project#14613) (opensearch-project#14643)

(cherry picked from commit 58d1164)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…ensearch-project#14683)

(cherry picked from commit 51af2e2)

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Arpit-Bandejiya <abandeji@amazon.com>
…pensearch-project#14690)

* Adding QueryGroup schema (opensearch-project#13669)

* rebase with opensearch/main

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add resourceLimitGroupId propagation logic from coordinator to data nodes

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add sandbox schema

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add resourceLimitGroupTests

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add resourceLimitGroupMetadata tests

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* run spotlessApply

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add mode field in ResourceLimitGroup schema

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix breaking testcases

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add task cancellation skeleton

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add multitenant labels in searchSource builder

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* write custom xcontent parser for ResourceLimitGroup

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* remove unrelated changes

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* remove non-existing import fro cluster settings

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* remove non releated changes

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add _id as the resourceLimitGroup key

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add change to register resource limit group metadata

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add updatedAt in resource limit group

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* rename resourceLimitGroup to queryGroup

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* address the comments on PR

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* rename the mode member var to resiliency mode

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* address comments

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add change in CHANGELOG

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add tests for custom namedWritable QueryGroupMetadata

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* structure resourceLimits into an object

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add QueryGroup.toXContent test case

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix precommit errors

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix precommit errors

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix assemble errors

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix checkstyle errors

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* address comments

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

---------

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add min supported version in query group metadata

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

---------

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
…) (opensearch-project#14691)

(cherry picked from commit acc4631)

Signed-off-by: Chenyang Ji <cyji@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…search-project#14701)

(cherry picked from commit 5ef41cf)

Signed-off-by: Zelin Hao <zelinhao@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…arch-project#14700)

(cherry picked from commit a04cf24)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…pensearch-project#14695)

(cherry picked from commit b8dc46d)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…arch-project#14702)

(cherry picked from commit c4d960f)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
… calculation inside LocalShardBalancer (opensearch-project#14675) (opensearch-project#14689)

(cherry picked from commit 6d0484a)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
… for transient settings (opensearch-project#14476) (opensearch-project#14705)

(cherry picked from commit b068355)

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…pensearch-project#14707)

(cherry picked from commit 605543b)

Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
opensearch-trigger-bot bot and others added 26 commits August 31, 2024 21:43
…pinned timestamp setting (opensearch-project#15457) (opensearch-project#15553)

(cherry picked from commit 30ed15d)

Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sachin Kale <kalsac@amazon.com>
…arch-project#15511) (opensearch-project#15540)

* Extract SegmentReplicator class from SegmentReplicationTargetService.

This change separates code that initiates replication from the target service component in prepartion
for implementing a task to initate replication events on an interval.



* Pass timeout value to Replicator instead of fetching from shard



---------


(cherry picked from commit 2224d48)

Signed-off-by: Marc Handalian <marc.handalian@gmail.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Create Index using context (opensearch-project#15290)

Signed-off-by: Mohit Godwani <mgodwan@amazon.com>
…ch-project#15353) (opensearch-project#15566)

---------
Signed-off-by: Lakshya Taragi <lakshya.taragi@gmail.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>

Co-authored-by: Lakshya Taragi <157457166+ltaragi@users.noreply.github.com>
…ctor (opensearch-project#15416) (opensearch-project#15567)

(cherry picked from commit ef47252)

Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…#15574)

---------


(cherry picked from commit 6bae704)

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
) (opensearch-project#15570)

---------
Signed-off-by: Arpit Bandejiya <abandeji@amazon.com>

(cherry picked from commit b54e867)
…h-project#15526) (opensearch-project#15587)

---------

(cherry picked from commit cfcfe21)

Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sachin Kale <kalsac@amazon.com>
…ject#15465) (opensearch-project#15589)

* Add limit on number of processors in Ingest pipelines


(cherry picked from commit 41ba00a)

Signed-off-by: Rai <nndri@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…edIndex (opensearch-project#15598) (opensearch-project#15605)

(cherry picked from commit 022acba)

Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Sachin Kale <kalsac@amazon.com>
…t#13966) (opensearch-project#15594)

* Support cancellation for admin apis (opensearch-project#13966)

* Support cancellation for admin apis - add implementation for _cat/shards

Signed-off-by: Somesh Gupta <someshgupta987@gmail.com>
(cherry picked from commit 9f81479)
…t#15100) (opensearch-project#15582)

Simplify diff calculation logic for remote routing table

Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com>
…bugEnabled (opensearch-project#15204) (opensearch-project#15597)

* Fix test MasterServiceTests.testClusterStateUpdateLoggingWithDebugEnabled


(cherry picked from commit b23c9de)

Signed-off-by: Sumit Bansal <sumitsb@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…rch-project#15509) (opensearch-project#15591)

(cherry picked from commit bcd09ab)

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…ct#15569)

* [SnapshotV2] Support centralize snapshot creation (opensearch-project#15124)

* Initial Commit to support centralize snapshot creation and implicit locking mechanism

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Fix deserilization error

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Fix gradle spotless check

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Fix listener

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Fix test

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Fix snapshot generation

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Modify cluster setting name

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Add more tests

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Uncomment pin timestamp code

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Modify log messages

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Add spotless check failure fix

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Fix completion listener for snapshot v2

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Elevate cluster state update priority for repository metadata update task

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Add more integ tests

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Add priority as IMMEDIATE for cluster state repo update task only for v2 snapshots

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Fix build error

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Fix spotless error

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Add repository setting for snapshot v2

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Address review comments

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Add integ test to verify snapshot creation if shallow copy repo setting is disabled

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Fix spotless vilation error

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Address review comment

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Address review comments

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Add min version check for backward compatibility

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* address review comments

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* add integ test for master failover scenario

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Add more integ tests

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* refactor code

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* add changelog

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Add pinned timestamp setting in integ tests

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

---------

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
Signed-off-by: Anshu Agarwal <anshuagarwal11@gmail.com>
Co-authored-by: Anshu Agarwal <anshukag@amazon.com>
(cherry picked from commit 23cba28)
Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Fix spotless check failure

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* fix :server:japicmp failure

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* backort PR#15602

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

---------

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
Signed-off-by: Anshu Agarwal <anshuagarwal11@gmail.com>
Co-authored-by: Anshu Agarwal <anshukag@amazon.com>
…n is false (opensearch-project#15615)

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
…nsearch-project#15618)

(cherry picked from commit 12f6493)

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…it is required (opensearch-project#15617)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
…project#15616) (opensearch-project#15619)

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
Co-authored-by: Anshu Agarwal <anshukag@amazon.com>
(cherry picked from commit c62626d)
…om (opensearch-project#15010) (opensearch-project#15628)

local to remote pool for hot to warm tiering

Signed-off-by: Neetika Singhal <neetiks@amazon.com>
(cherry picked from commit c19cf88)
…t#15626) (opensearch-project#15629)

(cherry picked from commit bc02f90)

Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…#15579) (opensearch-project#15623)

* Abstract out RemoteFsTranslog with pinned timestamps


(cherry picked from commit deeb2de)

Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…ct#15624 to 2.x (opensearch-project#15595)

* [SnapshotV2] Snapshot Status API changes (opensearch-project#15409)

---------

Signed-off-by: Lakshya Taragi <lakshya.taragi@gmail.com>
(cherry picked from commit 925f41b)

* Remove breaking changes from opensearch-project#15409 (opensearch-project#15624)

Signed-off-by: Lakshya Taragi <lakshya.taragi@gmail.com>
(cherry picked from commit a60b668)
* [ClusterManagerTaskThrottler Improvements] : Add shallow check in ClusterManagerTaskThrottler to fail fast before computeIfPresent to avoid lock when queue is full

Signed-off-by: Sumit Bansal <sumitsb@amazon.com>
(cherry picked from commit 17b5f98)
@sumitasr sumitasr closed this Sep 4, 2024
Copy link
Contributor

github-actions bot commented Sep 4, 2024

❕ Gradle check result for 9517328: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Copy link

codecov bot commented Sep 4, 2024

Codecov Report

Attention: Patch coverage is 46.32546% with 409 lines in your changes missing coverage. Please review.

Project coverage is 71.74%. Comparing base (58467f6) to head (9517328).
Report is 3272 commits behind head on main.

Files with missing lines Patch % Lines
...rc/main/java/org/opensearch/client/RestClient.java 27.08% 104 Missing and 1 partial ⚠️
...va/org/opensearch/client/ReactiveDataProducer.java 0.00% 63 Missing ⚠️
...va/org/opensearch/client/ReactiveDataConsumer.java 0.00% 53 Missing ⚠️
...org/opensearch/client/ReactiveRequestProducer.java 0.00% 49 Missing ⚠️
...ensearch/gradle/test/rest/RestResourcesPlugin.java 0.00% 18 Missing ⚠️
.../java/org/opensearch/client/StreamingResponse.java 0.00% 17 Missing ⚠️
...rg/opensearch/client/ReactiveResponseConsumer.java 0.00% 15 Missing ⚠️
...n/java/org/opensearch/client/StreamingRequest.java 43.47% 12 Missing and 1 partial ⚠️
...n/annotation/processor/ApiAnnotationProcessor.java 58.06% 7 Missing and 6 partials ⚠️
.../search/pipeline/common/SortResponseProcessor.java 81.96% 5 Missing and 6 partials ⚠️
... and 15 more
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #15646      +/-   ##
============================================
+ Coverage     71.40%   71.74%   +0.33%     
- Complexity    62313    64134    +1821     
============================================
  Files          5102     5249     +147     
  Lines        293175   299689    +6514     
  Branches      42717    43642     +925     
============================================
+ Hits         209345   215007    +5662     
- Misses        66227    66731     +504     
- Partials      17603    17951     +348     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.