Release version 2.1.0

[gaiksaya]

This is a component issue for 2.1.0.
Coming from opensearch-build#1818. Please follow the following checklist.
Please refer to the DATES in that post.

How to use this issue

This Component Release Issue

This issue captures the state of the OpenSearch release, on component/plugin level; its assignee is responsible for driving the release. Please contact them or @mention them on this issue for help.
Any release related work can be linked to this issue or added as comments to create visiblity into the release status.

Release Steps

There are several steps to the release process; these steps are completed as the whole component release and components that are behind present risk to the release. The component owner resolves the tasks in this issue and communicate with the overall release owner to make sure each component are moving along as expected.

Steps have completion dates for coordinating efforts between the components of a release; components can start as soon as they are ready far in advance of a future release. The most current set of dates is on the overall release issue linked at the top of this issue.

The Overall Release Issue

Linked at the top of this issue, the overall release issue captures the state of the entire OpenSearch release including references to this issue, the release owner which is the assignee is responsible for communicating the release status broadly. Please contact them or @mention them on that issue for help.

What should I do if my plugin isn't making any changes?

If including changes in this release, increment the version on 2.0 branch to 2.1.0 for Min/Core, and 2.1.0.0 for components. Otherwise, keep the version number unchanged for both.

Preparation

• Assign this issue to a release owner.
• Finalize scope and feature set and update the Public Roadmap.
• All the tasks in this issue have been reviewed by the release owner.
• Create, update, triage and label all features and issues targeted for this release with v2.1.0.

CI/CD

• All code changes for 2.1.0 are complete.
• Ensure working and passing CI.
• Check that this repo is included in the distribution manifest.

Pre-Release

• Update to the 2.1.0 release branch in the distribution manifest.
• Increment the version on the parent branch to the next development iteration.
• Gather, review and publish release notes following the rules and back port it to the release branch.git-release-notes may be used to generate release notes from your commit history.
• Confirm that all changes for 2.1.0 have been merged.
• Add this repo to the manifest for the next developer iteration.

Release Testing

• Find/fix bugs using latest tarball and docker image provided in parent release issue and update the release notes if necessary.
• Code Complete: Test within the distribution, ensuring integration, backwards compatibility, and performance tests pass.
• Sanity Testing: Sanity testing and fixing of critical issues found.
• File issues for all intermittent test failures.

Release

• Complete documentation.
• Verify all issued labeled for this release are closed or labelled for the next release.

Post Release

• Prepare for an eventual security fix development iteration by incrementing the version on the release branch to the next eventual patch version.
• Add this repo to the manifest of the next eventual security patch version.
• Suggest improvements to this template.
• Conduct a retrospective, and publish its results.

[ankitkala]

Performed the sanity for CCR using docker image.

88665a0f606e:2.1 ankikala$ curl -u 'admin:admin' -k "https://localhost:9200/_cat/nodes?v"
ip         heap.percent ram.percent cpu load_1m load_5m load_15m node.role node.roles                                        cluster_manager name
172.19.0.3           29          96  44    1.39    0.51     0.33 dimr      cluster_manager,data,ingest,remote_cluster_client *               opensearch-leader1
88665a0f606e:2.1 ankikala$ curl -u 'admin:admin' -k "https://localhost:9201/_cat/nodes?v"
ip         heap.percent ram.percent cpu load_1m load_5m load_15m node.role node.roles                                        cluster_manager name
172.19.0.2           29          96  35    1.27    0.50     0.33 dimr      cluster_manager,data,ingest,remote_cluster_client *               opensearch-follower1
88665a0f606e:2.1 ankikala$ curl -u 'admin:admin' -k -X PUT "https://localhost:9201/_cluster/settings?pretty" -H 'Content-Type: application/json' -d'
> {
>   "persistent": {
>     "cluster": {
>       "remote": {
>         "remote-cluster": {
>           "seeds": [
>             "172.19.0.3:9300"
>           ],
>           "transport.ping_schedule": "30s"
>         }
>       }
>     }
>   }
> }'
{
  "acknowledged" : true,
  "persistent" : {
    "cluster" : {
      "remote" : {
        "remote-cluster" : {
          "seeds" : [
            "172.19.0.3:9300"
          ],
          "transport" : {
            "ping_schedule" : "30s"
          }
        }
      }
    }
  },
  "transient" : { }
}
88665a0f606e:2.1 ankikala$ curl -k -u 'admin:admin' -X PUT "https://localhost:9200/remote-index?pretty" -H 'Content-Type: application/json' -d'
> {
>   "settings": {
>     "index": {
>       "number_of_shards": 1,
>       "number_of_replicas": 0
>     }
>   }
> }
> '

{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "index" : "remote-index"
}
88665a0f606e:2.1 ankikala$ 
88665a0f606e:2.1 ankikala$ curl -k -u 'admin:admin' -X PUT "https://localhost:9201/_plugins/_replication/remote-index/_start?pretty" -H 'Content-Type: application/json' -d'
> {
>   "leader_alias": "remote-cluster",
>   "leader_index": "remote-index",
>   "use_roles": {
>     "leader_cluster_role": "all_access",
>     "follower_cluster_role": "all_access"
>   }
> }'
{
  "acknowledged" : true
}
88665a0f606e:2.1 ankikala$ 
88665a0f606e:2.1 ankikala$ 
88665a0f606e:2.1 ankikala$ curl -k -u 'admin:admin' "https://localhost:9200/_cat/indices?v"
health status index                        uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   remote-index                 h7sSTa8wTgKstDwBCcaBIQ   1   0          0            0       208b           208b
yellow open   security-auditlog-2022.07.04 0SPv8_daS0-Tof_kqgul9g   1   1          6            0    112.1kb        112.1kb
green  open   .opendistro_security         _4A5XahKQQqNg0Z8RHtWNA   1   0         10            0     68.8kb         68.8kb
88665a0f606e:2.1 ankikala$ curl -k -u 'admin:admin' "https://localhost:9201/_cat/indices?v"
health status index                        uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   remote-index                 HZ6nojvcQlWS1bO3bzY-wA   1   0          0            0       208b           208b
yellow open   security-auditlog-2022.07.04 _a4fRCGjS5eNuyf1LlArSQ   1   1          4            0     58.6kb         58.6kb
green  open   .opendistro_security         LrntvG5tQTmztzajqYivXw   1   0         10            0     68.8kb         68.8kb
88665a0f606e:2.1 ankikala$ curl -k -u 'admin:admin' "https://localhost:9201/_plugins/_replication/remote-index/_status?pretty"
{
  "status" : "SYNCING",
  "reason" : "User initiated",
  "leader_alias" : "remote-cluster",
  "leader_index" : "remote-index",
  "follower_index" : "remote-index",
  "syncing_details" : {
    "leader_checkpoint" : -1,
    "follower_checkpoint" : -1,
    "seq_no" : 0
  }
}
88665a0f606e:2.1 ankikala$ curl -k -u 'admin:admin' -X POST "https://localhost:9200/remote-index/_doc/?pretty" -H 'Content-Type: application/json' -d'
> {
>     "user" : "kimchy",
>     "post_date" : "2009-11-15T14:12:12",
>     "message" : "trying out Elasticsearch5"
> }
> '
{
  "_index" : "remote-index",
  "_id" : "HDYJyoEBZaW2hRrmhozb",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 1
}
88665a0f606e:2.1 ankikala$ curl -k -u 'admin:admin' "https://localhost:9201/_plugins/_replication/remote-index/_status?pretty"
{
  "status" : "SYNCING",
  "reason" : "User initiated",
  "leader_alias" : "remote-cluster",
  "leader_index" : "remote-index",
  "follower_index" : "remote-index",
  "syncing_details" : {
    "leader_checkpoint" : 0,
    "follower_checkpoint" : 0,
    "seq_no" : 1
  }
}
88665a0f606e:2.1 ankikala$ curl -k -u 'admin:admin' "https://localhost:9200/_plugins/_replication/leader_stats?pretty"
{
  "num_replicated_indices" : 1,
  "operations_read" : 1,
  "translog_size_bytes" : 233,
  "operations_read_lucene" : 0,
  "operations_read_translog" : 1,
  "total_read_time_lucene_millis" : 0,
  "total_read_time_translog_millis" : 19,
  "bytes_read" : 185,
  "index_stats" : {
    "remote-index" : {
      "operations_read" : 1,
      "translog_size_bytes" : 233,
      "operations_read_lucene" : 0,
      "operations_read_translog" : 1,
      "total_read_time_lucene_millis" : 0,
      "total_read_time_translog_millis" : 19,
      "bytes_read" : 185
    }
  }
}88665a0f606e:2.1 ankikala$ curl -k -u 'admin:admin' "https://localhost:9201/_plugins/_replication/follower_stats?pretty"
{
  "num_syncing_indices" : 1,
  "num_bootstrapping_indices" : 0,
  "num_paused_indices" : 0,
  "num_failed_indices" : 0,
  "num_shard_tasks" : 1,
  "num_index_tasks" : 1,
  "operations_written" : 1,
  "operations_read" : 1,
  "failed_read_requests" : 0,
  "throttled_read_requests" : 0,
  "failed_write_requests" : 0,
  "throttled_write_requests" : 0,
  "follower_checkpoint" : -1,
  "leader_checkpoint" : 0,
  "total_write_time_millis" : 1111,
  "index_stats" : {
    "remote-index" : {
      "operations_written" : 1,
      "operations_read" : 1,
      "failed_read_requests" : 0,
      "throttled_read_requests" : 0,
      "failed_write_requests" : 0,
      "throttled_write_requests" : 0,
      "follower_checkpoint" : -1,
      "leader_checkpoint" : 0,
      "total_write_time_millis" : 1111
    }
  }
}
88665a0f606e:2.1 ankikala$ curl -k -u 'admin:admin' -X POST "https://localhost:9201/_plugins/_replication/remote-index/_pause?pretty" -H 'Content-Type: application/json' -d '{}'
{
  "acknowledged" : true
}
88665a0f606e:2.1 ankikala$ 
88665a0f606e:2.1 ankikala$ curl -k -u 'admin:admin' "https://localhost:9201/_plugins/_replication/remote-index/_status?pretty"
{
  "status" : "PAUSED",
  "reason" : "User initiated",
  "leader_alias" : "remote-cluster",
  "leader_index" : "remote-index",
  "follower_index" : "remote-index"
}
88665a0f606e:2.1 ankikala$ curl -k -u 'admin:admin' -X POST "https://localhost:9201/_plugins/_replication/remote-index/_resume?pretty" -H 'Content-Type: application/json' -d '{}'
{
  "acknowledged" : true
}
88665a0f606e:2.1 ankikala$ curl -k -u 'admin:admin' "https://localhost:9201/_plugins/_replication/remote-index/_status?pretty"
{
  "status" : "SYNCING",
  "reason" : "User initiated",
  "leader_alias" : "remote-cluster",
  "leader_index" : "remote-index",
  "follower_index" : "remote-index",
  "syncing_details" : {
    "leader_checkpoint" : 0,
    "follower_checkpoint" : 0,
    "seq_no" : 1
  }
}
88665a0f606e:2.1 ankikala$ curl -k -u 'admin:admin' -X POST "https://localhost:9200/remote-index/_doc/?pretty" -H 'Content-Type: application/json' -d'
> {
>     "user" : "kimchy2",
>     "post_date" : "2009-11-15T14:12:12",
>     "message" : "trying out Elasticsearch5"
> }
> '
{
  "_index" : "remote-index",
  "_id" : "KTYOyoEBZaW2hRrm7Izt",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 1,
  "_primary_term" : 1
}
88665a0f606e:2.1 ankikala$ curl -k -u 'admin:admin' "https://localhost:9201/_plugins/_replication/remote-index/_status?pretty"
{
  "status" : "SYNCING",
  "reason" : "User initiated",
  "leader_alias" : "remote-cluster",
  "leader_index" : "remote-index",
  "follower_index" : "remote-index",
  "syncing_details" : {
    "leader_checkpoint" : 1,
    "follower_checkpoint" : 1,
    "seq_no" : 2
  }
}
88665a0f606e:2.1 ankikala$ 
88665a0f606e:2.1 ankikala$ curl -k -u 'admin:admin' -X POST "https://localhost:9201/_plugins/_replication/remote-index/_stop?pretty" -H 'Content-Type: application/json' -d '{}'
{
  "acknowledged" : true
}
88665a0f606e:2.1 ankikala$ curl -k -u 'admin:admin' "https://localhost:9201/_plugins/_replication/remote-index/_status?pretty"
{
  "status" : "REPLICATION NOT IN PROGRESS"
}
88665a0f606e:2.1 ankikala$ curl -k -u 'admin:admin' -X POST "https://localhost:9201/remote-index/_doc/?pretty" -H 'Content-Type: application/json' -d'
> {
>     "user" : "kimchy3",
>     "post_date" : "2009-11-15T14:12:12",
>     "message" : "trying out Elasticsearch5"
> }
> '
{
  "_index" : "remote-index",
  "_id" : "VIMPyoEB5BmKdIODb8ma",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 2,
  "_primary_term" : 4
}

[gaiksaya]

Hi @ankitkala ,

We found below issues tagged with v2.1.0 that were not closed or resolved in this release. Can you take a look and close the ones that are resolved or retag them with next version you see fit?
Thanks!

#389  [BUG] Replication status API response should never be RUNNING            bug, v2.1.0          about 1 month ago
#387  Check for performance regression on additional metrics                   enhancement, v2.1.0  about 2 months ago
#361  [BUG] connect_transport_exception - connect_timeout[30s]                 bug, v2.1.0          about 2 months ago
#351  [BUG] Gracefully handle the bootstrap failures                           bug, v2.1.0          about 2 months ago
#323  [BUG] Add code coverage data with security tests                         bug, v2.1.0          about 2 months ago
#267  [BUG] Avoid excessive logging during certain exception types             bug, v2.1.0          about 2 months ago
#244  Update integTest gradle scripts to run via remote cluster independently  enhancement, v2.1.0  about 1 month ago