[Retrospective] Release 1.3.0

[zelinh]

How to use this issue?

Please add comments to this issue, they can be small or large in scope. Honest feedback is important to improve our processes, suggestions are also welcomed but not required.

What will happen to this issue post release?

There will be a discussion(s) about how the release went and how the next release can be improved. Then this ticket will be updated with the notes of that discussion along side action items.

[evs-xsarus]

I was waiting on the release of 1.2.5 as I believed it had the fix for running the docker using docker-compose based on https://opensearch.org/docs/latest/opensearch/install/docker/#sample-docker-compose-file-for-development .

After testing the 1.3.0 release, the problem still exists:

ERROR: for opensearch-node1 Cannot start service opensearch-node1: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "./opensearch-docker-entrypoint.sh": stat ./opensearch-docker-entrypoint.sh: no such file or directory: unknown

as described in:

• Fix issue with docker volume-mounted config file #1130
• Starting OpenSearch Docker containers fails when using a custom config OpenSearch#768

Leaving this out, makes it hard to start development using OpenSearch. It's a missed chance to apply this change to this latest release.

[dblock]

We committed high risk changes late, for example opensearch-project/asynchronous-search#104 came in almost at the very end and turned out to be fairly non-trivial.

[bbarani]

We should disable the auto-build process once the release candidate is finalized to avoid last minute changes getting in to the release candidates (Ex: opensearch-project/index-management#311)

[peterzhuamazon]

I was waiting on the release of 1.2.5 as I believed it had the fix for running the docker using docker-compose based on https://opensearch.org/docs/latest/opensearch/install/docker/#sample-docker-compose-file-for-development .

After testing the 1.3.0 release, the problem still exists:

ERROR: for opensearch-node1 Cannot start service opensearch-node1: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "./opensearch-docker-entrypoint.sh": stat ./opensearch-docker-entrypoint.sh: no such file or directory: unknown

as described in:

* [Fix issue with docker volume-mounted config file #1130](https://github.com/opensearch-project/opensearch-build/pull/1130)

* [Starting OpenSearch Docker containers fails when using a custom config OpenSearch#768](https://github.com/opensearch-project/OpenSearch/issues/768)

Leaving this out, makes it hard to start development using OpenSearch. It's a missed chance to apply this change to this latest release.

Hi @evs-xsarus

Only come to know this DEV compose file today.
I have opened an issue with documentation, as I also see errors but not the same:
opensearch-project/documentation-website#464

The one we link on the website with 2 data 1 dashboards runs well:
https://opensearch.org/samples/docker-compose.yml

Thanks.

[dblock]

We should disable the auto-build process once the release candidate is finalized to avoid last minute changes getting in to the release candidates (Ex: opensearch-project/index-management#311)

Disagree. This optimizes for the happy path that the RC is what we release. Instead of having a release train, it introduces a manual process of disabling builds the moment we produce an RC. It will waste a day re-enabling builds, and merging additional PRs on individual repos if we decide that a release candidate is bad. The docker build overwriting previous builds is #1762, but once that's fixed there's no reason not to build a new build and then ignore it because we picked an earlier release candidate to release.

[dreamer-89]

Thanks @zelinh for driving the release.

I see below steps needs to be added in the template for OpenSearch repository. Created improvements request in #1787

• Step for incrementing the main next release version. This caused continuous gradle check failures on PR. Thanks to @reta for quickly fixing this. Fixing bwcVersions and bwc builds OpenSearch#2430
• Step for incrementing the next patch version. Added here: Bump the version to 1.3.1 OpenSearch#2509

[ohltyler]

The component release issues weren't clear on the expectations and led to a lot of last-minute changes. I've included some proposed changes in #1718

[zelinh]

Reflection on OpenSearch Release 1.3.0.

Thanks all contributors for your great effort on authoring code changes, reviewing pull requests, dedications on issues so that we could drive a successful 1.3.0 OpenSearch release.
Special kudos for all our automation processes that helped us drive the fastest release launch call ever.

Area for improvements

• We created public GitHub release issues on all release plugins repositories. It took quite some time for few repos to assign the release owners and start work on the release tasks.
• GitHub issue label for 1.3.0 missed to label release issues on some of the plugin repos. We asked teams to triage and label all related release issues with this label. Without that, it's difficult for us to keep track of those issues especially to the current version release.
• Preparation for the release notes was not included in the proper component issue stage which caused delay for collecting them. Everyone feel free to draft PR for our plugin release template here so that we could provide better release experience in the future.
• Last minutes code commit changes after cutoff time pushed the docker build overwriting the release candidates. We had to disable our CI on 1.3.0 to stop accepting any further changes.
• Campaign issues were only included in the overall release issue in build repo; it's hard for plugin teams to quickly refer to and work on the certain issue on their repositories.

Action Items

• OpenSearch release template improvements #1787
• [TESTS] Do not log status code in test result if PASS #1767
• [Enhancement] Add extra docker tags specifically for different builds #1762
• Improve component release issues #1718
• [BUG] Direct self-reference leading to cycle OpenSearch#1891
• Anomaly Detection: Adding Validation API "Non-Blocker" Checks  anomaly-detection#265
• Add backwards compatibility tests for automation job-scheduler#53
• Update integTest gradle scripts to run via remote cluster independently cross-cluster-replication#244
• CI Workflows can't run with JDK 8 or 17 observability#520

With this we are closing out the retro items for 1.3.0.