[PROPOSAL] deprecate the changelog file

[seraphjiang]

What/Why

Today, there are OpenSearch repo and OpenSearch-Dashboards repo maintain both release-notes as well as change log. Although there is some benefit to clear change log history. There are also down side 1. It is a bit redundant with release note. 2. It is hard to determine which change should add to change log, in practices, developer is requested to add changelog for every PRs. Because changelog file is single file cross all branches. It leads to unnecessary merge conflict hell when more than 1 developer raise PR and try to merge and backport at same time.

Both repos mentioned, the changelog was inspired by keep a changelog However, I was not able to find what are other top open source project adopt it and how it be adopted.

We should review and explore what's the real benefit we gain from changelog, be open to explore other large open source project to see if what's the best practices.

What are you proposing?

1. OpenSearch developer and OpenSearch Dashboards review the benefits and pain points of changelog
2. Research other large open source projects' best practices e.g. https://github.com/torvalds/linux, https://github.com/apache/lucene
3. consider deprecate changelog or find better way to resolve merge conflict hell

[dblock]

CHANGELOG was proposed and added in #1868, I'll add a comment about this there.

[dblock]

The two downsides mentioned are 1) merge hell, and 2) big projects don't do this. The first is a good argument and I believe @kotwanikunal has been trying to resolve it with automation, and that's one of the reasons why #1868 remains open. On the second argument, please research how other projects do it and make concrete proposals.

Authoring release notes before release has been a much bigger pain than merge hell, and those notes typically contain a ton of poorly written or incorrect changes (see #1886), so I am personally not inclined to reverse course on introducing CHANGELOG.

[rursprung]

the linux kernel uses the commit history as its changelog (see example).

this works, because they're extremely strict on their commit messages (which roughly follow these guidelines see e.g. this comment from Linus in another discussion). this requires good quality from the commiters and even better oversight from the reviewers. while i would like to see the commit messages of OpenSearch (and really, any project) to have this quality, i don't necessarily think that this is the best form for a changelog: there are lots of "internal" commits (technical things, refactorings, etc.) which are not relevant for users of the software. i see a dedicated changelog for these as something beneficial and thus would also vote in favour of keeping the changelog for now.

[dblock]

For consolidation purposes I am going to close this. Please put your comments in #1868 that is still open and the jury is out of what the "final state" should be.

[seraphjiang]

For consolidation purposes I am going to close this. Please put your comments in #1868 that is still open and the jury is out of what the "final state" should be.

my proposal is not only deprecating changelog in opensearch repo, but deprecate in all repo. i think we should keep this in meta repo to discuss. not moving to specific project discussion and close it.

[kavilla]

my proposal is not only deprecating changelog in opensearch repo, but deprecate in all repo. i think we should keep this in meta repo to discuss. not moving to specific project discussion and close it.

I think originally when the solution was looked in relied on PRs having correct labelling. But I feel more confident within the project that we utilize labelling that perhaps we can rely on this solution? Since we have grown as a project.

But reading the thread as well (sorry if I bring up something already addressed) the ideal state is that a changelog isnt a 1-to-1 to the commit history. Like if a feature came out in 2.x then I don't find much value on changelogs about any fast follows if it was unreleased before. I think part of the problems associated with a changelog and required per a commit is that it doesnt work well with feature development when multiple contributors are working on the same feature in a short amount of time. There is an assumption here that seems to prefer features to be almost in its final state prior to merge rather than iterating. With this insight maybe the changelog check can be relaxed when we know the files that have been change have not yet been relaxed (ie don't run the changelog check on data sources plugin in the OpenSearch Dashboards repo) then re-add it once it gets released.

But also want to call out perhaps it is a good thing that this conflict occurs if this becomes a huge pain during the times the community is iterating fast because it forces developers to rebase and verify when the code is changing so frequently on backports. Although it does defeat the purpose of automation.

The only other thing that I wanted to mention that backport PRs can be retriggered by adding and re adding the label. Not sure if we have to manually delete the branch to avoid branch name conflicts but to avoid the manual checkout can we just retrigger the backport.