Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify integration with sbt-release #187

Merged
merged 2 commits into from
Dec 5, 2023

Conversation

julienrf
Copy link
Collaborator

@julienrf julienrf commented Nov 30, 2023

There are still a couple of issues to address, IMHO:

  • Support escape hatches to manually supply the version number, the compatibility level, or a version qualifier (e.g., -RC1).
    I introduced a parameter qualifier that can be used to provide an optional version qualifier. I am not sure we should introduce other escape hatches: if someone wants to use a specific version or compatibility intention different from the one that is computed by default, then they would write something like releaseVersion := _ => "1.2.3", or run sbt release with-defaults release-version 1.2.3, or releaseVersion := ReleaseVersion.fromCompatibility(Compatibility.None).
  • The current PR makes a strong assumption that the workflow of the user will be the common one where sbt-release manages the version setting in a separate file version.sbt. So it assumes that the version has always been set in a specific manner (patch version has been incremented).
    I think these assumptions are reasonable.
  • Not enough tests in the PR
    I have amended my commit with more tests and documentation.

@julienrf julienrf marked this pull request as ready for review November 30, 2023 13:59
@julienrf
Copy link
Collaborator Author

One limitation is that the mode “compute the release version from the level of compatibility with the last release” does not work for the first release. This means you would have to publish a first release in a way, and then update the release workflow to publish the next releases. Ideally, it would be great to define the release workflow once and also handle the first release with it.

julienrf added a commit to julienrf/etag-caching that referenced this pull request Nov 30, 2023
- Fallback to the default releaseVersion behavior for the first release
- Read the version qualifier from an environment variable
@julienrf
Copy link
Collaborator Author

I added a commit that

  • resolves the “first release” issue
  • reads the release version qualifier from an environment variable VERSION_POLICY_RELEASE_QUALIFIER

I am now happy with the content of the PR, could you please have a look @rtyley?

Copy link
Contributor

@rtyley rtyley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great - very easy for sbt-release users to get this new 'compatibility-appropriate version number' functionality from sbt-version-policy, without sbt-version-policy having to depend on sbt-release! 💯

I also like the term 'unconstrained compatibility' for the general new mode of operation we're adding!

Comment on lines +46 to +53
// If there are no previous versions to assess the compatibility with,
// fallback to the default release version function, which drops the qualifier
// from the version set in the file `version.sbt`
// (e.g., "1.0.0-SNAPSHOT" => "1.0.0")
(version: String) =>
Version(version)
.map(_.withoutQualifier.string + qualifier)
.getOrElse(Version.formatError(version))
Copy link
Contributor

@rtyley rtyley Dec 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah clever, I like it! In order to work out the new version number, if there's no previously released version to compare compatibility with, we just read the current snapshot version specified in version.sbt (eg "1.0.0-SNAPSHOT") and use it without the qualifier (eg "1.0.0") 👍

@rtyley
Copy link
Contributor

rtyley commented Dec 1, 2023

Just because they're the maintainer of sbt-release, and this PR might be interesting to them, I'm just going to @-mention @xuwei-k here (apologies, @xuwei-k, if this is not of interest, feel free to ignore!).

This PR is adding support for a new custom versioning strategy that determines the new version number based on sbt-version-policy's assessment of the binary/source compatibility of the new release with the previous release, using the recommended Scala semver rules.

An example of how this might used with sbt-release in a project is in guardian/etag-caching#21, here - it doesn't necessitate either sbt-release or sbt-version-policy having a dependency on each other (because the exposed interface is really just String => String for the version number), though we did copy & paste a small amount of code from sbt-release (Version.scala - both projects are Apache V2) to parse version-numbers and calculate a version string with the appropriate bump.

import sbtversionpolicy.withsbtrelease.ReleaseVersion

...

releaseVersion := {
   ReleaseVersion.fromAggregatedAssessedCompatibilityWithLatestRelease().value
}

We haven't used this code in earnest yet, but should have something usable by sometime next week!

@julienrf julienrf merged commit 1335e94 into scalacenter:main Dec 5, 2023
2 checks passed
@julienrf julienrf deleted the with-sbt-release branch December 5, 2023 09:10
rtyley pushed a commit to guardian/etag-caching that referenced this pull request Dec 5, 2023
rtyley added a commit to guardian/play-secret-rotation that referenced this pull request Dec 5, 2023
This is the second library, after https://github.com/guardian/etag-caching, to adopt
https://github.com/guardian/gha-scala-library-release-workflow/blob/main/README.md .

The changes required to adopt the automated workflow:

* No need to set `sonatypeProfileName` or `ThisBuild / publishTo`,
  that's now done by the workflow.
* Remove the sign, publish, release & push steps of sbt-release's `releaseProcess`
  configuration, because the workflow does those now, and the workflow only
  wants `sbt release` to create the versioning commits, and tag them appropriately.
  The workflow does the rest itself.
* Remove `sbt-pgp` plugin because it's no longer used - the workflow does the signing
  using `gpg` directly.

Additionally, we add **automatic version numbering** based on compatibility assessment
performed by `sbt-version-policy`:

* Add the `sbt-version-policy` plugin, to allow it to do the compatibility assessment.
  This also sets the `versionScheme` for this library to `early-semver`, which is
  the recommended versioning for Scala libraries, and `sbt-version-policy` and
  correct sbt-eviction-issue-detection pretty much depend on the `versionScheme`
  being `early-semver`. Thus we also need to switch to using a new semver version
  number for our library version!
* Add the `releaseVersion := fromAggregatedAssessedCompatibilityWithLatestRelease().value`
  sbt setting, which will intelligently set the release version based on `sbt-version-policy`'s
  compatibility assessment, thanks to scalacenter/sbt-version-policy#187 .
* Use `publish / skip := true`, rather than other hacks like `publish := {}` or
  `publishArtifact := false`, to tell sbt not to publish modules that we don't want published
  (typically, the 'root' module) - this is important because `sbt-version-policy` won't
  understand those hacks, but _will_ understand `publish / skip := true` means that it doesn't
  need to assess compatibility there.
rtyley added a commit to guardian/play-secret-rotation that referenced this pull request Dec 5, 2023
This is the second library, after https://github.com/guardian/etag-caching, to adopt
https://github.com/guardian/gha-scala-library-release-workflow/blob/main/README.md .

The changes required to adopt the automated workflow:

* No need to set `sonatypeProfileName` or `ThisBuild / publishTo`,
  that's now done by the workflow.
* Remove the sign, publish, release & push steps of sbt-release's `releaseProcess`
  configuration, because the workflow does those now, and the workflow only
  wants `sbt release` to create the versioning commits, and tag them appropriately.
  The workflow does the rest itself.
* Remove `sbt-pgp` plugin because it's no longer used - the workflow does the signing
  using `gpg` directly.

Additionally, we add **automatic version numbering** based on compatibility assessment
performed by `sbt-version-policy`:

* Add the `sbt-version-policy` plugin, to allow it to do the compatibility assessment.
  This also sets the `versionScheme` for this library to `early-semver`, which is
  the recommended versioning for Scala libraries, and `sbt-version-policy` and
  correct sbt-eviction-issue-detection pretty much depend on the `versionScheme`
  being `early-semver`. Thus we also need to switch to using a new semver version
  number for our library version!
* Add the `releaseVersion := fromAggregatedAssessedCompatibilityWithLatestRelease().value`
  sbt setting, which will intelligently set the release version based on `sbt-version-policy`'s
  compatibility assessment, thanks to scalacenter/sbt-version-policy#187 .
* Use `publish / skip := true`, rather than other hacks like `publish := {}` or
  `publishArtifact := false`, to tell sbt not to publish modules that we don't want published
  (typically, the 'root' module) - this is important because `sbt-version-policy` won't
  understand those hacks, but _will_ understand `publish / skip := true` means that it doesn't
  need to assess compatibility there.
rtyley added a commit to guardian/play-secret-rotation that referenced this pull request Dec 5, 2023
This is the second library, after https://github.com/guardian/etag-caching, to adopt
https://github.com/guardian/gha-scala-library-release-workflow/blob/main/README.md .

The changes required to adopt the automated workflow:

* No need to set `sonatypeProfileName` or `ThisBuild / publishTo`,
  that's now done by the workflow.
* Remove the sign, publish, release & push steps of sbt-release's `releaseProcess`
  configuration, because the workflow does those now, and the workflow only
  wants `sbt release` to create the versioning commits, and tag them appropriately.
  The workflow does the rest itself.
* Remove `sbt-pgp` plugin because it's no longer used - the workflow does the signing
  using `gpg` directly.

Additionally, we add **automatic version numbering** based on compatibility assessment
performed by `sbt-version-policy`:

* Add the `sbt-version-policy` plugin, to allow it to do the compatibility assessment.
  This also sets the `versionScheme` for this library to `early-semver`, which is
  the recommended versioning for Scala libraries, and `sbt-version-policy` and
  correct sbt-eviction-issue-detection pretty much depend on the `versionScheme`
  being `early-semver`. Thus we also need to switch to using a new semver version
  number for our library version!
* Add the `releaseVersion := fromAggregatedAssessedCompatibilityWithLatestRelease().value`
  sbt setting, which will intelligently set the release version based on `sbt-version-policy`'s
  compatibility assessment, thanks to scalacenter/sbt-version-policy#187 .
* Use `publish / skip := true`, rather than other hacks like `publish := {}` or
  `publishArtifact := false`, to tell sbt not to publish modules that we don't want published
  (typically, the 'root' module) - this is important because `sbt-version-policy` won't
  understand those hacks, but _will_ understand `publish / skip := true` means that it doesn't
  need to assess compatibility there.
rtyley added a commit to guardian/play-secret-rotation that referenced this pull request Dec 5, 2023
This is the second library, after https://github.com/guardian/etag-caching, to adopt
https://github.com/guardian/gha-scala-library-release-workflow/blob/main/README.md .

The changes required to adopt the automated workflow:

* No need to set `sonatypeProfileName` or `ThisBuild / publishTo`,
  that's now done by the workflow.
* Remove the sign, publish, release & push steps of sbt-release's `releaseProcess`
  configuration, because the workflow does those now, and the workflow only
  wants `sbt release` to create the versioning commits, and tag them appropriately.
  The workflow does the rest itself.
* Remove `sbt-pgp` plugin because it's no longer used - the workflow does the signing
  using `gpg` directly.

Additionally, we add **automatic version numbering** based on compatibility assessment
performed by `sbt-version-policy`:

* Add the `sbt-version-policy` plugin, to allow it to do the compatibility assessment.
  This also sets the `versionScheme` for this library to `early-semver`, which is
  the recommended versioning for Scala libraries, and `sbt-version-policy` and
  correct sbt-eviction-issue-detection pretty much depend on the `versionScheme`
  being `early-semver`. Thus we also need to switch to using a new semver version
  number for our library version!
* Add the `releaseVersion := fromAggregatedAssessedCompatibilityWithLatestRelease().value`
  sbt setting, which will intelligently set the release version based on `sbt-version-policy`'s
  compatibility assessment, thanks to scalacenter/sbt-version-policy#187 .
* Use `publish / skip := true`, rather than other hacks like `publish := {}` or
  `publishArtifact := false`, to tell sbt not to publish modules that we don't want published
  (typically, the 'root' module) - this is important because `sbt-version-policy` won't
  understand those hacks, but _will_ understand `publish / skip := true` means that it doesn't
  need to assess compatibility there.
rtyley added a commit to guardian/play-secret-rotation that referenced this pull request Dec 6, 2023
This is the second library, after https://github.com/guardian/etag-caching, to adopt
https://github.com/guardian/gha-scala-library-release-workflow/blob/main/README.md .

The changes required to adopt the automated workflow:

* No need to set `sonatypeProfileName` or `ThisBuild / publishTo`,
  that's now done by the workflow.
* Remove the sign, publish, release & push steps of sbt-release's `releaseProcess`
  configuration, because the workflow does those now, and the workflow only
  wants `sbt release` to create the versioning commits, and tag them appropriately.
  The workflow does the rest itself.
* Remove `sbt-pgp` plugin because it's no longer used - the workflow does the signing
  using `gpg` directly.

Additionally, we add **automatic version numbering** based on compatibility assessment
performed by `sbt-version-policy`:

* Add the `sbt-version-policy` plugin, to allow it to do the compatibility assessment.
  This also sets the `versionScheme` for this library to `early-semver`, which is
  the recommended versioning for Scala libraries, and `sbt-version-policy` and
  correct sbt-eviction-issue-detection pretty much depend on the `versionScheme`
  being `early-semver`. Thus we also need to switch to using a new semver version
  number for our library version!
* Add the `releaseVersion := fromAggregatedAssessedCompatibilityWithLatestRelease().value`
  sbt setting, which will intelligently set the release version based on `sbt-version-policy`'s
  compatibility assessment, thanks to scalacenter/sbt-version-policy#187 .
* Use `publish / skip := true`, rather than other hacks like `publish := {}` or
  `publishArtifact := false`, to tell sbt not to publish modules that we don't want published
  (typically, the 'root' module) - this is important because `sbt-version-policy` won't
  understand those hacks, but _will_ understand `publish / skip := true` means that it doesn't
  need to assess compatibility there.
rtyley added a commit to guardian/play-secret-rotation that referenced this pull request Dec 6, 2023
Releasing v1.0.0 with https://github.com/guardian/gha-scala-library-release-workflow
worked fine, but trying to release v1.0.1 failed, due to the unusual configuration
of the `secret-generator` subproject:

https://github.com/guardian/play-secret-rotation/actions/runs/7113763098/job/19366398526#step:4:153

```
[error] (aws-parameterstore-lambda / mimaPreviousClassfiles) sbt.librarymanagement.ResolveException: Error downloading com.gu.play-secret-rotation:secret-generator_2.13:1.0.0
[error]   Not found
[error]   Not found
[error]   not found: /home/runner/.ivy2/localcom.gu.play-secret-rotation/secret-generator_2.13/1.0.0/ivys/ivy.xml
[error]   not found: https://repo1.maven.org/maven2/com/gu/play-secret-rotation/secret-generator_2.13/1.0.0/secret-generator_2.13-1.0.0.pom
[error] (aws-parameterstore-lambda / versionPolicyFindDependencyIssues) sbt.librarymanagement.ResolveException: Error downloading com.gu.play-secret-rotation:secret-generator_2.13:1.0.0
```

Nothing actually needs `secret-generator` as a library artifact - `aws-parameterstore-lambda` uses `sbt-assembly` to incorporate all it's dependencies into a single jar, so even though it uses secret-generator, it doesn't need it as a separate artifact. However, the new compatibility-based version-numbering functionality introduced in scalacenter/sbt-version-policy#187 wants to compare with previous versions of project dependencies, and it doesn't understand the dependency isn't published...

It's easier to publish the artifact, so I'm doing that here!
rtyley added a commit to guardian/play-secret-rotation that referenced this pull request Dec 6, 2023
Releasing v1.0.0 with https://github.com/guardian/gha-scala-library-release-workflow
worked fine, but trying to release v1.0.1 failed, due to the unusual configuration
of the `secret-generator` subproject:

https://github.com/guardian/play-secret-rotation/actions/runs/7113763098/job/19366398526#step:4:153

```
[error] (aws-parameterstore-lambda / mimaPreviousClassfiles) sbt.librarymanagement.ResolveException: Error downloading com.gu.play-secret-rotation:secret-generator_2.13:1.0.0
[error]   Not found
[error]   Not found
[error]   not found: /home/runner/.ivy2/localcom.gu.play-secret-rotation/secret-generator_2.13/1.0.0/ivys/ivy.xml
[error]   not found: https://repo1.maven.org/maven2/com/gu/play-secret-rotation/secret-generator_2.13/1.0.0/secret-generator_2.13-1.0.0.pom
[error] (aws-parameterstore-lambda / versionPolicyFindDependencyIssues) sbt.librarymanagement.ResolveException: Error downloading com.gu.play-secret-rotation:secret-generator_2.13:1.0.0
```

Nothing actually needs `secret-generator` as a library artifact - `aws-parameterstore-lambda` uses `sbt-assembly` to incorporate all it's dependencies into a single jar, so even though it uses secret-generator, it doesn't need it as a separate artifact. However, the new compatibility-based version-numbering functionality introduced in scalacenter/sbt-version-policy#187 wants to compare with previous versions of project dependencies, and it doesn't understand the dependency isn't published...

It's easier to publish the artifact, so I'm doing that here!
rtyley added a commit to guardian/facia-scala-client that referenced this pull request Dec 13, 2023
This replaces the old release process which had developers manually running
`sbt release` on their own laptops - each developer had to obtain their own
PGP key and Sonatype credentials, which was an elaborate & fiddly process.

Now there's a single set of release credentials, available through GitHub
Organisation Secrets, like we already have with NPM.

### Required changes

The changes required to adopt the automated workflow:

* No need to set these sbt configuration keys, as they're now taken
  care of by the workflow:
  * `sonatypeProfileName`
  * `publishTo`
  * `scmInfo` & `pomExtra`
* Remove the sign, publish, release & push steps of sbt-release's
  `releaseProcess` configuration, because the workflow does those now, and
  the workflow only wants `sbt release` to create the versioning commits,
  and tag them appropriately. The workflow does the rest itself.
* Remove `sbt-pgp` plugin because it's no longer used - the workflow does the signing using `gpg` directly.
* Grant this repo access to the GitHub Organisation Secrets containing the Maven Release
  credentials with guardian/github-secret-access#21
* Unusually, drop running the tests as part of the release for now, as the
  tests in this project require special credentials (see #272)

Additionally, we add **automatic version numbering** based on compatibility assessment performed by `sbt-version-policy`:

* Add the `sbt-version-policy` plugin, to allow it to do the compatibility assessment. This also sets the `versionScheme` for this library to `early-semver`, which is the recommended versioning for Scala libraries, and `sbt-version-policy` & correct sbt-eviction-issue-detection pretty much depend on the `versionScheme` being `early-semver`. Thus we also need to switch to using a new semver version number for our library version!
* Add the `releaseVersion := fromAggregatedAssessedCompatibilityWithLatestRelease().value` sbt setting, which will intelligently set the release version based on `sbt-version-policy`'s compatibility assessment, thanks to scalacenter/sbt-version-policy#187 .
* Use `publish / skip := true`, rather than other hacks like `publish := {}` or `publishArtifact := false`, to tell sbt not to publish modules that we don't want published (typically, the 'root' module) - this is important because `sbt-version-policy` won't understand those hacks, but _will_ understand `publish / skip := true` means that it doesn't need to assess compatibility there.

Recent prior example of adding `gha-scala-library-release-workflow` to
a repo: guardian/play-googleauth#208
davidfurey added a commit to guardian/tags-thrift-schema that referenced this pull request Feb 20, 2024
This replaces the old release process which had developers manually running
`sbt release` on their own laptops - each developer had to obtain their own
PGP key and Sonatype credentials, which was an elaborate & fiddly process.

Now there's a single set of release credentials, available through GitHub
Organisation Secrets, like we already have with NPM.

The changes required to adopt the automated workflow:

* No need to set these sbt configuration keys, as they're now taken
  care of by the workflow:
  * `scmInfo` & `pomExtra`
  * `homepage`
  * `developers`
  * `releasePublishArtifactsAction`
  * `publishTo`
* Remove the publish, release & push steps of sbt-release's
  `releaseProcess` configuration, because the workflow does those now, and
  the workflow only wants `sbt release` to create the versioning commits,
  and tag them appropriately. The workflow does the rest itself.
* Remove `sbt-pgp` plugin because it's no longer used - the workflow does the signing using `gpg` directly.
* Grant this repo access to the GitHub Organisation Secrets containing the Maven Release
  credentials with guardian/github-secret-access#21

Additionally, we add **automatic version numbering** based on compatibility assessment performed by `sbt-version-policy`:

* Add the `sbt-version-policy` plugin, to allow it to do the compatibility assessment. This also sets the `versionScheme` for this library to `early-semver`, which is the recommended versioning for Scala libraries, and `sbt-version-policy` & correct sbt-eviction-issue-detection pretty much depend on the `versionScheme` being `early-semver`. Thus we also need to switch to using a new semver version number for our library version!
* Add the `releaseVersion := fromAggregatedAssessedCompatibilityWithLatestRelease().value` sbt setting, which will intelligently set the release version based on `sbt-version-policy`'s compatibility assessment, thanks to scalacenter/sbt-version-policy#187 .

This change also drops support for Scala 2.11 since it does not seem to support the -release:11 scalac option which is required to ensure that the Java 17 workflow produces Java 11 compatible bytecode.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants