Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CD guide for releasing plugins #4341

Merged
merged 8 commits into from
May 21, 2021
Merged
11 changes: 5 additions & 6 deletions content/doc/developer/plugin-development/incrementals.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,11 @@ For more details, please refer to the link:https://github.com/jenkinsci/incremen
== Enabling Incrementals

Inside the (downstream) plugin, which is supposed to be the API consumer, you can enable incrementals by running:
```
mkdir -p .mvn
echo -Pconsume-incrementals >> .mvn/maven.config
```

link:https://github.com/jenkinsci/incrementals-tools[Another way] is to run `mvn incrementals:incrementalify`.
[source,shell]
----
mvn incrementals:incrementalify
----

== Create a Pull Request in Jenkins core

Expand All @@ -53,7 +52,7 @@ You can do this either by rebasing or by merging the latest changes from the mas

== Wait for all checks to pass

Wait for 'continuous-integration/jenkins' to perform checks.
Wait for Jenkins to perform checks.

== Obtain the Incrementals version

Expand Down
2 changes: 2 additions & 0 deletions content/doc/developer/publishing/_chapter.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,7 @@ guides:
#- adopt-a-plugin
#- changing-maintainers
- releasing
- releasing-cd
- releasing-manually
- removing-from-distribution
- releasing-experimental-updates
156 changes: 156 additions & 0 deletions content/doc/developer/publishing/releasing-cd.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
---
title: Setting up automated plugin release
layout: developerguide
references:
- url: /jep/229
title: 'JEP-229: Continuous Delivery of Jenkins Components and Plugins'
---

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
WARNING: Continuous Delivery of Jenkins Components and Plugin (jep:229[]) is currently in preview.
See the referenced Jenkins enhancement proposal and references in it to see the current state.
Any feedback from early adopters will be appreciated.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where is it marked as preview?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not. Should be reworded.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
NOTE: Continuous Delivery of Jenkins Components and Plugin (jep:229[]) is relatively new.
Any feedback from early adopters will be appreciated.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Roadmap-wise, an initiative is in preview until the respective JEP is formally accepted

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/jenkinsci/jep/tree/master/jep/14 does not state that as far as I can tell.

I don't think this discussion is for this pull request.

Nothing I can see anywhere states that this requires a preview disclaimer, I think the current disclaimer is appropriate.

Please take the bits about github actions long term to the mailing list as it doesn't belong in this PR

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FTR the public roadmap had a preview disclaimer until formal publishing and JEP acceptance approval by the Jenkins Governance Meeting and then by the BDFL delegate. See #3527 for the pull request which removed the disclaimer

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That may be true but does not appear to be required?

NOTE: Continuous Delivery of Jenkins Components and Plugin (jep:229[]) is relatively new.
Any feedback from early adopters will be appreciated.
== Prerequisites

. Plugin must be 'incrementalized', see link:../../plugin-development/incrementals[incrementals]. A new plugin created from the link:https://github.com/jenkinsci/archetypes/[archetypes] will already have this setup.
. Enabled the continuous delivery flag in link:https://github.com/jenkins-infra/repository-permissions-updater/[repository permission updater] (RPU) for your plugin.


[source,yaml]
----
cd:
enabled: true
----

Once that has been merged, start checking https://github.com/jenkinsci/your-plugin/settings/secrets/actions and you should soon see `MAVEN_TOKEN` and `MAVEN_USERNAME` appear under Repository secrets.

== Release notes

Next, if you already have Release Drafter configured, remove any tag-template override in `.github/release-drafter.yml`, and delete `.github/workflows/release-drafter.yml` if using GitHub Actions (or remove the app from the repo otherwise). If you have not yet configured Release Drafter, just create `.github/release-drafter.yml` containing only:

[source,yaml]
----
_extends: .github
----

== Adding the GitHub action

Create `.github/workflows/cd.yaml` as a copy of the link:https://github.com/jenkinsci/.github/blob/master/workflow-templates/cd.yaml[cd.yaml template].

If this is your first time using a GitHub action on this project then navigate to the 'Actions' tab
in your GitHub repository and enable GitHub actions.
Comment on lines +38 to +39
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this actually necessary? I thought it sufficed to just commit the right files to the default branch.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pretty sure recently I had to do this. Happy to be proved wrong.


If you have a `.github/dependabot.yml`, it is a good idea to add:

[source,yaml]
----
- package-ecosystem: github-actions
directory: /
schedule:
interval: daily
----

== POM file modifications

For a regular component whose version number is not that meaningful:
timja marked this conversation as resolved.
Show resolved Hide resolved

[source,diff]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should create a new goal like incrementalify (or option to that goal) to do this sort of edit. And/or a flag in archetypes?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes that would be great, I was thinking about this at least in context of the IRC bot, is there any reason not to automatically opt new plugins in to the cd.enabled flag? i.e. don't bother adding a flag to Jira in the hosting issue.

Also do we need a new flag in archetypes or can we assume plugins use the is hosted on jenkinsci flag should use this setup?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there any reason not to automatically opt new plugins in to the cd.enabled flag?

That would be a question for @daniel-beck probably.

can we assume plugins us[ing] the is hosted on jenkinsci flag should use this setup?

We should probably get some more diverse adoption before making this the default for new plugins.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there any reason not to automatically opt new plugins in to the cd.enabled flag?

Assuming this is asking "why not set this even if unused":

  • This effectively changes the permission model; anyone with commit access to the repo can deploy (even if jumping through hoops to get the token). Right now folks can add whoever they want to GH as committers and it doesn't affect the bits we distribute; cd.enabled changes this.
  • Every repo adds about 1-2 seconds to the execution time of the RPU run.

----
--- a/.mvn/maven.config
+++ b/.mvn/maven.config
@@ -1,2 +1,3 @@
-Pconsume-incrementals
-Pmight-produce-incrementals
+-Dchangelist.format=%d.v%s
--- a/pom.xml
+++ b/pom.xml
@@ -7,7 +7,7 @@
- <version>${revision}${changelist}</version>
+ <version>${changelist}</version>
<packaging>hpi</packaging>
@@ -26,8 +26,7 @@
<properties>
- <revision>1.23</revision>
- <changelist>-SNAPSHOT</changelist>
+ <changelist>999999-SNAPSHOT</changelist>
<jenkins.version>2.176.4</jenkins.version>
</properties>
----

Here a CI/release build (`-Dset.changelist` specified) will be of the form `123.vabcdef456789`. A snapshot build will be `999999-SNAPSHOT`: arbitrary but treated as a snapshot by Maven and newer than any release.
timja marked this conversation as resolved.
Show resolved Hide resolved

It's worth communicating this to your users, as they will see a very different version number format than before.
The best way to do this is to add a line to the release notes: link:https://github.com/jenkinsci/azure-artifact-manager-plugin/releases/tag/86.va2aa4b1038c7[example note].

For a component whose version number ought to reflect a release version of some wrapped component:

[source,diff]
----
--- a/.mvn/maven.config
+++ b/.mvn/maven.config
@@ -1,2 +1,3 @@
-Pconsume-incrementals
-Pmight-produce-incrementals
+-Dchangelist.format=%d.v%s
--- a/pom.xml
+++ b/pom.xml
@@ -10,12 +10,12 @@
<artifactId>some-library-wrapper</artifactId>
- <version>${revision}${changelist}</version>
+ <version>${revision}-${changelist}</version>
<packaging>hpi</packaging>
<properties>
- <revision>4.0.0-1.3</revision>
- <changelist>-SNAPSHOT</changelist>
+ <revision>4.0.0</revision>
+ <changelist>999999-SNAPSHOT</changelist>
<jenkins.version>2.176.4</jenkins.version>
----

Here the version numbers will look like `4.0.0-123.vabcdef456789` or `4.0.0-999999-SNAPSHOT`, respectively.
When you pick up a new third-party component like `4.0.1`, your version numbers will match;
to refer to the third-party component, just use:

[source,xml]
----
<version>${revision}</version>
----

== Releasing

Now whenever Jenkins reports a successful build of your default branch,
and at least one pull request had a label indicating it was of interest to users
(e.g. enhancement rather than chore), your component will be released to Artifactory and
release notes published in GitHub.
You do not need any special credentials or local checkout; just merge pull requests with suitable titles and labels.

You can also trigger a deployment explicitly, if the current commit has a passing check from Jenkins. Visit https://github.com/jenkinsci/your-plugin/actions?query=workflow%3Acd and click Run workflow.
If you prefer to only deploy explicitly, not on every push, just comment out the check_run section in the workflow.

== Fallback

You can also release manually if you have configured your machine for link:../releasing-manually[manual release].
To cut a release:

[source,shell]
----
git checkout master
git pull --ff-only
mvn -Dset.changelist \
-DaltDeploymentRepository=maven.jenkins-ci.org::default::https://repo.jenkins-ci.org/releases/ \
clean deploy
----

== Troubleshooting

Check that `MAVEN_TOKEN` and `MAVEN_USERNAME` appear under Repository secrets.

=== The upload to the Maven repository fails with "401 Unauthorized"

Unauthorized means that the credentials were invalid, or not sent by Maven.

This normally means that the secrets configured in the repository have expired, create an issue in the INFRA project on link:https://issues.jenkins.io/[Jira], and let the team know in #jenkins-infra on link:https://freenode.net[Freenode].

Alternatively you can temporarily update the secrets yourself with your own personal credentials.

=== Further troubleshooting help

If none of the provided solutions help, send an email to the link:/mailing-lists[Jenkins developers mailing list] and explain what you did, and how it failed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or better to file an INFRA ticket?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well, it'll get sorted a lot quicker in IRC...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I agree that it will be faster to ask for help on IRC, creating a ticket will be useful for documentation purposes.

Copy link
Member Author

@timja timja May 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

better? assuming you meant the lines above, or was i mis-reading =/

issues with releasing have normally gone to the developer mailing list, (unless it's obviously an infra issue)

84 changes: 84 additions & 0 deletions content/doc/developer/publishing/releasing-manually.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
---
title: Performing a Plugin Release manually
layout: developerguide
---

[IMPORTANT]
.Prefer automated releases instead of releasing manually
====
See link:../releasing-cd[setting up automated plugin release] instead of this guide.
====

== Prerequisites

. Make sure you have permissions to release the plugin. link:../requesting-hosting/[See this guide to learn more details]

== Artifactory Credentials for Maven

You will need to tell Maven your credentials to access link:../artifact-repository[Artifactory].
Log in to Artifactory, and from your profile, obtain the _encrypted password_.

Create the file `~/.m2/settings.xml` (`~` representing your user home directory, e.g. `/home/yourname` or `C:\Users\yourname`) if needed, and make sure it contains the `<servers>` element as shown below:

----
<settings xmlns="https://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://maven.apache.org/SETTINGS/1.0.0
https://maven.apache.org/xsd/settings-1.0.0.xsd">

<servers>
<server>
<id>maven.jenkins-ci.org</id> // <1>
<username>your_user_name_here</username>
<password>your_encrypted_password_here</password>
</server>
</servers>

</settings>
----
<1> This is not a valid host name anymore, but still the ID used by default in the Jenkins plugin parent POM.
You may need to add additional `<server>` entries if your plugin POM overrides the inherited Maven `<distributionManagement>`, but this should be rare.

== Set up GitHub to accept your SSH key

Maven Release Plugin will automatically push to the repository when performing a release, so you need to link:https://help.github.com/articles/adding-a-new-ssh-key-to-your-github-account/[set up GitHub to accept your SSH key].

See the link:https://help.github.com/articles/connecting-to-github-with-ssh/[GitHub help on SSH] for more information.

== Performing the release

With GitHub and Maven credentials set up, performing a release should be as easy as running the following command:

----
mvn release:prepare release:perform
----

// Not sure about this:
// NOTE: While it is be possible to specify the username and password on the command line, that would require your accounts on GitHub and the Jenkins community to match, and prevent you from using two factor authentication on GitHub.
// Neither is a recommend practice.

== Troubleshooting

First, make sure your plugin uses a reasonably up to date link:../../plugin-development/updating-parent[parent POM].
This will prevent the vast majority of problems in releasing the plugin, such as outdated Maven plugins, or obsolete host names.

=== The upload to the Maven repository fails with "401 Unauthorized"

Unauthorized means that your credentials were invalid, or not sent by Maven.

Make sure you've updated your encrypted password since the last time you changed your password on link:https://accounts.jenkins.io[the account app].

=== The upload to the Maven repository fails with "403 Forbidden"

The two most common explanations for this error:

* You don't have permission to upload to the specified path.
link:../requesting-hosting/#request-upload-permissions[Learn more about how to request upload permissions].
Check that the path you're allowed to upload to matches the actual upload attempt (i.e. no typos).
* The specified release already exists and you try to overwrite it.
We do not allow replacing existing releases.
Specify a different, previously unused version number during the release process.

=== Further troubleshooting help

If none of the provided solutions help, send an email to the link:/mailing-lists[Jenkins developers mailing list] and explain what you did, and how it failed.
86 changes: 7 additions & 79 deletions content/doc/developer/publishing/releasing.adoc
Original file line number Diff line number Diff line change
@@ -1,84 +1,12 @@
---
title: Performing a Plugin Release
layout: developerguide
references:
- url: ../releasing-cd
title: Through GitHub
- url: ../releasing-manually
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would keep manual releases here, but no strong opinion

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you mean ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I mean is keeping the manual release guide content here until JEP-229 is accepted. I am not blocking the PR by this comment

title: Manually on your machine
---

== Prerequisites

. Make sure you have permissions to release the plugin. link:../requesting-hosting/[See this guide to learn more details]

== Artifactory Credentials for Maven

You will need to tell Maven your credentials to access link:../artifact-repository[Artifactory].
Log in to Artifactory, and from your profile, obtain the _encrypted password_.

Create the file `~/.m2/settings.xml` (`~` representing your user home directory, e.g. `/home/yourname` or `C:\Users\yourname`) if needed, and make sure it contains the `<servers>` element as shown below:

----
<settings xmlns="https://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://maven.apache.org/SETTINGS/1.0.0
https://maven.apache.org/xsd/settings-1.0.0.xsd">

<servers>
<server>
<id>maven.jenkins-ci.org</id> // <1>
<username>your_user_name_here</username>
<password>your_encrypted_password_here</password>
</server>
</servers>

</settings>
----
<1> This is not a valid host name anymore, but still the ID used by default in the Jenkins plugin parent POM.
You may need to add additional `<server>` entries if your plugin POM overrides the inherited Maven `<distributionManagement>`, but this should be rare.

== Set up GitHub to accept your SSH key

Maven Release Plugin will automatically push to the repository when performing a release, so you need to link:https://help.github.com/articles/adding-a-new-ssh-key-to-your-github-account/[set up GitHub to accept your SSH key].

See the link:https://help.github.com/articles/connecting-to-github-with-ssh/[GitHub help on SSH] for more information.

== Performing the release

With GitHub and Maven credentials set up, performing a release should be as easy as running the following command:

----
mvn release:prepare release:perform
----

// Not sure about this:
// NOTE: While it is be possible to specify the username and password on the command line, that would require your accounts on GitHub and the Jenkins community to match, and prevent you from using two factor authentication on GitHub.
// Neither is a recommend practice.

== Troubleshooting

First, make sure your plugin uses a reasonably up to date link:../../plugin-development/updating-parent[parent POM].
This will prevent the vast majority of problems in releasing the plugin, such as outdated Maven plugins, or obsolete host names.

=== The upload to the Maven repository fails with "401 Unauthorized"

Unauthorized means that your credentials were invalid, or not sent by Maven.

Make sure you've updated your encrypted password since the last time you changed your password on link:https://accounts.jenkins.io[the account app].

=== The upload to the Maven repository fails with "403 Forbidden"

The two most common explanations for this error:

* You don't have permission to upload to the specified path.
link:../requesting-hosting/#request-upload-permissions[Learn more about how to request upload permissions].
Check that the path you're allowed to upload to matches the actual upload attempt (i.e. no typos).
* The specified release already exists and you try to overwrite it.
We do not allow replacing existing releases.
Specify a different, previously unused version number during the release process.

=== Further troubleshooting help

If none of the provided solutions help, send an email to the link:/mailing-lists[Jenkins developers mailing list] and explain what you did, and how it failed.

== Releasing from CI

Alternately, you can configure your plugin to be released automatically whenever interesting changes are pushed, or on demand.
In this case you only need write permission to GitHub; you never need local credentials.
Follow link:/redirect/continuous-delivery-of-plugins[this guide] for details.
Historically Jenkins plugins were released on developer machines.
We now recommend plugins be released via GitHub, either automatically or with a manual trigger.
2 changes: 1 addition & 1 deletion content/redirect/continuous-delivery-of-plugins.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
---
layout: redirect
redirect_url: https://github.com/jenkinsci/incrementals-tools/blob/master/README.md#superseding-maven-releases
timja marked this conversation as resolved.
Show resolved Hide resolved
redirect_url: /doc/developer/publishing/releasing-cd/
timja marked this conversation as resolved.
Show resolved Hide resolved
---