RelEng: ReleaseProcess

Table of Contents (Last Generated August 27, 2018)

• Permissions and Accounts Required
  • Access to post to the following Google groups
  • Access to post to the following Ansible Slack channels
  • Access to post to the following Red Hat distribution lists
  • Access to change the channel topic on irc channels
  • Accounts on
  • Credentials for
• During the Development Stage
• Before code freeze
• Starting a new release branch
  • Reporting deprecated features
  • Protecting the stable branches
• Ongoing lead up to release
• Release procedure
  • Release pre-check
  • Make release (2.5)
    • Changelogs for 2.5
  • Make release (2.6+)
  • Builds
  • Publish release (2.5)
  • Publish release (2.6+)
  • Job cleanup
  • Notifications
    • Google Groups (mailing lists)
    • IRC
    • Ansible Slack
    • Twitter
    • Internal email
  • Notification Sample Content
    • Short form (Twitter, IRC, etc)
    • Long form (email, ML, Google Groups, etc)

Permissions and Accounts Required

Access to post to the following Google groups:

• https://groups.google.com/forum/#!forum/ansible-devel
• https://groups.google.com/forum/#!forum/ansible-project
• https://groups.google.com/forum/#!forum/ansible-announce

Access to post to the following Ansible Slack channels:

• #core_internal (private channel)
• #core_network
• #ship_it
• #testing

Access to post to the following Red Hat distribution lists:

• ansible-releases (are there even any restrictions on this?)

Access to change the channel topic on irc channels:

• #ansible

Accounts on:

• Twitter
• IRC

Credentials for:

• ansible user on Launchpad: https://launchpad.net/~ansible/

GitHub team:

• release manager team to push to stable branches.

During the Development Stage

• Add links to all of the items on the roadmap for the release.
  • This lets others see the status of each feature
  • Can link to proposal, PR, issue, project board depending on the feature

Before code freeze

• Check in with developers whether features are merged
• Remove features that are going to be punted from the roadmap for the release.

Starting a new release branch

This is done at code freeze, usually 2 weeks before beta 1 is released.

• Create branch stable-X.Y from devel:
  • git checkout -b stable-X.Y
• Open a PR against devel with the following changes:
  • Delete everything under changelogs/fragments/ so the fragments won't be duplicated in the $(Y+1) release.
  • Create a stub fragment for the new branch as changelogs/fragments/vX.$(Y+1).0-initial-commit.yaml with an empty dict ({}).
  • Create a new blank porting guide:

     hacking/build-ansible.py porting-guide --version X.$(Y+1)
     git add docs/docsite/rst/porting_guides/porting_guide_core_X.$(Y+1).rst
    

  • Add the porting guide to the top of the list in docs/docsite/rst/porting_guides/core_porting_guides.rst.
  • In lib/ansible/release.py:
    • set __version__ to X.$(Y+1).0.dev0.
    • set __codename__ to the selected codename for the next release.
  • Add the new codename to .github/RELEASE_NAMES.yml so that we know that name has already been used.
  • Update changelogs/changelog.yaml setting ancestor to X.Y.0.
  • Verify that a full sanity test run passes without warnings (other than for "base branch"):
    • ansible-test sanity --docker -v --allow-disabled
    • The PR will require temporary ignore entries in test/sanity/ignore.txt for deprecation related sanity test failures.

Reporting Recurring Cleanups

This procedure is to discover cleanups that need to be performed every release and get them assigned to Core Team members for fixing every release. Currently reported cleanups are:

• deprecated features in code
• deprecated features in config
• bundled libraries that need updating

First, make sure an ansible-core X.Y project has been created to track the issues.

Steps to create issues for deprecated-config and update-bundled sanity tests:

• Run hacking/create_deprecation_bug_reports.py to see what issues need to be created.
• If the output looks acceptable, run again with the --create option.

Steps to create issues for pylint reported deprecations:

• Create issues for all scheduled code deprecations. This can be done using the hacking/create_deprecated_issues.py script:
  • Create a file with the ansible-deprecated-version issues discovered by pylint when running ansible-test sanity --test pylint.
  • Execute hacking/build-ansible.py file-deprecation-tickets deprecated.txt --template hacking/deprecated_issue_template.md --project-name 'ansible-core $(X.Y)' with the correct version substituted.

Protecting the stable branches

Our github admins (mattclay, jctanner, jimi-c) can modify branch permissions:

• Protect this branch - Disables force-pushes to this branch and prevents it from being deleted.
• Include administrators - Enforce all configured restrictions for administrators.
• There's a github group, release manager team, that needs to be given access to all branches.

Any branch matching stable-2.[0-9]* is protected so only the release manager team and admins can push to them. If you have problems pushing or the branch naming changes, talk to a github admin to figure out what is wrong.

Ongoing lead up to release

• Review and merge backport PRs against the stable-X.Y branch
  • The github search for prs against a branch is: https://github.com/ansible/ansible/issues?q=is%3Aopen+base%3Astable-X.Y
• If this is an rc-less release (minor releases as opposed to major releases), stop merging backport PRs several days before release (Monday is good last merge day for a release on a Thursday), so that Tower has a chance to test.
• Ping the tower liaison in the #testing slack channel once you're done merging so they can be sure to test before the next rc/final release.

Release procedure

Release pre-check

• Check status of Jenkins Tower Compatibility Test (VPN) for target branch for "green" (talk w/ tower QE point person if not)
  • This doesn't block the alpha and beta releases because we're working off devel which is a moving target but will block rcs. Check it at alpha and beta and let the Tower QE team know that something is failing so that it can be fixed before rc.
  • If the Test hasn't run since the last commit, you will need to run the Jenkins Stable Ansible Test (VPN) with GIT_TESTED_BRANCHES set to origin/stable-X.Y (and GIT_REFSPEC just left to its default value). This will kick off Ansible builds, Tower Installs, and the Tower compatibility tests. The test pipeline takes about 1.5 hours to complete. The final results will be available at Jenkins Tower Compatibility Test (VPN).
  • Tower is in the process of deprecating this test in favour of a different one. So this test will almost always be red. For now, you need to ping them every release to ask them if the branch is green. It's best to do this on Monday, after you've merged all of the backports for this release.
• Check https://app.shippable.com/github/ansible/ansible/runs?branchName=stable-X.Y for "Success" (devel for the alpha release)
• Ensure no open issues/PRs tagged tower_blocker, https://github.com/ansible/ansible/labels/tower_blocker , or coordinate with tower QE point person and others on Tower
• Ensure no open issues/PRs tagged P2, https://github.com/ansible/ansible/labels/P2 , or coordinate with core team manager about those
• Notify members of the documentation team who have merge privileges for backports that a release is pending and they should hold off on any backport merges until the release is completed.

Make release (2.6+)

• From stable-X.Y (devel for alpha release) checkout root:

  checkout stable-X.Y from ansible/ansible remote
  

• Make sure you’re using the checked out version of ansible:

  source hacking/env-setup
  

• Change to the release directory

  cd packaging/release
  

• Make the release:

  make release version={version}
  

  Where the version is:
  • Alpha releases: X.Y.0a1
  • Beta releases: X.Y.0b1
  • Release candidates: X.Y.0rc1
  • Final: X.Y.0
• Push the release if you are satisfied with the changes from the previous step:

  git push
  

Builds

• Wait for the Shippable build of the stable branch to pass before continuing: https://app.shippable.com/github/ansible/ansible/dashboard

• Run Ansible Release Jenkins job

  • Jenkins: Ansible Release
    • GIT_SHA == release SHA
    • PUBLISH release (or preview if it’s a preview)
    • MAJOR_MINOR X.Y portion of the release
    • LATEST checked if X.Y is the current major release, else not
  • Hit Build

• If X.Y is current major release:

  • Do not do this for pre-releases!!
  • Once DEB build has finished, visit https://launchpad.net/~ansible/+archive/ubuntu/ansible-X.Y/+packages (logged in as jenkins@ansible.com), poll for completed build status
  • Once all Launchpad package builds have green checkboxes (can take anywhere from 15m to several hours), click "Copy packages" in upper right corner, select current X.Y.Z packages, select Destination PPA as ansible, Destination series as "The same series", Copy options as "Copy existing binaries", then hit "Copy Packages"

NOTE: You don’t need to wait for the docs to be published unless they will be referenced in the release announcements and are new or have changed since the last release. For a new X.Y.0 release you will likely want to wait, but probably not for a X.Y.1+ release.

• Check that packaged versions made it to:
  • https://pypi.org/project/ansible/#history
  • https://releases.ansible.com/ansible/
  • https://releases.ansible.com/ansible/rpm/
  • https://launchpad.net/~ansible/+archive/ubuntu/ansible-X.Y

Publish release (2.6+)

• Once packaged releases are verified working, publish (tag) the release:

  make publish
  

• Push the release if you are satisfied with the changes from the previous step:

  git push --follow-tags
  

• If this is a rc or beyond, the red hat release manager now needs to make rpm builds

Job cleanup

• If this is a new major release (X.Y.0) remove the now unsupported stable branch from the Jenkins Nightly RPM job.
• Alert tower @qe in #testing that the branch is EOL and doesn't need to be tested anymore.

---

Notifications

Google Groups (mailing lists)

• Beta and beyond: https://groups.google.com/forum/#!forum/ansible-devel

• Final + servicing: https://groups.google.com/forum/#!forum/ansible-project

• Final + servicing: https://groups.google.com/forum/#!forum/ansible-announce

• Use the script hacking/build-ansible.py release-announcement --name 'Toshio Kuratomi' --version 2.7.9 to generate a sample release announcement, both short form for twitter and long form for email. (--version can be repeated to make a unified announcement for multiple versions)

  • You may want to edit the announcement to add additional information.
  • Pre-releases will definitely want to edit bits about the expected next release.

Internal email

• Beta and beyond: ansible-releases@redhat.com
  • You can use the template from above and add a line to say that this is an upstream/community release.

Twitter

• RC and beyond (mention @ansible for marketing ReTweet)
  • Example: @ansible 2.6.0 Release Candidate 4 is now available! Get it on PyPI: pip install ansible==2.6.0rc4, https://releases.ansible.com/ansible/ or GitHub. Happy testing!
• Final + servicing: Use the script in hacking/build-ansible.py release-announcement

NOTE: Copying a tweet such as the one above from the Twitter UI may result in additional space(s) being added after the URL.

IRC

• Beta and beyond: #ansible, #ansible-devel

  • You will need to hack the URL and add ?oldui=1 to make it publicly available without Google login.
  • Example: Release Candidate 3 of Ansible 2.6.0 is now available: https://groups.google.com/forum/?oldui=1#!topic/ansible-announce/t0leZ4_0ifQ

• If the release is a stable release, update the topic on the #ansible channel.

  • Example: https://docs.ansible.com * latest releases: 2.6.0 / 2.5.5 / 2.4.5.0

Ansible Slack

• All: #core_internal
• rc and beyond: @elathram in #marketing - Include the link to the tweet
• Final + servicing: #ship_it