Skip to content

Creating new releases

Jump to bottom
Joshua D. Campbell edited this page Jul 26, 2023 · 24 revisions

Create a GitHub release

A) Checkout the "devel" branch and make sure you have the latest changes:

git checkout devel
git pull upstream devel

Here, "upstream" is referring to the remote that links to the singleCellTK repo under the "compbiomed" organization.

B) Perform checks

A series of major checks should be made to ensure everything is updated:

  1. All new items and major bug fixes are listed in the "NEWS.md" file. The date should be updated in the NEWS file
  2. The build is passing on GitHub Actions.
  3. The build does not have new errors or warnings with BiocCheck::BiocCheck(). Note that there may be a warning due to file sizes in the .git directory. This can be ignored. The BiocCheck logs can also be observed in a GitHub Actions workflow.

C) Bump up the version in the ‘DESCRIPTION’ file and NEWS.md

  • Minor releases should bump the last number. For example: 2.0.0 → 2.0.1
  • Major releases should push the second number to “99”. This will cause the major release number to be updated on the next Bioconductor release. For example, 1.99.x in the Bioconductor development branch will become 2.0.0 in the next Bioconductor release branch.
  • When working on creating a Bioconductor release, after pulling and merging from bioc/master, the version number should be automatically updated where the second number is odd. If this number is higher than the current version, usually we can just use it. If the bioc devel version (what you pulled) is lower than the current version, you should make sure the second number is odd. Examples:
Current Version Bioc devel Version Should bump to
2.4.1 2.5.0 2.5.x/2.7.x/...
2.1.3 2.5.2 2.5.x/2.7.x/...
2.6.2 2.3.1 2.7.x/2.9.x/...

Note: After each major Bioconductor release, the number should be updated on the ‘master’ and ‘devel’ branches in GitHub by pulling from the Bioconductor repos. This is described in the Create a Bioconductor release section.

  • In NEWS.md, describe the updates in that come with the new release at the top of the file, following the format of previous entries in the file.

D) Build new docs

The current website documentation and tutorials generated by pkgdown should be rebuilt with the new version number. pkgdown uses the local installation of the package to run all the commands and generate any other meta-information, so please be sure to build and install the new version locally before building new docs. In R, the following function can be run to rebuild the site:

pkgdown::build_site()

Make sure to quickly look around the new website to ensure everything is in order!

E) Commit changes

The new changes in the DESCRIPTION, NEWS, and pkgdown files can be committed:

git commit -a -m "new version 2.0.1"

F) Merge into master

Checkout master, merge changes from devel, and push to the compbiomed repo:

git checkout master
git merge devel
git push upstream master

G) Create a new release on GitHub

  1. Go to the Releases page

  2. Click on “Draft a new release”

  3. Create a new tag corresponding to the release number. Make sure to include the “v” before the number (e.g. v2.0.1)

  4. The Target field should be set to the ‘master’ branch

  5. The Release title should follow the format “singleCellTK release v2.0.1”

  6. Copy the changes from the NEWS.md file. Make sure to prefix each change with an asterisk so it will appear as a bullet point. For example:

* Added new cool function
* Fixed a bug in the function 'foo'
  1. Copy new html documentation files on public website camplab.net/sctk [Ali TODO]

H) Create new Docker images on DockerHub

  1. Here are the steps to update sctk-qc
  • Include new linux and python dependencies in Dockerfile (append to the list of packages installed via apt-get / pip install)
  • Include new R depencencies in install_packages.R if not specified in DESCRIPTION file
  • Build new docker image under the same folder of Dockerfile
docker build -t campbio/sctk_qc:new_version .
  • Test new docker image. Debug until it works well in example dataset
  1. Here are the steps to update sctk-shiny [Ali TODO]

I) Update server [Ali TODO]

Create a Bioconductor release

Bioconductor releases only happen twice per year during the Spring (April) and Fall (October). The schedule is published here. Pushes to BioC should be done at least a month before the major release dates to ensure that any new bugs or issues can be resolved in time.

A) Initial setup

These steps only need to be done once to ensure you have access to the repo.

1. Add Bioc remote

Add the Bioconductor GitHub to your remotes. Here we generally use the term "bioc" to refer to this remote rather than "upstream" which is used frequently on the Bioconductor websites. You can check to see if you have a bioc remote set up with the following command:

git remote -v

If you have already added the Bioc remote, there should be some output that looks like this:

bioc    git@git.bioconductor.org:packages/singleCellTK.git (fetch)
bioc    git@git.bioconductor.org:packages/singleCellTK.git (push)
origin  https://github.com/{yourUserName}/singleCellTK.git (fetch)
origin  https://github.com/{yourUserName}/singleCellTK.git (push)
upstream        https://github.com/compbiomed/singleCellTK.git (fetch)
upstream        https://github.com/compbiomed/singleCellTK.git (push)

where bioc refers to the Bioconductor remote, origin refers to your forked GitHub repository, and upstream refers to the main GitHub repository which belongs to "compbiomed". If you don't see the first two lines that start with bioc, please run under local SCTK repo directory:

git remote add bioc git@git.bioconductor.org:packages/singleCellTK.git

2. Get credential

Make sure you have the appropriate credentials to add to the Bioconductor repo. You can check which packages you have access to as a manager at BiocCredentials. If you do not have access, an existing manager needs to email the BioC list according to these instructions.

3. Add a SSH key

A SSH key needs to be created on your local machine (or where ever you are pushing your local repo to BioC). The SSH key can be added to your GitHub account using these instructions. You may need to copy the key manually using the BiocCredentials site (there is an option to add keys after you login).

Other questions may be answered on the BioC FAQ page.

B) Fetch and merge previous changes

The previous changes made by Bioconductor should be retrieved:

git checkout devel
git fetch bioc
git merge bioc/master
# Fix the conflict that usually happens in the DESCRIPTION file for the version number
# Bump version number as needed. Make sure the new version number is valid for Bioc Devel
# Commit the change

⚠️ It should only really be the version number in the DESCRIPTION file that has been updated. Please also refer to the version number section or Bioc Version Numbering to make sure the committed version is valid to be pushed to Bioc.

The Bioconductor repository has multiple branches. In terms of git branches which you can see in your terminal, they are master and RELEASE_X_XX. Here, RELEASE_X_XX is for the record of all released version, while master is the developmental branch. This is different from compbiomed/singleCellTK repo, where master is for latest release and devel is the developmental branch.

C) Create a new GitHub release

Repeat steps D-G in the Create a GitHub release section to create a new Github release on the master branch.

D) Push to Bioconductor

Here is the command to push the master branch to the Bioconductor development branch:

git checkout master
git push bioc master

The latest Bioconductor checks for the devel branch can be found here:

https://bioconductor.org/checkResults/devel/bioc-LATEST/singleCellTK/

And the latest checks for the release branch can be found here:

http://bioconductor.org/checkResults/release/bioc-LATEST/singleCellTK/

E) Bioconductor post-release

The following steps need to be completed after each new Bioconductor release to ensure that there is a release version on GitHub that matches Bioconductor. This will ensure that appropriate documentation with the corresponding version number can be put on the website. Bioconductor will create a new branch of the form RELEASE_X_XX which has the latest release. The middle number of the release will be bumped to the next even number in this release (e.g. v2.7.3 will get changed to v2.8.0). Then on the Bioconductor repo's master branch (which serves as the development branch), the version number will be incremented again (e.g. v2.8.0 to v2.9.0). Thus the public version will be v2.8.0 and the development/master branch will be v2.9.0.

  1. Fetch the new version stored in the branch that looks like RELEASE_X_XX (e.g. RELEASE_3_14), create a copy branch, and push it back to the main repo:
git fetch bioc
git checkout bioc/RELEASE_X_XX
git switch -c RELEASE_X_XX
  1. Merge from the release branch so that only the version number in the DESCRIPTION file should be updated.

⚠️ Do not merge the BioC release to devel branch yet, because there might already be some updates in devel after you pushed to bioc/master. Releases in GitHub and BioConductor should be kept consistent.

git checkout master
git merge RELEASE_X_XX

⚠️ DO NOT merge from the master branch of bioc remote (i.e. bioc/master), because it is the developmental branch for the next Bioconductor release.

  1. Update the NEWS file

Add a section to the top of the NEWS.md file that follows this format:

Changes in Version 2.10.0 (2023-04-25)
================================================================================
* Updated version to match Bioconductor X.XX

Ideally, the date will match the Bioconductor commit which can be found by typing git log. Make sure the Bioconductor release version matches the branch.

  1. Repeat steps D through I from the Create a GitHub release workflow to update the documentation and create the new GitHub release. Note that in step F, you do NOT need to merge from devel. You only need to do the last line to push the master to upstream repo.

  2. Update devel branch so the newest changes will be included in the development branch:

git checkout devel
git merge master

🎉 Congratulations on the new BioConductor Release!

Clone this wiki locally