New release system #3249

Saransh-cpp · 2023-08-08T04:23:26Z

Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes #2881

Type of change

Please add a line in the relevant section of CHANGELOG.md to document the change (include PR #) - note reverse order of PR #s. If necessary, also add to the list of breaking changes.

New feature (non-breaking change which adds functionality)
Optimization (back-end change that speeds up the code)
Bug fix (non-breaking change which fixes an issue)

Key checklist:

No style issues: $ pre-commit run (or $ nox -s pre-commit) (see CONTRIBUTING.md for how to set this up to run automatically when committing locally, in just two lines of code)
All tests pass: $ python run-tests.py --all (or $ nox -s tests)
The documentation builds: $ python run-tests.py --doctest (or $ nox -s doctests)

You can run integration tests, unit tests, and doctests together at once, using $ python run-tests.py --quick (or $ nox -s quick).

Further checks:

Code is commented, particularly in hard-to-understand areas
Tests added that prove fix is effective or that feature works

.github/workflows/publish_pypi.yml

.github/workflows/update_version.yml

agriyakhetarpal

Just realised that these file paths were not changed earlier, it's good that we caught them before the 23.9 release:

https://github.com/pybamm-team/PyBaMM/blob/develop/scripts/update_version.py#L43-L47

should point to docs/_static/versions.json instead of docs/source/_static/versions.json

…nto new-release

Saransh-cpp · 2023-08-08T20:34:35Z

This should be ready to review. Copying release_workflow.md here as this PR's description -

Release workflow

rc0 releases (automated)

The update_version.yml workflow will run on every 1st of January, May and September, creating 2 PRs -
1. Incrementing the version to YY.MMrc0 by running scripts/update_version.py in the following files -
  - pybamm/version.py
  - docs/conf.py
  - CITATION.cff
  - vcpkg.json
  - docs/source/_static/versions.json
2. A PR from develop to main
The version PR should be merged into develop, and then the develop-to-main PR should be merged into main.
Once the tests pass, create a new GitHub pre-release with the same tag (YY.MMrc0) from the main branch and a description copied from CHANGELOG.md.
This release will automatically trigger publish_pypi.yml and create a pre-release on PyPI.

rcX releases (manual)

If a new release candidate is required after the release of rc0 -

Fix a bug in main (no new features should be added to main once rc0 is released) and develop individually.
Run update_version.yml manually while using append_to_tag to specify the release candidate version number (rc1, rc2, ...).
This will create a PR incrementing the version to YY.MMrcX by running scripts/update_version.py in the following files -
- pybamm/version.py
- docs/conf.py
- CITATION.cff
- vcpkg.json
- docs/source/_static/versions.json
The version PR should be merged into main, because merging it into develop would require merging develop into main, something we don't want (develop will have new features).
Once the tests pass, create a new GitHub pre-release with the same tag from the main branch and a description copied from CHANGELOG.md.
This release will automatically trigger publish_pypi.yml and create a pre-release on PyPI.
Manually merge main back to develop if any conflicts arise.

Actual release (manual)

Once satisfied with the release candidates -

Run update_version.yml manually, leaving the append_to_tag field blank ("") for an actual release.
This will create a PR incrementing the version to YY.MM by running scripts/update_version.py in the following files -
- pybamm/version.py
- docs/conf.py
- CITATION.cff
- vcpkg.json
- docs/source/_static/versions.json
The version PR should be merged into main, because merging it into develop would require merging develop into main, something we don't want (develop will have new features).
Once the tests pass, create a new GitHub release with the same tag from the main branch and a description copied from CHANGELOG.md.
This release will automatically trigger publish_pypi.yml and create a release on PyPI.

Other checks

Some other essential things to check throughout the release process -

Update baseline of registries in vcpkg-configuration.json as the latest commit id from pybamm-team/sundials-vcpkg-registry
Update CHANGELOG.md with a summary of the release
Update jax and jaxlib to the latest version in pybamm.util and fix any bugs that arise
If building wheels on Windows gives a vcpkg related error - revert the baseline of default-registry to a stable commit in vcpkg-configuration.json
Make sure the URLs in docs/source/_static/versions.json are valid

codecov · 2023-08-08T20:52:58Z

Codecov Report

Patch and project coverage have no change.

Comparison is base (c1c7c29) 99.70% compared to head (2e5b04f) 99.70%.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #3249   +/-   ##
========================================
  Coverage    99.70%   99.70%           
========================================
  Files          248      248           
  Lines        18894    18894           
========================================
  Hits         18839    18839           
  Misses          55       55

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

brosaplanella

Looks good to me, but let's wait to merge until the others are happy with it as this is quite a major point. Two main questions/comments:

Is the CHANGELOG updated automatically or do we need to do it manually (I couldn't quite tell)?
With this approach we will not have a separate branch for a specific version. Not sure we need it, but that's a major change from what we discussed in Reduce release frequency #2881 so just wanted to highlight it.

Saransh-cpp · 2023-08-09T14:29:49Z

Oh, I see, I missed incorporating a few points. I'll look into that. Thanks for the review!

martinjrobins

Looks great :) I just had one suggestion.

.github/release_workflow.md

Saransh-cpp · 2023-08-20T03:51:14Z

New logic with the branch stuff below! Here are some links from my fork -

The v23.8 branch - this should be created automatically on scheduled runs
Commit updating the version to rc1 - this was created by manually running update_version.yml
Commit updating the version to v23.8 and a PR from 23.8 to main - this was created by manually running update_version.yml

rc0 releases (automated)

The update_version.yml workflow will run on every 1st of January, May and September, updating incrementing the version to YY.MMrc0 by running scripts/update_version.py in the following files -
- pybamm/version.py
- docs/conf.py
- CITATION.cff
- vcpkg.json
- docs/_static/versions.json
  
  These changes will be automatically pushed to a new branch YY.MM.
Create a new GitHub pre-release with the tag YY.MMrc0 from the YY.MM branch and a description copied from CHANGELOG.md.
This release will automatically trigger publish_pypi.yml and create a pre-release on PyPI.

rcX releases (manual)

If a new release candidate is required after the release of rc0 -

Fix a bug in YY.MM (no new features should be added to YY.MM once rc0 is released) and develop individually.
Run update_version.yml manually while using append_to_tag to specify the release candidate version number (rc1, rc2, ...).
This will increment the version to YY.MMrcX by running scripts/update_version.py in the following files -
- pybamm/version.py
- docs/conf.py
- CITATION.cff
- vcpkg.json
- docs/_static/versions.json
  
  These changes will be automatically pushed to the existing branch YY.MM.
Create a new GitHub pre-release with the same tag (YY.MMrcX) from the YY.MM branch and a description copied from CHANGELOG.md.
This release will automatically trigger publish_pypi.yml and create a pre-release on PyPI.

Actual release (manual)

Once satisfied with the release candidates -

Run update_version.yml manually, leaving the append_to_tag field blank ("") for an actual release.
This will increment the version to YY.MMrcX by running scripts/update_version.py in the following files -
- pybamm/version.py
- docs/conf.py
- CITATION.cff
- vcpkg.json
- docs/_static/versions.json
  
  These changes will be automatically pushed to the existing branch YY.MM.
Next, a PR from YY.MM to main will be generated that should be merged once all the tests pass.
Create a new GitHub release with the same tag from the main branch and a description copied from CHANGELOG.md.
This release will automatically trigger publish_pypi.yml and create a release on PyPI.

Other checks

Some other essential things to check throughout the release process -

If updating our custom vcpkg registory entries pybamm-team/sundials-vcpkg-registry or pybamm-team/casadi-vcpkg-registry (used to build Windows wheels), make sure to update the baseline of the registories in vcpkg-configuration.json to the latest commit id.
Update CHANGELOG.md with a summary of the release
Update jax and jaxlib to the latest version in pybamm.util and setup.py, fixing any bugs that arise
Make sure the URLs in docs/_static/versions.json are valid

agriyakhetarpal

Thanks, @Saransh-cpp, this looks good! I added some minor review comments that you can address.

.github/workflows/release_reminder.yml

.github/workflows/update_version.yml

scripts/update_version.py

agriyakhetarpal · 2023-08-20T05:20:38Z

There is also a draft PR for adding towncrier (#3278) that we would need to address before the next release, just a heads up

brosaplanella

Looks good to me, thanks!

Co-authored-by: Agriya Khetarpal <74401230+agriyakhetarpal@users.noreply.github.com>

.github/workflows/update_version.yml

…nto new-release

Saransh-cpp · 2023-08-20T19:03:31Z

Thanks for the reviews, @agriyakhetarpal and @brosaplanella!

Saransh-cpp · 2023-08-29T17:17:03Z

A gentle bump for a review on this, @martinjrobins. We would be able to test this new system if the PR is merged before 1st September.

…n `pyproject.toml`

martinjrobins

looks great, happy to merge

Saransh-cpp · 2023-09-19T09:20:59Z

Thanks, @martinjrobins!

…n `pyproject.toml`

Saransh-cpp added 6 commits August 7, 2023 21:07

Initial try at new release system

b7901c6

Logic for rc2, rc3, ...

2c63ed6

Don't edit CHANGELOG automatically

0e32e7f

Fix versioning

cb0415c

Fix release_workflow

246ca6b

Format markdown

405dc74

Saransh-cpp commented Aug 8, 2023

View reviewed changes

Apply suggestions from code review

db34b64

Saransh-cpp marked this pull request as draft August 8, 2023 04:26

Saransh-cpp mentioned this pull request Aug 8, 2023

Update publish workflow #3222

Closed

8 tasks

agriyakhetarpal requested changes Aug 8, 2023

View reviewed changes

Saransh-cpp added 2 commits August 8, 2023 16:31

Update version.json file path

38313a1

Merge branch 'new-release' of https://github.com/Saransh-cpp/PyBaMM i…

d36bbd5

…nto new-release

Saransh-cpp marked this pull request as ready for review August 8, 2023 20:34

Saransh-cpp requested review from martinjrobins, valentinsulzer and brosaplanella August 8, 2023 20:35

brosaplanella approved these changes Aug 9, 2023

View reviewed changes

Saransh-cpp marked this pull request as draft August 9, 2023 14:29

martinjrobins requested changes Aug 16, 2023

View reviewed changes

.github/release_workflow.md Outdated Show resolved Hide resolved

Saransh-cpp added 7 commits August 19, 2023 23:39

Use branching + Martin's suggestion

42a4993

Fix syntax

07566dc

Just rc

ffbf836

Single quotes

26c1eb0

Run on fork

10c7d71

Install all

3bdd128

PR from new branch to main

c88e744

Saransh-cpp added 2 commits August 19, 2023 23:40

Update CHANGELOG programmatically

84c16b2

Fix PR description

1a0b439

CHANGELOG is now updated automatically

afe5e44

Saransh-cpp marked this pull request as ready for review August 20, 2023 03:52

Saransh-cpp requested review from brosaplanella, martinjrobins and agriyakhetarpal August 20, 2023 03:52

agriyakhetarpal approved these changes Aug 20, 2023

View reviewed changes

.github/workflows/release_reminder.yml Outdated Show resolved Hide resolved

.github/workflows/update_version.yml Outdated Show resolved Hide resolved

.github/workflows/update_version.yml Outdated Show resolved Hide resolved

scripts/update_version.py Outdated Show resolved Hide resolved

brosaplanella approved these changes Aug 20, 2023

View reviewed changes

Simplify cron job syntax

94009b6

Co-authored-by: Agriya Khetarpal <74401230+agriyakhetarpal@users.noreply.github.com>

Saransh-cpp commented Aug 20, 2023

View reviewed changes

.github/workflows/update_version.yml Outdated Show resolved Hide resolved

Saransh-cpp added 3 commits August 20, 2023 14:59

Run only in pybamm-team

6d36974

No docs for rc versions

3b4c507

Merge branch 'new-release' of https://github.com/Saransh-cpp/PyBaMM i…

e52e5a8

…nto new-release

Resolve conflicts

434f1a0

agriyakhetarpal added a commit to agriyakhetarpal/PyBaMM that referenced this pull request Sep 4, 2023

pybamm-team#3049, pybamm-team#3249, pybamm-team#2881 Update version i…

2e332b5

…n `pyproject.toml`

Resolve conflicts

2e5b04f

martinjrobins approved these changes Sep 19, 2023

View reviewed changes

Saransh-cpp merged commit 8da245e into pybamm-team:develop Sep 19, 2023
31 of 34 checks passed

Saransh-cpp deleted the new-release branch September 19, 2023 09:21

agriyakhetarpal mentioned this pull request Sep 28, 2023

Create 23.09 (final or rc0) release #3374

Closed

Saransh-cpp mentioned this pull request Oct 1, 2023

Fix release workflow link in issue template #3384

Closed

This was referenced Oct 21, 2023

Add support for pyproject.toml #3301

Merged

i2885 towncrier support and release workflow integration #3469

Open

js1tr3 pushed a commit to js1tr3/PyBaMM that referenced this pull request Aug 12, 2024

pybamm-team#3049, pybamm-team#3249, pybamm-team#2881 Update version i…

b9cac7a

…n `pyproject.toml`

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New release system #3249

New release system #3249

Saransh-cpp commented Aug 8, 2023 •

edited

Loading

agriyakhetarpal left a comment

Saransh-cpp commented Aug 8, 2023

codecov bot commented Aug 8, 2023 •

edited

Loading

brosaplanella left a comment

Saransh-cpp commented Aug 9, 2023

martinjrobins left a comment

Saransh-cpp commented Aug 20, 2023

agriyakhetarpal left a comment

agriyakhetarpal commented Aug 20, 2023

brosaplanella left a comment

Saransh-cpp commented Aug 20, 2023

Saransh-cpp commented Aug 29, 2023

martinjrobins left a comment

Saransh-cpp commented Sep 19, 2023

New release system #3249

New release system #3249

Conversation

Saransh-cpp commented Aug 8, 2023 • edited Loading

Description

Type of change

Key checklist:

Further checks:

agriyakhetarpal left a comment

Choose a reason for hiding this comment

Saransh-cpp commented Aug 8, 2023

Release workflow

rc0 releases (automated)

rcX releases (manual)

Actual release (manual)

Other checks

codecov bot commented Aug 8, 2023 • edited Loading

Codecov Report

brosaplanella left a comment

Choose a reason for hiding this comment

Saransh-cpp commented Aug 9, 2023

martinjrobins left a comment

Choose a reason for hiding this comment

Saransh-cpp commented Aug 20, 2023

rc0 releases (automated)

rcX releases (manual)

Actual release (manual)

Other checks

agriyakhetarpal left a comment

Choose a reason for hiding this comment

agriyakhetarpal commented Aug 20, 2023

brosaplanella left a comment

Choose a reason for hiding this comment

Saransh-cpp commented Aug 20, 2023

Saransh-cpp commented Aug 29, 2023

martinjrobins left a comment

Choose a reason for hiding this comment

Saransh-cpp commented Sep 19, 2023

Saransh-cpp commented Aug 8, 2023 •

edited

Loading

codecov bot commented Aug 8, 2023 •

edited

Loading