The Stock Synthesis development workflow
Stock Synthesis is a generalized population dynamics modeling program used for stock assessments. It is used both for operational assessments and for research.
Stock Synthesis has one primary developer, but a total team of six. The other team members occasionally contribute directly to the Stock Synthesis codebase, but primarily work on other aspects (e.g., documentation, development/maintenance of tools, contributing ideas on improving Stock Synthesis).
Our workflow has evolved over time, mostly to facilitate better communication amongst team members and to produce a more reliable product for end users of Stock Synthesis through the use of increasingly standardized operating procedures and automated steps. There was a learning curve, so over time we have slowly formalized our workflow and arrived at our current state based on our team members’ comfort level with being able to manage the steps.
We use github flow
The default branch is main and is the current "working" version of Stock Synthesis building up to the next release. It should be passing all checks on github. Small changes that are very unlikely to change Stock Synthesis model results (e.g., editing readmes, updating a few words in a warning message) may be pushed directly to the main branch.
For larger changes, an issue should be opened and assigned to the author to communicate to the SS development team and the SS community the issue. Next, a feature branch should be created off of main and development should be done in the feature branch. When ready to merge in the work to main, a pull request should be used.
A pull request should be opened to merge the feature branch into the main branch. Before merging in, the following requirements should be met:
- All github actions checks are passing.
- A member of the Stock Synthesis development team should test the feature and/or review the code. For example, this person can download the Stock Synthesis executable generated from the pull request and test on at least 1 model that the feature developed works as intended.
- Once the reviewer is satisfied that the code is working as expected, the feature branch can be merged into the main branch and deleted.
- The issue can be marked as "Resolved" and double check that the issue was added to the correct milestone. A milestone is a list of issues to be worked on to accomplish a larger goal, often a release (see below for more info on milestones).
- If documentation or a change in other tools is needed, open an issue for it in the pertinent repository. For e.g., an issue is opened in ss-documentation if a new feature needs to be documented. These issues can be linked even though they are in different repositories.
Versions of Stock Synthesis are available from the build-ss3 github actions workflow. Download the artifact to get a version of Stock Synthesis from the desired run to test out locally.
The default GitHub labels were augmented to a new set of labels that was carefully thought about to help organize issues. Note this list evolves to the needs of the maintainers as we adapt our workflow over time.
Milestones are used to plan what features will go into the next release of Stock Synthesis. Typically, we don’t finish the entire list of issues. So, some lower priority issues are bumped to the next release and milestone when it is time to release a new version. It is easier to look at issues associated with a milestone than the full list of issues (the full list of issues hovers at around 100).
In addition, issues are also added to the milestone as they are worked on. This makes it easier to write the release notes when the time for a release comes.
- When new issues come in, they are labeled either as bug or new request (if they fit these categories). Other labels should be added as appropriate.
- New requests should be checked against current issues for duplication.
- Modify the issue titles if needed to ensure the issue is clear with the title alone.
- New requests may be labeled "wishlist" and removed from the label “new request” if they are high priority.
- Bugs should be assigned immediately to the upcoming milestone because they are considered the highest priority to address.
- Add yourself as the assignee if you intend to work the issue.
- Add someone else as the assignee if it is something only they can work on (it is also fine to ask someone in the comments if they would like to be assigned if unsure).
- Assignee will assign the label "In progress" when working on an issue.
Communication can happen about the issue as it is being worked on by the assignee. Others involved in the issue should be tagged with their github usernames to increase the chance that they will see the comment.
When work on an issue has been completed, the "resolved" label should be added and it should be added to the milestone for the next release (e.g., 3.30.17 milestone). The issue should remain open until the need for changes has been noted for documentation and auxiliary tools. This includes entries in the change log and possibly revisions to the user manual, r4ss, and the SSI. The final entry to the Issue commentary should summarize the steps taken to effect the correction or change. It should only be closed if it requires no further action or if the actions to take have been documented in another issue (e.g., if a new feature needs to be documented), an issue can be opened in the ss-documentation repository to do this. Issues connected to a milestone will be closed (if it is appropriate to do so) shortly before a release.
Tag naming conventions are included in the practices for maintainers page.
We still use the Stock Synthesis forum on vlab to communicate new releases to users, as well as for users to ask questions. The Stock Synthesis vlab website provides links to Stock Synthesis executables and resources, although many are managed on GitHub. Users also send their questions to nmfs.stock.synthesis@noaa.gov and (very occasionally) open GitHub issues. Information for users is also included in the readme on GitHub.
- Workflow evolves over time, so assume this will happen. Your team will learn by trial and error what you like and don’t like (e.g., we used to use vlab to host our repositories, but now use GitHub; we tried out using a google “room” group chat (now called spaces) for communication, but we didn’t like it).
- Choose a workflow that your team can use. E.g., we don't use git-flow because it is far too complicated for our team’s knowledge of git right now. Instead, we opted for a simpler main branch + feature branches structure which we can all reliably use.
- Integrate testing and review into your workflow. Many of us are used to just pushing up code without any checks, but this means the mistakes are caught later on (or maybe not caught at all). For Stock Synthesis, we haven’t needed to release as many “fixes” between releases since adding testing and (somewhat informal) code review to our workflow.
- Don’t forget about documentation. We’ve used github issues in the ss-documentation repository to help us remember to make changes to documentation.
- Automate repetitive tasks as much as you can. E.g., we used to have to log onto separate mac and linux machines and manually compile Stock Synthesis; now gh actions does this for us, which saves time. We also used to run tests locally on our computers, and now gh actions runs it for us, so we run tests more frequently with less work.
- Stock Synthesis is one integrated code that must be compiled in its entirety. This precludes unit testing.
- As new features are added they typically bring in a feature that is untestable by the existing testbed of examples. Of course, it is still important to know that the code to implement the new feature has not broken something, but testing of new features requires custom examples.
- Our team has other responsibilities, and a constantly evolving workflow can be challenging to keep track of.
- When we release “fixes” they also include any new features that had been worked on, because we don’t keep fixes and new features on separate branches.