Suggestions from Imperial College London RSE team

[alexdewar]

Description

Hi all! Apologies for having taken so long to do this, but I've finally got around to writing up some of my notes after going through the repo with @AdrianDAlessandro. There's loads of great stuff in here and we've picked up plenty of ideas for things to add to our own template.

This is just a collection of random thoughts and suggestions. By no means am I saying that you should be doing all of these things -- they're just some ideas. In some cases, I think you could make life easier with some slight changes and there are also a couple of features we'd like to see for our own use-case. I'm mostly just hoping that this will serve as a starting off point for a discussion about some of the technical details of the project. Very happy to pitch in and write up any of these suggestions as proper issues and to contribute PRs if you like.

Tooling

Ruff config in pyproject.toml

• I'd personally be inclined to require docstrings for public modules and packages (D100 and D104)
• I think the dummy-variable-rgx could be simplified to ^_. That said, are you sure you want it? I'd be inclined to just disallow unused variables across the board, but that's really a matter of personal taste.
• Comment on line 120 is incorrect

mypy

• You could provide an option to use mypy for static type checking. (We tend to use it in all our projects, though I can also see that it's not necessarily something that every user would want as it does introduce complexity)

Documentation tools

I've noticed you're using Sphinx, which is a perfectly reasonable choice, but we tend to use MkDocs as we've found it's a bit easier to get it to automatically document all your code. If you'd like to add support for this to the template (either instead of or in addition to the Sphinx support), just let me know and I'll have a go.

pre-commit

I've noticed you have an option for a pre-commit hook to lint and format things with ruff.

We tend to use the pre-commit tool for running these sorts of checks. If you haven't come across it before, it's a Python-based tool which can be configured to run any number of linters and formatters as pre-commit checks. You can include it in your pyproject.toml as a dev dependency and then users just need to install the hooks with $ pre-commit install.

The advantages of it cf. hand-rolling scripts are:

• If you have uncommitted changes in your repo, it will handle it gracefully, by stashing the changes before running the checks and restoring them afterwards
• Easy re-use of other people's hooks (there are looooads available)
• Hooks can be written in other languages besides Python, e.g. there is a hook for running the JavaScript-based markdownlint tool (users don't even have to install node for it to work)
• Hooks can be automatically run with the pre-commit CI GitHub app, with changes applied automatically
• Hooks are versioned and the app can also open a PR when there are new versions available

Even if you decide that you don't want end-users to have to bother with it, you could use it for this repo yourselves.

Here's an example config file for one of my projects, as an example of the kind of things you can use it for.

Dependency management

Generally for our projects we pin the versions of dependencies, which we do either with poetry or the more lightweight pip-tools. We then use @dependabot to update the dependency versions at regular intervals. This makes life easier from a developer standpoint as you know all the developers are using (essentially) the same environment and that this is the same as the one used by the CI system etc. On the other hand, we mostly work on standalone Python apps rather than libraries where you might want more flexibility in terms of the supported versions of dependencies, so I appreciate that this kind of setup won't work for everyone. But it is a feature we kind of need. Very happy to help if this is something you'd like!

Release workflows

There are instructions provided to users about how to publish to PyPI using twine, however it is something you can do via a GitHub Action, which makes the process easier and less error prone. See an example here: https://github.com/EnergySystemsModellingLab/MUSE_OS/blob/main/.github/workflows/publish.yml

On a related note, I've also recently come across this Action. It allows you to make a release on GitHub simply by bumping the version number in your project; it then uses the contents of your changelog to generate release notes. I've not used it yet, but it looks super useful!

Misc

Example Python code

I personally think the example code and unit test, while useful as a kind of documentation, are perhaps a little too complex for a template. Perhaps it would be better to put them online somewhere and just have a link in a docstring to a "getting started" page?

Editorconfig

I've got various small suggestions:

• Just include the file unconditionally - no need to make it an option
• I see you're enforcing Unix line endings with editorconfig. You're probably better off telling git to regularise line endings with a .gitattributes file (see here).
• Makefiles need tabs cf. spaces, otherwise they won't work, so it'd be worth adding a rule for this
• It seems like you're not trimming whitespace for documentation. Just wondering why?

VS Code config

You can add recommend extensions to users by adding a .vscode/extensions.json file to the repo. In your case, you may want to recommend the Python and ruff extensions.

You can also change some VS Code editor settings for the repo with a .vscode/settings.json file, e.g. setting it to automatically format files with ruff on save. That said, I know some people hate the concept of having these sorts of user settings stored in a repo, so it's v much a personal choice -- just a thought.

Licencing

I'm slightly confused that there's a "GPLv3 or later" licence option but not a plain "GPLv3" one. It seems that the "or later" part was added in commit 729682e, but as the commit message says, not everyone wants that. It's not a big deal as far as I'm concerned, but technically they're different licences, so it may be worth having a separate "GPLv3 only" licence. I'd personally be inclined to just drop the "or later" variant.

[bouweandela]

I think the dummy-variable-rgx could be simplified to ^_. That said, are you sure you want it? I'd be inclined to just disallow unused variables across the board, but that's really a matter of personal taste.

Sometimes there is no way around this, e.g. when a parent class (perhaps from a package beyond your control) defines a method with certain arguments, but the derived class you're implementing does not use all of these arguments in the method.

[alexdewar]

Ah, that makes sense. We don't usually have the warning turned on for unused arguments -- just variables defined elsewhere -- so don't encounter this problem.

[egpbos]

What a great list of suggestions, thanks for compiling and sharing this! Many of them sound like great changes (or at least additional options) to me.

One comment to clarify: the "GPLv3 or later" convention basically is a statement of trust in the GPL series of licenses. The historical problem it solves is that for instance code that was released under GPLv2 could not easily be "upgraded" to v3, because the licenses are incompatible. With the "or later" addition (we didn't come up with this, by the way) such upgrades become trivial (although it leaves the issue of not being able to deprecate the old license, but one problem at a time I guess ;) ).

[alexdewar]

Ah ok. I was aware that "GPLv3 or later" was a thing some people used, but wasn't aware of the nuance. Thanks for the info.