Add version automation rules

[agjohnson]

This feature will implement a series of project rules to manipulate new versions. It will focus on automation of activating versions to start, but will eventually aim to give users the ability to manage a version's state, aliases, and privacy without touching RTD on each new release.

The major use case we're targeting as a proof of concept is that users want to enable new tags automatically, by matching tag/branch names.

Concepts

Version automation rules will be constructed of matches and actions. Rule matches can act on tags and/or branches, selectable in the model. Rules will have priority and can be moved around.

Rule matches might be:

• Regex match
• Most recent commit
• Highest version branch/tag

Actions might be:

• Set active
• Set alias
• Set private privacy
• Set public privacy
• Deactive

We'll only focus on a few of these at a time.

Work

For our inital pass, regex matching is the most flexible, and version activation is a simple and useful action target to implement. We'll do a first pass at a UI, which might look like the docker hub UI for version automation:
#4001 (comment)

It's a bad UX however, and we'll eventually have helpers for this instead of relying on developer concepts like regex.

• Implement modeling that is flexible enough for our future rules/actions
• Implement regex matching on to trigger rule action
• Implement action that enabled version

[davidfischer]

Version aliases -- build version 2.0 from releases/2.x branch

This part may already be implemented here. It is not exposed to end users though.

[davidfischer]

I'm envisioning something like the following. Is that what you're thinking?

[davidfischer]

Probably replace "branches" with "tags".

[ericholscher]

We also have a bunch of logic around this where we implemented "supported" versions:

• https://github.com/rtfd/readthedocs.org/blob/master/readthedocs/projects/models.py#L303-L311
• https://github.com/rtfd/readthedocs.org/blob/master/readthedocs/projects/models.py#L259-L279

It turned out to be incredibly difficult to automate, and lead to lots of errors. We should likely remove all of that code from the codebase, as it's hidden in the Project forms because it was confusing users:

• https://github.com/rtfd/readthedocs.org/blob/master/readthedocs/projects/forms.py#L202

[ericholscher]

This was used to mark docs as "unsupported" when people landed on them. It would turn the RTD bar red if it was unsupported and show users a message (similar to our "not the latest version" warning). It ended up being way too complicated and most people don't have standardized way of handling supported versions.

I could see having a more flexible language/framework for this stuff, but at the end of the day, automating things around version numbers is hard, as we've learned with stable versions and all the mess around those too :)

[davidfischer]

I tend to agree. Explicit is better than implicit. Maybe the best way is to just have a checkbox in the Version form where a version can be marked deprecated. Is that different than "active"?

In the immediate future, I really only need version aliases. There's a model for them but it doesn't appear to do anything.

[ssbarnea]

I am quite curious when we will a solution for publishing only tags. At this moment I wasn't able to find any workaround for doing this.

[davidfischer]

See #4505 for version aliases (to some degree)

[humitos]

Maybe the best way is to just have a checkbox in the Version form where a version can be marked deprecated. Is that different than "active"?

Yes.

When you mark a version as inactive, the documentation's version won't be served and it won't be shown in the versions flyout. On the other hand, from what you said, marking a version as deprecated will be served and shown but with a message on the top.

The idea of deprecated version with a custom message on the top can be achieved with this extension (currently, it needs to re-build the old version but I'm thinking on a way to not need that)

[agjohnson]

Ah interesting. This might meld into our discussion on version level privacy and hiding the version. Perhaps we're really talking about a status or something field. It sounds like we have a few states that are exclusive:

• Active
• Active but deprecated
• Active but hidden
• Active but deprecated and hidden? Is this a thing? Does it need to be different from deprecated?
• Inactive

[humitos]

Active but deprecated and hidden? Is this a thing? Does it need to be different from deprecated?

If deprecated means that RTD will automatically add a version warning banner, I'd say this is a thing: "you don't want to expose the version in the flyout but you want to serve it with a warning message"

[agjohnson]

One other feature that might roll into these rules (at least internally, we might use the same code), is pull request automation. There might be a rule for any branch that matches pull, do a separate pull request process. It wouldn't be surfaced to user, and it might just be hardcoded, but might be a good way to keep rules in a single place. Keep this in mind for now.

Lets split this up into 2 phases:

Edit: disregard. we discussed a better batching of this work

[humitos]

These are the rules that DockerHub allows and I suppose could help us to design ours:

One other feature that might roll into these rules (at least internally, we might use the same code), is pull request automation. There might be a rule for any branch that matches pull, do a separate pull request process

I think these things should be related, yes: #1340

[stsewd]

So, our first step here is modeling.

This is what we or need to agreed:

• Allow us to override the way we match the rules (so we can support exact match, regex, or using https://pypi.org/project/semver/)
• Allow us to give priorities over rules
• Apply the given rule to branches/tags only?

Another thoughts:

• Aliases could only work with exact matches I guess?
• Are we going to re-run this rules for every version or only when a new version is created?
• How are we going to detect stable? (This kind of depends on ^)

So what I can think having something like this initially:

Apply the rules to only new created versions, if they match, mark them as active and with the default privacy level (public for .org and private for .com) till we figure out what to do with the privacy levels.

We can manage aliases only for exact matches in the beginning till we figure out how to do it with the others (capturing groups, etc), so defaulting to the same slug for the alias. This can be a problem if we only apply this to new created versions, actually.

Also, would be good to have some uses cases around this:

• Activate and build automatically only when we have a new mayor version/new patch version: (how we do the comparison? Against what version?)
• Activate and build automatically all new versions
• Activate and build only versions that match a pattern

Note: When I said versions, I mean branches and tags

[stsewd]

I think maybe version aliases are better in the version model? Or match all the versions (not only new ones) with exact matches only?

[ericholscher]

Feels like v1 here is just building the ability to expose our existing rules to users:

• latest version selection
• stable version selection
• Auto-building the latest tagged version for projects with any tags built (do we still do this?)

I feel like a regex solution is moving a bit farther down the path of building UI/UX for users, without really understanding what they could be used for.

It also feels like Version aliases are mixed in here, when implementing latest, since it's effectively an alias (latest -> master).

[agjohnson]

latest and stable is more of a refactor of our logic, and isn't the feature we necessarily want here. It's more of a byproduct once we have automation rules set up, and gives us a place to explicitly state how the project determines latest and stable.

This feature is more for examining the incoming tag/branches and automatically taking actions based on matches. Actions might be automatically activating a version, setting an alias, setting privacy level, or deactivating a version.

We've identified several use cases, the easiest to start with is simply enabling new versions. We will use regex however, as it's the most flexible to start. It's not good UX though, but gives users a usable feature.

[agjohnson]

I've updated the description to be more discrete amount of work. I added #5318 to address automating aliases, and #5319 to address refactoring our latest/stable logic. We'll re-evaluate next steps after that, which would be better UX rules.

Also, to clarify previous comments here:

• We are addressing difficulty with automating version by surfacing the exact rules to users. Problems with automating this in the past was either due to rules that were too generalized to be 100% useful, or rules that were implicit and hidden from users.
• Regex and the UI around this is rather hamfisted, but it's a flexible way to start a feature
• I am backtracking a little on complicating this feature with something such as PR building. We can discuss this later

[stsewd]

Another action suggested by @humitos is set default version, also maybe set default branch too?

[stsewd]

Actually set default branch is covered by #5319

[stsewd]

The modeling and UI was already done. We have more specific issues:

• Add alias support to version automation #5318
• Add latest/stable rules to version automation #5319
• Write docs for automation rules #6059
• Support option to invert match on automation rules #6354
• Support custom action for automation rules with webhook + api #6375