Docker image tags missing

[cbaker6]

New Issue Checklist

• I am not disclosing a vulnerability.
• I am not just asking a question.
• I have searched through existing issues.
• I can reproduce the issue with the latest version of Parse Server.

Issue Description

Changes made to the docker CI and images pushed to Docker Hub after #7548 don't conform to a traditional best practices of "stable" docker tags. This is a bug as it worked previously and was broken when the auto release process was added. Any developer who uses docker to keep their parse-server up-to-date as well as other docker images will run into issues with the current configuration. Examples of images on Docker Hub that follow a stable tag:

• NodeJS
• MongoDB
• Postgres

Theres are many resources online that discuss best practices for tagging. Even the docker github actions updates tags like latest by default:

Some notes for an online resource:

Stable tags mean a developer, or a build system can continue to pull a specific tag, which continues to get updates. Stable doesn’t mean the contents are frozen, rather stable implies the image should be stable, for the intent of that version. To keep it “stable”, it’s serviced to keep the evil people from corrupting our systems. (ok, evil can also be a simple mistake that has huge impacts)

An example:

A framework team ships 1.0. They know they’ll ship updates, including minor updates. To support stable tags for a given major and minor version, they have two sets of stable tags.

:1 – a stable tag for the major version. 1 will represent the “newest” or “latest” 1.* version.
:1.0 a stable tag for version 1.0, allowing a developer to bind to updates of 1.0, and not be rolled forward to 1.1
:latest which will point to the latest stable tag, no matter what the current major version is.

Steps to reproduce

Attempt to pull the most up-to-date version of the latest tag:

• latest (should be the newest image of the latest version possible)

Actual Outcome

• latest is outdated by over a month and doesn't look like it's receiving any updates
• beta doesn't exist
• alpha doesn't exist
• x doesn't exist
• x.y doesn't exist
• x.y-beta doesn't exist
• x.y-alpha doesn't exist
• x.y.z-beta doesn't exist
• x.y.z-alpha doesn't exist
• x.y.z-beta.w exists
• x.y.z-alpha.w exists

Expected Outcome

Looking at the current auto-release structure, attempts to pull any of the following tags that should contain the most up-to-date versions of their respective image:

• latest (should be the newest image of the latest version)
• beta (should be the latest beta)
• alpha (should be the latest alpha)
• x (should be the latest of the major semver version)
• x.y (should be the latest of the major/minor semver version)
• x.y-beta (should be the latest beta of the major/minor semver version)
• x.y-alpha (should be the latest alpha of the major/minor semver version)
• x.y.z-beta (should be the latest beta of the specific semver version) - I think you can skip this, but if you really want it...
• x.y.z-alpha (should be the latest alpha of the specific semver version) - I think you can skip this, but if you really want it...
• x.y.z-beta.w (should be the beta.w of the specific semver version)
• x.y.z-alpha.w (should be the alpha.w of the specific semver version)

This same behavior should also be replicated on the Parse Dashboard repo.

Environment

Server

• Parse Server version: ^5.0.0.alpha
• Operating system: Linux
• Local or remote host (AWS, Azure, Google Cloud, Heroku, Digital Ocean, etc): local docker image

Database

• System (MongoDB or Postgres): N/A
• Database version: N/A
• Local or remote host (MongoDB Atlas, mLab, AWS, Azure, Google Cloud, etc): N/A

Client

• SDK (iOS, Android, JavaScript, PHP, Unity, etc): N/A
• SDK version: N/A

Logs

[parse-github-assistant]

Thanks for opening this issue!

• 🚀 You can help us to fix this issue faster by opening a pull request with a failing test. See our Contribution Guide for how to make a pull request, or read our New Contributor's Guide if this is your first time contributing.

[mtrezza]

There are several tags that we simply won’t support, like a x or x.y.-alpha tag. There should however be an alpha and beta tag, just like they exist on npm, but they don’t yet exist on docker because we are still in the process of implementing release automation. They will eventually be added to docker, but that is a low priority issue. In the meantime developers can use discrete version tags.

There may not be a need anymore for a latest tag, as we offer more discrete labels, but if docker users expect that label by convention, then we may re-add it, otherwise we let it become obsolete by not updating it anymore, as is the current intention.

What should the latest tag be, the latest stable release or the latest commit on the development branch?

[cbaker6]

What should the latest tag be, the latest stable release or the latest commit on the development branch?

Looking at the MongoDB tags latest is the latest version released (highest major.minor.patch, they have the same digest). So in the current release structure, I don't think latest needs to update on beta/alpha, but can update when a new major, minor, or patch is available. If the latest option of flavor is on auto, it may handle this properly, but I don't know how it behaves when alpha and beta are appended to the tags.

If docker users want to live on the edge, they can use the beta and alpha tags once they are added.

[mtrezza]

If the latest option of flavor is on auto, it may handle this properly, but I don't know how it behaves when alpha and beta are appended to the tags.

Our multi-branch model requires that we define what we want latest to be and we control it in the workflow, so the somewhat opaque auto option is not suitable / needed in our case.

So in the current release structure, I don't think latest needs to update on beta/alpha, but can update when a new major, minor, or patch is available.

Do you mean it should be stable? Then it would be on our release branch. That means latest will l update every 3 months (unless there's a hotfix) according to our release cycle, after a release has matured as beta and goes to stable release.

[cbaker6]

Do you mean it should be stable? Then it would be on our release branch. That means latest will l update every 3 months (unless there's a hotfix) according to our release cycle, after a release has matured as beta and goes to stable release.

This should be fine and docker users will know they can always get the latest release using the standard latest tag.

[mtrezza]

If the latest label should be the the latest stable release, I am not sure there is any point in keeping the latest tag. Because the latest tag cannot be used in a production environment, it will contain braking changes as it always jumps to the latest available version, which would for example be the jump from 4.x to 5.x in January 2022. In that sense, I would even argue to discourage people from using it because it is absolutely opaque in what it contains - but then, why would we publish it if we discourage from using it?

We have had this conversation during the implementation of release automation, and the current intention of deprecating the latest tag comes out of some controversy that is pretty much summarized in this article (and many others) and the fact that "latest" can be interpreted in different ways and lead to confusion, i.e. whether it is the latest development commit or latest production ready release. I still haven't seen an official docker reference as to what latest is intended to mean; that could give us some direction at least.

[mtrezza]

It seems that npm also uses the latest tag as in "latest stable", regardless of major version changes. We are also promoting this on the parse server npm bades in the README. It probably makes sense to also offer that for docker for consistency. I'm not sure about the use case though, maybe it's just a quick way of saying, give me the most recent version, for whatever purpose. It doesn't make much sense to me in a dev or prod environment though. @cbaker6 since you argued for keeping the latest tag, how do you use it or think that others would use it?

[cbaker6]

@cbaker6 since you argued for keeping the latest tag, how do you use it or think that others would use it?

For development, devs often leave their setups that depend on parse (or other images) pointed at latest to test if their setup still works with the latest version, without having to continuously change the tag to point to the specific latest version. The same goes for patched and minor updates. For docker, stable tags essentially give you similar behavior to ~5.0.0 and ^5.0.0. The request for the alpha and beta tags and their siblings essentially yield the same benefits of allowing devs to use the updated alphas and betas without manually updating a tag every time.

For production, devs will most likely point to a specific version or a tag that is stable for minor or patched updates.

[mtrezza]

Alright, let's add the latest tag to docker then.

[mtrezza]

I think we can close this; latest tag is used for docker images that are built based on releases from the release branch, i.e. for us it means "latest stable". This is in line with how we do it on npm. I would not use a different strategy for every registry. If there is a genuine need in the developer community for a "latest development" tag, then we could think about introducing a latest-beta and latest-alpha tag - for every registry we publish in.

[cbaker6]

The issue just isn't about latest. It looks like a number of tags are still missing from the Expected Outcomes section which can be helpful for developers who want images related to patched and minor updates:

Expected Outcome

Looking at the current auto-release structure, attempts to pull any of the following tags that should contain the most up-to-date versions of their respective image:

• latest (should be the newest image of the latest version)
• beta (should be the latest beta)
  alpha (should be the latest alpha)
• x (should be the latest of the major semver version)
• x.y (should be the latest of the major/minor semver version)
• x.y-beta (should be the latest beta of the major/minor semver version)
• x.y-alpha (should be the latest alpha of the major/minor semver version)
• x.y.z-beta (should be the latest beta of the specific semver version) - I think you can skip this, but if you really want it...
• x.y.z-alpha (should be the latest alpha of the specific semver version) - I think you can skip this, but if you really want it...
• x.y.z-beta.w (should be the beta.w of the specific semver version)
• x.y.z-alpha.w (should be the alpha.w of the specific semver version)

[mtrezza]

Got it. It looks like we have everything covered except for the alpha and beta tags, which I agree would be nice to have.

Other labels like x.y-beta are not something we plan to implement for the time being because:

• they require complex release logic --> probably low cost/value ratio to implement this
• they are missing the patch version which can cause issues in release automation
• may not be semver compatible, see semver specs:

  A normal version number MUST take the form X.Y.Z where X, Y, and Z are non-negative integers, and MUST NOT contain leading zeroes. X is the major version, Y is the minor version, and Z is the patch version. Each element MUST increase numerically. For instance: 1.9.0 -> 1.10.0 -> 1.11.0.

[cbaker6]

It looks like the latest may only be updating on minor changes and doesn't update on patch. The last time it was updated was on 5.2.0 4 months ago: https://hub.docker.com/r/parseplatform/parse-server/tags?page=2

[mtrezza]

Docker images need to be created manually (using the GH workflow) when releasing hotfixes. The workflows seemed to run fine, so this needs some more investigation why the images apparently were not pushed to the registry:

https://github.com/parse-community/parse-server/runs/7129817778?check_suite_focus=true#step:13:2

[mtrezza]

I went through the recent releases and made sure they have corresponding images on docker hub. If there is a release missing, please let me know and I'll manually push it. I'm not sure whether the CI needs a fix regarding docker releases, or whether this was just a one-time issue due to the manual release workflow or maybe on docker hub's side. No further CI investigation planned at this point.

It seems that the latest tag is not updated on docker hub. That is likely because the hotfix releases require to trigger a manual release workflow. The manual release workflow never uses the latest tag. Possible solutions:

• a) Allow the docker image upload process to proceed even on hotfix releases where auto-release backmerge fails; that's risky because the auto-release workflow may fail for other reasons in which case the docker image shouldn't be pushed
• b) Add an option to the manual docker release workflow to apply the latest label