Discussion: micromamba releases should be built and released in this repository

[pavelzw]

Xref
mamba-org/micromamba-releases#5
#2536

Right now, the micromamba release workflow is the following:

• somebody releases a new version
• the Conda Forge Auto tick bot doesn’t realize that there is a new version here because of the tag structure of this repository (see https://conda-forge.org/status/ -> version updates -> errored)
• Somebody needs to manually create a PR incrementing the version of the recipe
• the conda-forge build fails (Update to 1.2.0 conda-forge/micromamba-feedstock#107, Release 1.4.4 conda-forge/micromamba-feedstock#120, Update to 1.4.5 conda-forge/micromamba-feedstock#128)
• Somebody needs to fix the build
• The build gets merged
• The build is available on anaconda.org
• micro.mamba.pm points to the latest anaconda.org build
• mamba/micromamba-releases also gets their releases from there using this workflow

I think this release workflow is highly inefficient and would propose this alternative:

• on push of a tag, the CI builds its own static build versions (the windows build got way less flakey with Use only vcpkg for static windows build #2520, macOS and linux are already pretty reliable imo)
• using softprops/action-gh-release, a release with the binaries attached is automatically drafted
• Somebody needs to press publish (this also could be skipped if we say draft: false)
• …
• Profit

Like this, we could guarantee that there is always the most recent version available when a release is created.

[pavelzw]

@AntoinePrv @JohanMabille what do you think of this proposal? I could create a PR implementing this in CI if you are on board with it.

[jonashaag]

I strongly support this proposal. The current release process needs too much human interaction and is scattered over multiple repositories. I'm also in favour of hosting the binaries here rather than in a separate repository.

[AntoinePrv]

We have plan for a non statically linked micro-mamba to be the replacement to python mamba so I think we should keep a feedstock and make sure it is up to date.

On push of a tag, the CI builds its own static build versions

Would that be using the recipe from the feedstock?

[pavelzw]

Would that be using the recipe from the feedstock?

I would suggest keeping the recipe for the static build in this repository.

[pavelzw]

I am strongly in favor of a non-statically linked micromamba on conda-forge and deprecating the statically linked version on conda-forge.
Usually package manager do dynamic linking, that's what they are here for ;P
We can keep the static linking only to this repository.

[JohanMabille]

It would be weird to not have micromamba available on conda-forge. I agree the releasing process is error-prone and implies too many human interactions, but deprecating the micromamba feedstock looks a bit extreme to me.I think we can have both the automatic release on this repo with the process you described, and continue to update the feedstock repo. Ideally we should be able to use a bot / GHA to automagically update the feedstock upon release tag until we get back to a more traditional workflow when we split this repo and the conda-forge bot can detect releases.

[pavelzw]

deprecating the micromamba feedstock looks a bit extreme to me

I agree, maybe it came out wrong. I didn't mean to deprecate the micromamba-feedstock but the static build on conda-forge.

How about switching to a dynamically linked build in the micromamba feedstock (maybe also providing a micromamba-static output)? This would be more in-line with how the other packages are doing it.

The question here would be how many people rely on a static micromamba build being available on anaconda.org/conda-forge/micromamba (for whom such a switch would probably result in an error)... My guess would be not that many, but probably some that depend on a static build being available on

• https://micromamba.snakepit.net/api/micromamba/linux-$ARCH/$VERSION"
• https://micro.mamba.pm/api/micromamba/$PLATFORM-$ARCH/latest
• https://github.com/mamba-org/micromamba-releases/releases/latest/download/micromamba-$PLATFORM-$ARCH

Since mamba-org is in control of all these three domains, this could be migrated without collateral damage.

[wolfv]

I think there are some social aspects we should consider, too:

• the micromamba package is also used in the Miniconda and Mambaforge installers that are owned by conda-forge
• having a release made by the feedstock means that the conda-forge core team is able to handle the package, and conda-forge is in control
• people might trust conda-forge and the packaging transparency on there more than the buid from the repository

On the other hand the situation is clearly not perfect - the mix with vcpkg is not great.

One attempt I've done in the past is the boa-forge stuff where we'd maintain all the dependencies needed for micromamba. It has worked pretty well (for all platforms).
We could consider building all packages statically from source with something like boa-forge, and that could be potentially done in the feedstock.

[pavelzw]

Btw the new 1.5.2 and 1.5.3 releases are not available on conda-forge yet conda-forge/micromamba-feedstock#162

[AntoinePrv]

We are aware, this is because it requires some changes to the feedstock bu these release don't provide much addition to micromamba/

[pavelzw]

Another issue with building only on conda-forge and not having a clean build+release pipeline only in this repo:

the alpha candidates are now listed in https://api.anaconda.org/release/conda-forge/micromamba/latest

This leads to https://micromamba.snakepit.net/api/micromamba/linux-64/latest and https://micro.mamba.pm/api/micromamba/linux-64/latest providing the alpha releases which should not happen.
The https://github.com/mamba-org/micromamba-releases/ repo had the same issue but was fixed in mamba-org/micromamba-releases@fb241b8

If we provided proper release artifacts in this repo, the alpha release would not be necessary.

[JohanMabille]

If we provided proper release artifacts in this repo, the alpha release would not be necessary.

Alpha releases are totally orthgonal to where we host the releases. The problem here is that https://api.anaconda.org/release/conda-forge/micromamba/latest does not filter pre releases, and anaconda does not provide any stable API for getting the latest stable release. And anyway, we need alpha releases so that clients can early test major incompatible changes.

That being said, now that we have the same code base for mamba and (micro)mamba (only the linkage is different), I agree that we should:

• provide libmamba/libmambapy/mamba (dynamic) on conda-forge
• host the static build of mamba on this repo

I think when this is done, we can archive https://github.com/mamba-org/micromamba-releases

I understand the social aspect of building static mamba on conda-forge, but I think the drawbacks outweigh the advantages.

[oursland]

What's the value-add of a dynamically linked micromamba or mamba?

I am encountering an issue affecting CI builds of a project I am on because the upgrade process of dynamic libraries breaks mamba on Windows: #3292

A statically linked binary would prevent this issue altogether.

[jjerphan]

@oursland: as of 2.0, micromamba is a the statically linked version of mamba with is relevant for CI use-cases.