Container resources (e.g. redis) not compatible with Podman

[petedavis]

I noticed that you can run Aspire on Podman, so I tried the sample application with the environment variable set.

When running the application you get the following errors logged

fail: Aspire.Hosting.Dcp.dcpctrl.ContainerReconciler[0]
      could not create the container    {"Container": {"name":"cache"}, "Reconciliation": 2, "error": "CreateContainer command returned error code 125 Stdout: '' Stderr: 'Error: short-name resolution enforced but cannot prompt without a TTY\n'"}
fail: Aspire.Hosting.Dcp.dcpctrl.ContainerReconciler[0]
      container has failed to start     {"Container": {"name":"cache"}, "Reconciliation": 4, "error": "CreateContainer command returned error code 125 Stdout: '' Stderr: 'Error: short-name resolution enforced but cannot prompt without a TTY\n'"}
^C%

And shows on the Dashboard

The issue is the image names need an explicit prefix for Podman, unlike Docker wich assumes docker.io prefix if none exists. So for full Podman support, all image names from docker hub will need to be explicit with docker.io/redis as the image name.

Verified by changing the sample Program code to:

// var cache = builder.AddRedis("cache");

var cache = builder.AddResource(new RedisResource("cache"))
                    .WithAnnotation(new EndpointAnnotation(System.Net.Sockets.ProtocolType.Tcp, port: null, containerPort: 6379))
                    .WithAnnotation(new ContainerImageAnnotation { Image = "docker.io/redis", Tag = "7.2.4" })
                    .PublishAsContainer();

Which then works perfectly:

Environment:
Fedora 39
dotnet SDK 8.0.100
Aspire version: 8.0.0-preview.4.24156.9+692dc41a65da572a7df25d53a9f2880afe59fdd
Podman v4.9.4

[davidfowl]

We should get this into GA

[petedavis]

I was thinking that maybe there should be a warning log on startup for any registered container where the image name is not prefixed. Should indicate that unprefixed image names are not compatible with all runtimes? Would help with third-party package module developers doing the same thing when adding Aspire support?

[timheuer]

@davidfowl should container resource just be having this as a part of the base type? Registry,Image,Tag?

[davidfowl]

Yes, we have a choice to make. Do we make null mean docker.io in aspire or do we force people do specify it.

[timheuer]

@petedavis Can you confirm your settings a bit? I assume no Podman Desktop installation correct and you just have podman CLI on your Fedora OS? I was just trying this on my mac, but I have Podman Desktop which has registries configured by default in registries.conf and thus the pull worked.

[petedavis]

Sure @timheuer

I have podman installed via dnf, although I have installed Podman Desktop after, which I believe does work differently to having raw Podman service installation.

>  dnf list --installed | grep podman               
podman.x86_64                                        5:4.9.4-1.fc39                      @updates
podman-docker.noarch                                 5:4.9.4-1.fc39                      @updates

[davidfowl]

It's hard to know how common this will be. I imagine that podman has done lots of work to make their tooling a drop in replacement for docker. This must be a well known issue with a workaround?

[timheuer]

Thanks @petedavis -- I assume in /etc/containers/registries.conf.d in both registries.conf and for any of the podman machines, none have a config with:

unqualified-search-registries = ["registry.fedoraproject.org", "registry.access.redhat.com", "docker.io", "quay.io"]

podman has done lots of work to make their tooling a drop in replacement for docker.

@davidfowl I think they have mostly, especially in Podman Desktop where when creating a podman machine it looks like it has some default registries configured (https://podman.io/docs/installation#registriesconf) with this 'unqualified' conf setting (I did nothing to create this, but did all config through Podman Desktop). I think using just podman CLI this is not created (or uncommented) by default.

[timheuer]

@davidfowl I feel like we could consider the same 'unqualified' aspect by modifying

aspire/src/Aspire.Hosting/ContainerResourceBuilderExtensions.cs

Line 140 in ff7a691

public static IResourceBuilder<T> WithImage<T>(this IResourceBuilder<T> builder, string image, string tag = "latest") where T : ContainerResource

to be something like:

    public static IResourceBuilder<T> WithImage<T>(this IResourceBuilder<T> builder, string image, string tag = "latest", string registry = "docker.io") where T : ContainerResource
    {
        if (builder.Resource.Annotations.OfType<ContainerImageAnnotation>().LastOrDefault() is { } existingImageAnnotation)
        {
            existingImageAnnotation.Registry = registry;
            existingImageAnnotation.Image = image;
            existingImageAnnotation.Tag = tag;
            return builder;
        }

        // if the annotation doesn't exist, create it with the given image and add it to the collection
        var containerImageAnnotation = new ContainerImageAnnotation() { Image = image, Tag = tag, Registry = registry };
        builder.Resource.Annotations.Add(containerImageAnnotation);
        return builder;
    }

Downside of the simpler approach is a documentation one I think. As specifying an Image tag that also includes fully qualified registry (as some of the Azure ones do) would fail in this default here, but maybe a better failure separating out the registry as already modeled in the AppModel. E.g. Cosmos would (and maybe should anyway?) change:

aspire/src/Aspire.Hosting.Azure.CosmosDB/AzureCosmosDBExtensions.cs

Line 96 in ff7a691

.WithAnnotation(new ContainerImageAnnotation { Image = "mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator", Tag = "latest" });

[petedavis]

My registreis.conf file has the same list. However, from the terminal, Podman doesn't automatically try and pull from all registries, it asks:

Maybe there are cases where it auto resolves

[timheuer]

Definitely strange -- both macOS and Windows for me yielded no prompt (despite also the docs saying it would):

[petedavis]

This is interesting. I only get a short name resolved if it is explicity listed in the /etc/containers/registries.conf.d/000-shortnames.conf file

> podman image pull alpine:3.13                                                                                                                                                                                
Resolved "alpine" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/alpine:3.13...
Getting image source signatures
Copying blob 72cfd02ff4d0 done   |
Copying config 6b5c5e0021 done   |
Writing manifest to image destination
6b5c5e00213a401b500630fd8a03f6964f033ef0e3a6845c83e780fcd5deda5c

It explicitly has the line

"alpine" = "docker.io/library/alpine

There must be an option to enable the using unqualified-search registries

Podman Desktop works out of the box and will pull redis:7.2.4 no worries. Podman Desktop seems to have behavior different to the package install and CLI.

After selecting a respository in the console, it saves a shortname cache which allows for pulling via shortname after.

> podman image pull redis:7.2.4
 Resolved "redis" as an alias (/home/pete/.cache/containers/short-name-aliases.conf
 Trying to pull docker.io/library/redis:7.2.4..

The cache file was updated after selecting the repository.

> cat ~/.cache/containers/short-name-aliases.conf                                                                                                                                                              
[aliases]
    redis = "docker.io/library/redis"

[timheuer]

Agreed @petedavis something is setting this and the docs don't make it clear what may be doing it (some GH issues of people noting that if Podman wants to be a true drop-in replacement for Docker that the unqualified setting needs to be default). But the guidance elsewhere that short-name usage alone is not a best practice (despite us all doing it ;-)) feels sound and prevents spoofing as well. We have the registries modeled here and should consider this change (applying similar 'unqualified search' default to docker.io IMO).