Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug when listing image tags with manifest list v2 #264

Closed
Enrico204 opened this issue Sep 7, 2022 · 9 comments
Closed

Bug when listing image tags with manifest list v2 #264

Enrico204 opened this issue Sep 7, 2022 · 9 comments
Labels
Milestone

Comments

@Enrico204
Copy link

Enrico204 commented Sep 7, 2022

Hi, I use this docker registry UI and I have an issue listing tag for images using manifest list for manifest v2, schema v2 (aka application/vnd.docker.distribution.manifest.list.v2+json).

Bug description

When pushing an image for manifest list v2, schema v2 (see below for example), the tag listing is showing the tag with an error (due to a 404 status for the HTTP request for that tag). However, when clicking on the tag history, the request is successful.

I traced down the request, and apparently the application/vnd.docker.distribution.manifest.list.v2+json is missing when list is not true:

(self.opts.list ? ', application/vnd.docker.distribution.manifest.list.v2+json' : '')

which is here and maybe somewhere else:

new DockerImage(props.image, tag, {

I think that is an error, but I admit that I'm not sure about that.

How to Reproduce

Steps to reproduce the behavior:

  1. Build/push an image using manifest list (v2)
  2. Open the UI, click on the image (an error appears after the list of tags is loaded)
  3. Click on the history button for the pushed manifest list, all details appear correctly

Expected behavior

List correctly all tags / support manifest list (v2) when listing tags.

Screenshots

Here the error when listing a tag:
Screenshot 2022-09-07 at 11-07-35 Docker Registry UI

Here is the full page of the tag history page:
Screenshot 2022-09-07 at 11-15-46 Docker Registry UI

System information

  • OS: Debian 11 (host), using docker container
  • Browser:
    • Name: Firefox
    • Version: 103
  • Docker registry UI:
    • Version: 2.2.1
    • Server: docker
    • Docker version: 20.10.5+dfsg1
    • Docker registry ui tag: 2
    • OS/Arch: linux/amd64
    • Tools: docker-compose

Additional context

Example container

As an example container, you can either pull the one I'm testing from Docker Hub: enrico204/debian-builder:stretch-v0-armel or you can build this Dockerfile:

FROM docker.io/arm32v5/debian:stretch

RUN apt-get update && apt-get install -yq mtr-tiny

Using this series of commands:

buildah manifest create test:armel
buildah bud -f Dockerfile --manifest test:armel --arch arm   --variant v7
buildah manifest push --all --format=docker test:armel docker://your.registry/test:armel

HTTP requests trace

When listing all tags for my image, I see an HTTP request to the tag which has this issue:

GET https://my.registry/v2/utilities/debian-builder/manifests/stretch-v0-armel
Status 404 Not Found
Version HTTP/2
Transferred 320 B (70 B size)
Referrer Policy strict-origin-when-cross-origin

    	
    content-length 70
    content-type application/json; charset=utf-8
    date Wed, 07 Sep 2022 09:25:00 GMT
    docker-distribution-api-version registry/2.0
    server nginx/1.21.6
    x-content-type-options nosniff
    X-Firefox-Spdy h2
    	
    Accept application/vnd.docker.distribution.manifest.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json
    Accept-Encoding gzip, deflate, br
    Accept-Language en-US,en;q=0.5
    Authorization Basic
    Cache-Control no-cache
    Connection keep-alive
    DNT 1
    Host my.registry
    Pragma no-cache
    Referer https://my.registry/?get=1
    Sec-Fetch-Dest empty
    Sec-Fetch-Mode cors
    Sec-Fetch-Site same-origin
    Sec-GPC 1
    TE trailers
    User-Agent Mozilla/5.0 (X11; Linux x86_64; rv:103.0) Gecko/20100101 Firefox/103.0

When opening the tag history I see this request instead:

GET https://my.registry/v2/utilities/debian-builder/manifests/stretch-v0-armel
Status 200 OK
Version HTTP/2
Transferred 762 B (315 B size)
Referrer Policy strict-origin-when-cross-origin
Request Priority Highest

    	
    content-length 315
    content-type application/vnd.docker.distribution.manifest.list.v2+json
    date Wed, 07 Sep 2022 09:28:16 GMT
    docker-content-digest sha256:fc2950e3a5646c210f1e3a242dc7d957291a3fae1bccb9d655c59d0c538141a3
    docker-distribution-api-version registry/2.0
    etag "sha256:fc2950e3a5646c210f1e3a242dc7d957291a3fae1bccb9d655c59d0c538141a3"
    server nginx/1.21.6
    x-content-type-options nosniff
    X-Firefox-Spdy h2
    	
    Accept application/vnd.docker.distribution.manifest.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, application/vnd.docker.distribution.manifest.list.v2+json
    Accept-Encoding gzip, deflate, br
    Accept-Language en-US,en;q=0.5
    Authorization Basic
    Connection keep-alive
    DNT 1
    Host my.registry
    Referer https://my.registry/?get=1
    Sec-Fetch-Dest empty
    Sec-Fetch-Mode cors
    Sec-Fetch-Site same-origin
    Sec-GPC 1
    User-Agent Mozilla/5.0 (X11; Linux x86_64; rv:103.0) Gecko/20100101 Firefox/103.0
@Joxit
Copy link
Owner

Joxit commented Sep 12, 2022

Hi, thank you for using my project 😄

This is not a error, your registry server should return the default manifest when there is no application/vnd.docker.distribution.manifest.list.v2+json in the Accept header. That's why I do not add this in the tag list page. This reduce the number of call to your registry (with list I have 3 calls instead of 2), for users with hundreds of tags, it's important.

The UI supports multi-arch images and single arch too... That's why I have to go deeper to troubleshoot this issue 😅

I already had issues with buildah (see #114), so I have some questions for you:

  • What's your registry server version ?
  • What's your buildah version ?
  • If you merge your two tags (stretch-v0-armel and stretch-v0) under the same manifest, does it work ? (you may see the history with two tabs, the first one for amd64 and the second for armv7)
  • I've tried your image and I have this error, do you know how to fix this ? :
STEP 1: FROM docker.io/arm32v5/debian:stretch
STEP 2: RUN apt-get update && apt-get install -yq mtr-tiny
error running container: error creating container for [/bin/sh -c apt-get update && apt-get install -yq mtr-tiny]: : exec: "crun": executable file not found in $PATH
error building at STEP "RUN apt-get update && apt-get install -yq mtr-tiny": error while running runtime: exit status 1
ERRO exit status 1    
  • Can you consider to use buildx instead of buildah ? This will easily create a proper multi-arch image

I really think the issue here is caused by buildah, for example, I copied your base image arm32v5/debian:stretch from docker hub to my private registry with skopeo (see #260) and it works (architecture arm and variant v5)

skopeo copy  --insecure-policy  --multi-arch=all  --dest-tls-verify=false docker://docker.io/arm32v5/debian:stretch docker://127.0.0.1:5000/arm32v5/debian:stretch

Tag list:
image
History:
image

When I do the same thing with your image enrico204/debian-builder:stretch-v0-armel and skopeo, I have the same behavior as you.

I also noticed that the architecture variant version is missing in your image built with buildah

{
    "created": "2022-09-07T16:37:50.119322002Z",
    "architecture": "arm",
    "os": "linux",
    "config": {
        "Env": [
            "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
            "DEBIAN_FRONTEND=noninteractive"
        ],
        "Cmd": [
            "bash"
        ],
        "Labels": {
            "io.buildah.version": "1.22.3"
        }
    },
    "rootfs": {
        "type": "layers",
        "diff_ids": [
            "sha256:ef35d4a1050a733d580c43820ac208a458e56c96b8cb7f23dca6aac082125e84",
            "sha256:97a1198586e7b5e0d4950c0f18a5a7acb1ba604e3f7f240ef33846c29d0f2dbd"
        ]
    },
    "history": [
        {
            "created": "2022-06-23T00:55:37.822983329Z",
            "created_by": "/bin/sh -c #(nop) ADD file:4fd9afff1465319b24ccf87076562babe87771f9c1251684705a3896e673aa09 in / "
        },
        {
            "created": "2022-06-23T00:55:38.740692309Z",
            "created_by": "/bin/sh -c #(nop)  CMD [\"bash\"]",
            "empty_layer": true
        },
        {
            "created": "2022-09-07T16:27:53.540608384Z",
            "created_by": "/bin/sh -c #(nop) COPY file:1f46e88741184e55f5401b2e7f1be15be9acf1895f695823f333be1c97666340 in /etc/apt/sources.list ",
            "empty_layer": true
        },
        {
            "created": "2022-09-07T16:27:53.663742161Z",
            "created_by": "/bin/sh -c #(nop) COPY file:84f6b9c2060b6787f2fde3757eab4d267ff648166eb49deffe4a0f6c4017891f in /usr/local/sbin/,preload.sh ",
            "empty_layer": true
        },
        {
            "created": "2022-09-07T16:27:53.663774668Z",
            "created_by": "/bin/sh -c #(nop) ENV DEBIAN_FRONTEND=noninteractive",
            "empty_layer": true
        },
        {
            "created": "2022-09-07T16:27:53.663789674Z",
            "created_by": "/bin/sh -c #(nop) ARG apt_proxy=",
            "empty_layer": true
        },
        {
            "created": "2022-09-07T16:38:56.65836569Z",
            "created_by": "|1 apt_proxy= /bin/sh -c echo \"Acquire::http::Proxy \\\"$apt_proxy\\\";\" > /etc/apt/apt.conf.d/99proxy &&     apt-get update &&     apt-get install -y --no-install-recommends lsb-release build-essential fakeroot debhelper         wget curl jq equivs gettext-base ca-certificates unzip         devscripts debian-keyring libdistro-info-perl quilt libx11-dev libxinerama-dev libxft-dev libfreetype6-dev pkg-config         git libfuse-dev libopus-dev libpam0g-dev nasm libjpeg-dev libssl-dev libx11-dev libxfixes-dev libxrandr-dev &&     apt-get install -y --no-install-recommends -t stretch-backports dkms &&     touch /etc/dkms/no-autoinstall &&     bash /usr/local/sbin/,preload.sh &&     rm /etc/apt/apt.conf.d/99proxy",
            "comment": "FROM docker.io/arm32v5/debian:stretch"
        }
    ]
}

Now the manifest for arm32v5/debian:stretch

{
  "architecture": "arm",
  "config": {
    "Hostname": "",
    "Domainname": "",
    "User": "",
    "AttachStdin": false,
    "AttachStdout": false,
    "AttachStderr": false,
    "Tty": false,
    "OpenStdin": false,
    "StdinOnce": false,
    "Env": [
      "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    ],
    "Cmd": [
      "bash"
    ],
    "Image": "sha256:1303ad235e869dd7eebf55f1be729533ea6faa971673c235bac98d14e84acf16",
    "Volumes": null,
    "WorkingDir": "",
    "Entrypoint": null,
    "OnBuild": null,
    "Labels": null
  },
  "container": "f5c20e096bf50c1e2fe72cc5b9cdf6890f4b3af5c404fb94d1074d818a1f30f3",
  "container_config": {
    "Hostname": "f5c20e096bf5",
    "Domainname": "",
    "User": "",
    "AttachStdin": false,
    "AttachStdout": false,
    "AttachStderr": false,
    "Tty": false,
    "OpenStdin": false,
    "StdinOnce": false,
    "Env": [
      "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    ],
    "Cmd": [
      "/bin/sh",
      "-c",
      "#(nop) ",
      "CMD [\"bash\"]"
    ],
    "Image": "sha256:1303ad235e869dd7eebf55f1be729533ea6faa971673c235bac98d14e84acf16",
    "Volumes": null,
    "WorkingDir": "",
    "Entrypoint": null,
    "OnBuild": null,
    "Labels": {}
  },
  "created": "2022-06-23T00:55:38.740692309Z",
  "docker_version": "20.10.12",
  "history": [
    {
      "created": "2022-06-23T00:55:37.822983329Z",
      "created_by": "/bin/sh -c #(nop) ADD file:4fd9afff1465319b24ccf87076562babe87771f9c1251684705a3896e673aa09 in / "
    },
    {
      "created": "2022-06-23T00:55:38.740692309Z",
      "created_by": "/bin/sh -c #(nop)  CMD [\"bash\"]",
      "empty_layer": true
    }
  ],
  "os": "linux",
  "rootfs": {
    "type": "layers",
    "diff_ids": [
      "sha256:ef35d4a1050a733d580c43820ac208a458e56c96b8cb7f23dca6aac082125e84"
    ]
  },
  "variant": "v5"
}

@Enrico204
Copy link
Author

Interesting! Thanks for the deep dive :-D and for your project :-)

I'll do those tests, and I'll report back any finding 👍

In the meantime:

  • I'm using Docker Registry (official image) version 2.8.1
  • buildah --version is 1.22.3 (image-spec 1.0.1-dev, runtime-spec 1.0.2-dev) (currently it's the latest version for Debian 11 from the OpenSUSE/Kubic repository)
  • I can merge stretch-v0-armel and stretch-v0 tags together, I'll try and report back :-)
  • Regarding the crun error: usually I see these kind of issues when the binfmt_misc capability is not correctly configured/enabled. If you try any other multi-arch build with buildx, or even simply executing docker run -it --rm docker.io/arm32v5/debian:stretch, does it works? If not, then the problem is binfmt_misc. If yes, then... I don't know, I'm learning this multi-arch thing right now :-(
  • I'll try to use buildx and report back if there is any change :-)

On buildah: I was looking at changelogs for newer version, and even if apparently there is no (mentioned) change in multi-arch builds, I'll try to build a newer version of buildah (in addition to all previous tests) to check if the behavior is still present in the latest version.

@Enrico204
Copy link
Author

Enrico204 commented Sep 13, 2022

It seems that there are two different issues: one is due to a buildah bug for the missing variant field in the image, which was fixed somewhere between 1.22 and 1.27. In fact:

  • docker buildx: the image correctly contains the variant field:
{
    "architecture": "arm",
    "config": {
        "Env": [
            "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
            "DEBIAN_FRONTEND=noninteractive"
        ],
        "Cmd": [
            "bash"
        ],
        "OnBuild": null
    },
    "created": "2022-09-13T11:51:01.444941118+02:00",
    "history": [
        {
            "created": "2022-06-23T00:55:37.822983329Z",
            "created_by": "/bin/sh -c #(nop) ADD file:4fd9afff1465319b24ccf87076562babe87771f9c1251684705a3896e673aa09 in / "
        },
        {
            "created": "2022-06-23T00:55:38.740692309Z",
            "created_by": "/bin/sh -c #(nop)  CMD [\"bash\"]",
            "empty_layer": true
        },
        {
            "created": "2022-09-13T11:40:38.277655491+02:00",
            "created_by": "COPY stretch.list /etc/apt/sources.list # buildkit",
            "comment": "buildkit.dockerfile.v0"
        },
        {
            "created": "2022-09-13T11:40:38.424311316+02:00",
            "created_by": "COPY preload-stretch.sh /usr/local/sbin/,preload.sh # buildkit",
            "comment": "buildkit.dockerfile.v0"
        },
        {
            "created": "2022-09-13T11:51:01.444941118+02:00",
            "created_by": "ENV DEBIAN_FRONTEND=noninteractive",
            "comment": "buildkit.dockerfile.v0",
            "empty_layer": true
        },
        {
            "created": "2022-09-13T11:51:01.444941118+02:00",
            "created_by": "ARG apt_proxy",
            "comment": "buildkit.dockerfile.v0",
            "empty_layer": true
        },
        {
            "created": "2022-09-13T11:51:01.444941118+02:00",
            "created_by": "RUN |1 apt_proxy= /bin/sh -c echo \"Acquire::http::Proxy \\\"$apt_proxy\\\";\" > /etc/apt/apt.conf.d/99proxy &&     apt-get update &&     apt-get install -y --no-install-recommends lsb-release build-essential fakeroot debhelper         wget curl jq equivs gettext-base ca-certificates unzip         devscripts debian-keyring libdistro-info-perl quilt libx11-dev libxinerama-dev libxft-dev libfreetype6-dev pkg-config         git libfuse-dev libopus-dev libpam0g-dev nasm libjpeg-dev libssl-dev libx11-dev libxfixes-dev libxrandr-dev &&     apt-get install -y --no-install-recommends -t stretch-backports dkms &&     touch /etc/dkms/no-autoinstall &&     bash /usr/local/sbin/,preload.sh &&     rm /etc/apt/apt.conf.d/99proxy # buildkit",
            "comment": "buildkit.dockerfile.v0"
        }
    ],
    "os": "linux",
    "rootfs": {
        "type": "layers",
        "diff_ids": [
            "sha256:ef35d4a1050a733d580c43820ac208a458e56c96b8cb7f23dca6aac082125e84",
            "sha256:339c53754499216fcb22c6bf9482fd47ec72a91abd00068ccf41160035df5755",
            "sha256:50c08c6d39ce3a15e24c03affb8536aa10f50613c481a1f63cc067b1a2e19de1",
            "sha256:134a09f0e7934db908e74cc6cf0c17ce829d9037554adb9b3b9b5e45347f3598"
        ]
    },
    "variant": "v5"
}
  • buildah version 1.27: the image correctly contains the variant field:
{
    "created": "2022-09-13T11:01:24.431026221Z",
    "architecture": "arm",
    "variant": "v5",
    "os": "linux",
    "config": {
        "Env": [
            "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
            "DEBIAN_FRONTEND=noninteractive"
        ],
        "Cmd": [
            "bash"
        ],
        "Labels": {
            "io.buildah.version": "1.27.0"
        }
    },
    "rootfs": {
        "type": "layers",
        "diff_ids": [
            "sha256:ef35d4a1050a733d580c43820ac208a458e56c96b8cb7f23dca6aac082125e84",
            "sha256:0003bddf2155fe6b47695b8f2a8905f7180125ffa78af35305a90d42325fc2d5"
        ]
    },
    "history": [
        {
            "created": "2022-06-23T00:55:37.822983329Z",
            "created_by": "/bin/sh -c #(nop) ADD file:4fd9afff1465319b24ccf87076562babe87771f9c1251684705a3896e673aa09 in / "
        },
        {
            "created": "2022-06-23T00:55:38.740692309Z",
            "created_by": "/bin/sh -c #(nop)  CMD [\"bash\"]",
            "empty_layer": true
        },
        {
            "created": "2022-09-13T10:49:15.787795095Z",
            "created_by": "/bin/sh -c #(nop) COPY file:1f46e88741184e55f5401b2e7f1be15be9acf1895f695823f333be1c97666340 in /etc/apt/sources.list ",
            "empty_layer": true
        },
        {
            "created": "2022-09-13T10:49:15.95591604Z",
            "created_by": "/bin/sh -c #(nop) COPY file:84f6b9c2060b6787f2fde3757eab4d267ff648166eb49deffe4a0f6c4017891f in /usr/local/sbin/,preload.sh ",
            "empty_layer": true
        },
        {
            "created": "2022-09-13T10:49:15.955965252Z",
            "created_by": "/bin/sh -c #(nop) ENV DEBIAN_FRONTEND=noninteractive",
            "empty_layer": true
        },
        {
            "created": "2022-09-13T10:49:15.955986651Z",
            "created_by": "/bin/sh -c #(nop) ARG ",
            "empty_layer": true
        },
        {
            "created": "2022-09-13T11:02:42.63150678Z",
            "created_by": "/bin/sh -c echo \"Acquire::http::Proxy \\\"$apt_proxy\\\";\" > /etc/apt/apt.conf.d/99proxy &&     apt-get update &&     apt-get install -y --no-install-recommends lsb-release build-essential fakeroot debhelper         wget curl jq equivs gettext-base ca-certificates unzip         devscripts debian-keyring libdistro-info-perl quilt libx11-dev libxinerama-dev libxft-dev libfreetype6-dev pkg-config         git libfuse-dev libopus-dev libpam0g-dev nasm libjpeg-dev libssl-dev libx11-dev libxfixes-dev libxrandr-dev &&     apt-get install -y --no-install-recommends -t stretch-backports dkms &&     touch /etc/dkms/no-autoinstall &&     bash /usr/local/sbin/,preload.sh &&     rm /etc/apt/apt.conf.d/99proxy",
            "comment": "FROM docker.io/arm32v5/debian:stretch"
        }
    ]
}

The difference between those two is minimal:

--- test1.json	2022-09-13 13:52:41.659596610 +0200
+++ test2.json	2022-09-13 13:52:41.659596610 +0200
@@ -1,5 +1,5 @@
 {
-    "created": "2022-09-13T11:51:01.444941118+02:00",
+    "created": "2022-09-13T11:01:24.431026221Z",
     "architecture": "arm",
     "variant": "v5",
     "os": "linux",
@@ -11,15 +11,15 @@
         "Cmd": [
             "bash"
         ],
-        "OnBuild": null
+        "Labels": {
+            "io.buildah.version": "1.27.0"
+        }
     },
     "rootfs": {
         "type": "layers",
         "diff_ids": [
             "sha256:ef35d4a1050a733d580c43820ac208a458e56c96b8cb7f23dca6aac082125e84",
-            "sha256:339c53754499216fcb22c6bf9482fd47ec72a91abd00068ccf41160035df5755",
-            "sha256:50c08c6d39ce3a15e24c03affb8536aa10f50613c481a1f63cc067b1a2e19de1",
-            "sha256:134a09f0e7934db908e74cc6cf0c17ce829d9037554adb9b3b9b5e45347f3598"
+            "sha256:0003bddf2155fe6b47695b8f2a8905f7180125ffa78af35305a90d42325fc2d5"
         ]
     },
     "history": [
@@ -33,31 +33,29 @@
             "empty_layer": true
         },
         {
-            "created": "2022-09-13T11:40:38.277655491+02:00",
-            "created_by": "COPY stretch.list /etc/apt/sources.list # buildkit",
-            "comment": "buildkit.dockerfile.v0"
+            "created": "2022-09-13T10:49:15.787795095Z",
+            "created_by": "/bin/sh -c #(nop) COPY file:1f46e88741184e55f5401b2e7f1be15be9acf1895f695823f333be1c97666340 in /etc/apt/sources.list ",
+            "empty_layer": true
         },
         {
-            "created": "2022-09-13T11:40:38.424311316+02:00",
-            "created_by": "COPY preload-stretch.sh /usr/local/sbin/,preload.sh # buildkit",
-            "comment": "buildkit.dockerfile.v0"
+            "created": "2022-09-13T10:49:15.95591604Z",
+            "created_by": "/bin/sh -c #(nop) COPY file:84f6b9c2060b6787f2fde3757eab4d267ff648166eb49deffe4a0f6c4017891f in /usr/local/sbin/,preload.sh ",
+            "empty_layer": true
         },
         {
-            "created": "2022-09-13T11:51:01.444941118+02:00",
-            "created_by": "ENV DEBIAN_FRONTEND=noninteractive",
-            "comment": "buildkit.dockerfile.v0",
+            "created": "2022-09-13T10:49:15.955965252Z",
+            "created_by": "/bin/sh -c #(nop) ENV DEBIAN_FRONTEND=noninteractive",
             "empty_layer": true
         },
         {
-            "created": "2022-09-13T11:51:01.444941118+02:00",
-            "created_by": "ARG apt_proxy",
-            "comment": "buildkit.dockerfile.v0",
+            "created": "2022-09-13T10:49:15.955986651Z",
+            "created_by": "/bin/sh -c #(nop) ARG ",
             "empty_layer": true
         },
         {
-            "created": "2022-09-13T11:51:01.444941118+02:00",
-            "created_by": "RUN |1 apt_proxy= /bin/sh -c echo \"Acquire::http::Proxy \\\"$apt_proxy\\\";\" > /etc/apt/apt.conf.d/99proxy &&     apt-get update &&     apt-get install -y --no-install-recommends lsb-release build-essential fakeroot debhelper         wget curl jq equivs gettext-base ca-certificates unzip         devscripts debian-keyring libdistro-info-perl quilt libx11-dev libxinerama-dev libxft-dev libfreetype6-dev pkg-config         git libfuse-dev libopus-dev libpam0g-dev nasm libjpeg-dev libssl-dev libx11-dev libxfixes-dev libxrandr-dev &&     apt-get install -y --no-install-recommends -t stretch-backports dkms &&     touch /etc/dkms/no-autoinstall &&     bash /usr/local/sbin/,preload.sh &&     rm /etc/apt/apt.conf.d/99proxy # buildkit",
-            "comment": "buildkit.dockerfile.v0"
+            "created": "2022-09-13T11:02:42.63150678Z",
+            "created_by": "/bin/sh -c echo \"Acquire::http::Proxy \\\"$apt_proxy\\\";\" > /etc/apt/apt.conf.d/99proxy &&     apt-get update &&     apt-get install -y --no-install-recommends lsb-release build-essential fakeroot debhelper         wget curl jq equivs gettext-base ca-certificates unzip         devscripts debian-keyring libdistro-info-perl quilt libx11-dev libxinerama-dev libxft-dev libfreetype6-dev pkg-config         git libfuse-dev libopus-dev libpam0g-dev nasm libjpeg-dev libssl-dev libx11-dev libxfixes-dev libxrandr-dev &&     apt-get install -y --no-install-recommends -t stretch-backports dkms &&     touch /etc/dkms/no-autoinstall &&     bash /usr/local/sbin/,preload.sh &&     rm /etc/apt/apt.conf.d/99proxy",
+            "comment": "FROM docker.io/arm32v5/debian:stretch"
         }
     ]
 }
\ No newline at end of file

Now the GUI correctly shows armv5:
image


The second problem is due to the tags/list API: on the docker image, a application/vnd.docker.distribution.manifest.v2+json is available, where on the buildah image a application/vnd.docker.distribution.manifest.list.v2+json is returned.

EDIT: I made these tests building for ARMv5 both with docker buildx and buildah. However I just discovered that, while buildah is creating and pushing a manifest list (the so-called "fat manifest"), buildx is creating and pushing a single manifest if there is only one platform involved. See next comment for tests with multiple architectures.

  • Response for tags/list API on the docker buildx image:
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "size": 2369,
      "digest": "sha256:fa6732293b75349f19affbe4c3ece3e37db215a07e96f92d0fc49a087b25046e"
   },
   "layers": [
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 44124653,
         "digest": "sha256:ad86940bdd4f85f3b8d536255a8263fa6320c5075402f2a29b22648c59285219"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 257,
         "digest": "sha256:825e4a382050ed30db6c78dcad2baefe89024cab8e60dde008a64b4d750cf143"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 467,
         "digest": "sha256:94feba32786aa660fab93667b1c49fd81a2457a9f362593f6926543920779007"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 292665524,
         "digest": "sha256:f9a68522f11ec1a40513c30473dad5ba698252eb0c9efcfdce662cf35c24afde"
      }
   ]
}
  • Response for tags/list API on the buildah image:
{
    "schemaVersion": 2,
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "manifests": [
        {
            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
            "size": 595,
            "digest": "sha256:365f075fe2b8b8e29bf1080044a631817f0fa1223f1d346629a3914b0c5751a1",
            "platform": {
                "architecture": "arm",
                "os": "linux",
                "variant": "v5"
            }
        }
    ]
}

@Enrico204
Copy link
Author

Enrico204 commented Sep 13, 2022

As I stated in the previous message "edited part", I did again those tests building for multiple platforms. The result is that, calling the API for getting the image manifest (aka http://localhost:8080/v2/debian-builder-docker/manifests/stretch-v0 or http://localhost:8080/v2/debian-builder-podman/manifests/stretch-v0) for the multi-arch image:

  • If application/vnd.docker.distribution.manifest.list.v2+json not is specified in the Accept request header:
    • For docker buildx images, there is a single manifest in the response
    • For buildah images, an error is returned: MANIFEST_UNKNOWN
  • If application/vnd.docker.distribution.manifest.list.v2+json is specified in the Accept request header:
    • For both docker buildx and buildah images, there is "fat manifest" (manifest list) in the response

So I think that buildah is not generating and pushing a manifest for the actual image tag, but only a "fat manifest"/manifest list. Which I don't know if it's the correct behavior or not. To me, it seems that buildah is right: there is no "single manifest" in a multi-arch image, it's actually a collection of manifests (AFAIK). And I think that docker buildx is pushing also a single manifest for compatibility reasons.

With buildx, I used the command:

docker buildx build --push -f Dockerfile -t 172.17.0.1:5000/debian-builder-docker:stretch-v0 --platform linux/arm/v7,linux/amd64 --output=type=registry,registry.insecure=true .

(the dockerfile from the 1st message is enough for these tests)

@Joxit
Copy link
Owner

Joxit commented Sep 18, 2022

Thank you for your time and your investigation 😄

IMO buildx is right here, when you work with a REST API, you may not return a 404 when the resource exists, you may have a default value when you do not provide any Accept or when any Accept are supported or a specific message instead of MANIFEST_UNKNOWN (since the manifest is known but in another format).

Anyway, this will not solve your issue.... I've already tried a few things that didn't work 😞

BTW, I added your name in my contributors list

@Joxit Joxit added the bug label Sep 18, 2022
@Joxit Joxit added this to the 2.3.0 milestone Sep 18, 2022
@Joxit Joxit closed this as completed in fb81859 Sep 19, 2022
@Enrico204
Copy link
Author

Thanks! :-)

@Joxit
Copy link
Owner

Joxit commented Sep 19, 2022

I pushed a fix, you can try it with the tag docker pull joxit/docker-registry-ui:main

Tell me if it's ok for you 😄

@Enrico204
Copy link
Author

I've updated the image and it's working perfectly :-)

@Joxit
Copy link
Owner

Joxit commented Sep 19, 2022

Great 😄 thank you for your patience ! I will release this in the next 2.3.0 version :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants