How to access docker hub using remote package ?

[serhatcetinkaya]

Hello,

I am using following go code to access container registries: https://play.golang.org/p/4lsp5gBiVzA

It works perfectly for ECR and GCR once I have proper entries for them in my ~/.docker/config.json. I've been trying to make the same thing work with docker hub but I always get UNAUTHORIZED error:

➜  REGISTRY="index.docker.io" go run main.go 
error listing repos: GET https://index.docker.io/v2/_catalog?last=&n=100: UNAUTHORIZED: authentication required; [map[Action:* Class: Name:catalog Type:registry]][]
➜  REGISTRY="registry-1.docker.io" go run main.go
error listing repos: GET https://registry-1.docker.io/v2/_catalog?last=&n=100: UNAUTHORIZED: authentication required; [map[Action:* Class: Name:catalog Type:registry]][]

I have proper configurations for both index.docker.io and registry-1.docker.io in my config. when I try to pull a private image for example with docker pull it works fine, but when I try to list images/repos with remote package it fails with the error above. Contents of ~/.docker/config.json:

more .docker/config.json
{
        "auths": {
                "https://index.docker.io/v1/": {
                        "auth": "$BASE64_AUTH_STRING"
                },
                "registry-1.docker.io": {
                        "auth": "$BASE64_AUTH_STRING"
                }
        }
}

docker config is created with following commands:

➜  docker login --username $USERNAME
➜  docker login --username $USERNAME https://registry-1.docker.io

Is there something I am doing wrong ?

[serhatcetinkaya]

@imjasonh
@jonjohnsonjr

Do you have any clue ? Thanks in advance

[serhatcetinkaya]

okay I found this: https://stackoverflow.com/questions/37082826/insufficient-scope-when-attempting-to-get-docker-hub-catalog
quoting:

The catalog endpoint does not work against Docker Hub because that endpoint actually lists all the repositories on a Registry, and we disabled it as it would list all repositories on Docker Hub.

so apparently go-containerregistry doesn't work with docker hub. also looks like following calls are doing what /v2/_catalog does:
https://gist.github.com/kizbitz/175be06d0fbbb39bc9bfa6c0cb0d4721#file-dockerhub-v2-api-organization-sh

Is there any sane way for working with docker hub from go code ?

[jonjohnsonjr]

so apparently go-containerregistry doesn't work with docker hub

I think this is a bit of an overstatement. There are some things that Docker Hub intentionally doesn't support, like catalog and image deletion, which is more of an issue with Docker Hub (and the registry API) than go-containerregistry :P

There is hub-tool, which uses Docker Hub's non-standard APIs to do many interesting things, but most of that code is internal/ as far as I can tell, so you wouldn't be able to import it without forking the project.

I've been attempting to rally support for a standard listing API over in distribution-spec, but as it stands there isn't really a good way to list everything in a repository :/

[serhatcetinkaya]

@jonjohnsonjr sorry I was not specific enough but yes, what I mean was to go-containerregistry doesn't work with docker hub because of docker hub's strange api limitations.

I will try couple of workarounds and I will update this issue in case any of them solves the problem.

thanks for the quick reply

[serhatcetinkaya]

the best solution I could come up was to deploy registry as pull through cache for docker hub and use its endpoint from go code. one thing is, since it is cache /v2/_catalog call doesn't work as soon as you deploy it. you have to pull images at least once to get repositories stored in cache. The other things work as expected.
also this is a great project, thanks everyone for creating and maintaining it :)

[jonjohnsonjr]

also this is a great project, thanks everyone for creating and maintaining it :)

Thanks, really appreciate it ❤️