DockerImageNotFoundException to pull images #666

lucasteles · 2022-11-10T13:41:25Z

Describe the bug
When I am trying to run Testcontainers and it needs to pull a new image it is giving me Docker.DotNet.DockerImageNotFoundException : Docker API responded with status code=NotFound, response={"message":"No such image: testcontainers/ryuk:0.3.4"}

Stack trace:

Setup failed for test fixture Monopoly.Tests.GlobalSetupFixture
Docker.DotNet.DockerImageNotFoundException : Docker API responded with status code=NotFound, response={"message":"No such image: testcontainers/ryuk:0.3.4"}

StackTrace:    at Docker.DotNet.ContainerOperations.<>c.<.cctor>b__30_1(HttpStatusCode statusCode, String responseBody)
   at Docker.DotNet.DockerClient.HandleIfErrorResponseAsync(HttpStatusCode statusCode, HttpResponseMessage response, IEnumerable`1 handlers)
   at Docker.DotNet.DockerClient.MakeRequestAsync(IEnumerable`1 errorHandlers, HttpMethod method, String path, IQueryString queryString, IRequestContent body, IDictionary`2 headers, TimeSpan timeout, CancellationToken token)
   at Docker.DotNet.ContainerOperations.CreateContainerAsync(CreateContainerParameters parameters, CancellationToken cancellationToken)
   at DotNet.Testcontainers.Clients.DockerContainerOperations.RunAsync(ITestcontainersConfiguration configuration, CancellationToken ct)
   at DotNet.Testcontainers.Clients.TestcontainersClient.RunAsync(ITestcontainersConfiguration configuration, CancellationToken ct)
   at DotNet.Testcontainers.Containers.TestcontainersContainer.Create(CancellationToken ct)
   at DotNet.Testcontainers.Containers.TestcontainersContainer.StartAsync(CancellationToken ct)
   at DotNet.Testcontainers.Containers.ResourceReaper.GetAndStartNewAsync(Guid sessionId, IDockerEndpointAuthenticationConfiguration dockerEndpointAuthConfig, String ryukImage, TimeSpan initTimeout, CancellationToken ct)
   at DotNet.Testcontainers.Containers.ResourceReaper.GetAndStartNewAsync(Guid sessionId, IDockerEndpointAuthenticationConfiguration dockerEndpointAuthConfig, String ryukImage, TimeSpan initTimeout, CancellationToken ct)
   at DotNet.Testcontainers.Containers.ResourceReaper.GetAndStartDefaultAsync(IDockerEndpointAuthenticationConfiguration dockerEndpointAuthConfig, CancellationToken ct)
   at DotNet.Testcontainers.Clients.TestcontainersClient.RunAsync(ITestcontainersConfiguration configuration, CancellationToken ct)
   at DotNet.Testcontainers.Containers.TestcontainersContainer.Create(CancellationToken ct)
   at DotNet.Testcontainers.Containers.TestcontainersContainer.StartAsync(CancellationToken ct)
   at Monopoly.Tests.GlobalSetupFixture.OneTimeSetup() in C:\d\a55\Monopoly\tests\Monopoly.Tests.Integration\GlobalSetupFixture.cs:line 19
   at NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter`1.BlockUntilCompleted()
   at NUnit.Framework.Internal.MessagePumpStrategy.NoMessagePumpStrategy.WaitForCompletion(AwaitAdapter awaiter)
   at NUnit.Framework.Internal.AsyncToSyncAdapter.Await(Func`1 invoke)
   at NUnit.Framework.Internal.Commands.SetUpTearDownItem.RunSetUpOrTearDownMethod(TestExecutionContext context, IMethodInfo method)
   at NUnit.Framework.Internal.Commands.SetUpTearDownItem.RunSetUp(TestExecutionContext context)
   at NUnit.Framework.Internal.Commands.OneTimeSetUpCommand.<>c__DisplayClass0_0.<.ctor>b__0(TestExecutionContext context)
   at NUnit.Framework.Internal.Commands.BeforeTestCommand.Execute(TestExecutionContext context)
   at NUnit.Framework.Internal.Commands.BeforeTestCommand.Execute(TestExecutionContext context)
   at NUnit.Framework.Internal.Execution.CompositeWorkItem.PerformOneTimeSetUp()

I am trying to run a Postgres container with:

postgresSql = new TestcontainersBuilder<PostgreSqlTestcontainer>()
    .WithHostname(dbHostName)
    .WithDatabase(dbCredentials)
    .Build();

await postgresSql.StartAsync();

I am getting this with any image:

var container = new TestcontainersBuilder<TestcontainersContainer>()
    .WithImage("hello-world")
    .Build();
await container.StartAsync();

If I pull manually with docker pull testcontainers/ryuk:0.3.4 from the terminal, It starts to give an error on the container image itself

OneTimeSetUp: Docker.DotNet.DockerImageNotFoundException : Docker API responded with status code=NotFound, response={"message":"No such image: hello-world:latest"}

OneTimeSetUp: Docker.DotNet.DockerImageNotFoundException : Docker API responded with status code=NotFound, response={"message":"No such image: postgres:11.14"}

So, it works if I manually pull the hello-world or postgres:11.14.

Desktop:

Version: 2.2.0, 2.1.0
Windows
Docker over WSL2
NET6 / NET7

The text was updated successfully, but these errors were encountered:

HofmeisterAn · 2022-11-10T13:49:42Z

What does TestcontainersSettings.OS.DockerEndpointAuthConfig.Endpoint resolves to? Can you enable the logging (e.g. forward messages to a file)?

lucasteles · 2022-11-10T13:59:25Z

@HofmeisterAn "npipe://./pipe/docker_engine"

HofmeisterAn · 2022-11-10T14:49:45Z

We need the logs, to see if it picks up a Docker registry configuration and if it starts to pull an image at all (log level trace).

lucasteles · 2022-11-10T16:43:29Z

@HofmeisterAn

Using this project https://github.com/lucasteles/nuke-testcontainers/tree/pull-bug

❯ dotnet run --project src
info: Program[0] *********** Begin Test **********
info: Program[0] Searching Docker registry credential in Auths
info: Program[0] Searching Docker registry credential in Auths
info: Program[0] Searching Docker registry credential in CredsStore
info: Program[0] Searching Docker registry credential in CredHelpers
info: Program[0] Docker registry credential index.docker.io found
info: Program[0] Docker image testcontainers/ryuk:0.3.4 created
Unhandled exception. Docker.DotNet.DockerImageNotFoundException: Docker API responded with status code=NotFound, response={"message":"No such image: testcontainers/ryuk:0.3.4"}

   at Docker.DotNet.ContainerOperations.<>c.<.cctor>b__30_1(HttpStatusCode statusCode, String responseBody)
   at Docker.DotNet.DockerClient.HandleIfErrorResponseAsync(HttpStatusCode statusCode, HttpResponseMessage response, IEnumerable`1 handlers)
   at Docker.DotNet.DockerClient.MakeRequestAsync(IEnumerable`1 errorHandlers, HttpMethod method, String path, IQueryString queryString, IRequestContent body, IDictionary`2 headers, TimeSpan timeout, CancellationToken token)
   at Docker.DotNet.ContainerOperations.CreateContainerAsync(CreateContainerParameters parameters, CancellationToken cancellationToken)
   at DotNet.Testcontainers.Clients.DockerContainerOperations.RunAsync(ITestcontainersConfiguration configuration, CancellationToken ct)
   at DotNet.Testcontainers.Clients.TestcontainersClient.RunAsync(ITestcontainersConfiguration configuration, CancellationToken ct)
   at DotNet.Testcontainers.Containers.TestcontainersContainer.Create(CancellationToken ct)
   at DotNet.Testcontainers.Containers.TestcontainersContainer.StartAsync(CancellationToken ct)
   at DotNet.Testcontainers.Containers.ResourceReaper.GetAndStartNewAsync(Guid sessionId, IDockerEndpointAuthenticationConfiguration dockerEndpointAuthConfig, String ryukImage, TimeSpan initTimeout, CancellationToken ct)
   at DotNet.Testcontainers.Containers.ResourceReaper.GetAndStartNewAsync(Guid sessionId, IDockerEndpointAuthenticationConfiguration dockerEndpointAuthConfig, String ryukImage, TimeSpan initTimeout, CancellationToken ct)
   at DotNet.Testcontainers.Containers.ResourceReaper.GetAndStartDefaultAsync(IDockerEndpointAuthenticationConfiguration dockerEndpointAuthConfig, CancellationToken ct)
   at DotNet.Testcontainers.Clients.TestcontainersClient.RunAsync(ITestcontainersConfiguration configuration, CancellationToken ct)
   at DotNet.Testcontainers.Containers.TestcontainersContainer.Create(CancellationToken ct)
   at DotNet.Testcontainers.Containers.TestcontainersContainer.StartAsync(CancellationToken ct)
   at Program.<Main>$(String[] args) in /home/lucasteles/dev/nuke-testcontainers/src/Program.cs:line 33
   at Program.<Main>(String[] args)

after manually pulling ryuk

❯  dotnet run --project .\src\
info: Program[0] *********** Begin Test **********
info: Program[0] Docker container e368a2c7c04dfe29bd0948bd9561a32cfba86ef433d537b479b863e5077af1db created
info: Program[0] Start Docker container e368a2c7c04dfe29bd0948bd9561a32cfba86ef433d537b479b863e5077af1db
info: Program[0] Searching Docker registry credential in CredsStore
info: Program[0] Searching Docker registry credential in CredHelpers
info: Program[0] Searching Docker registry credential in Auths
info: Program[0] Searching Docker registry credential in Auths
info: Program[0] Docker registry credential index.docker.io found
info: Program[0] Docker image postgres:11.14 created
Unhandled exception. Docker.DotNet.DockerImageNotFoundException: Docker API responded with status code=NotFound, response={"message":"No such image: postgres:11.14"}

   at Docker.DotNet.ContainerOperations.<>c.<.cctor>b__30_1(HttpStatusCode statusCode, String responseBody)
   at Docker.DotNet.DockerClient.HandleIfErrorResponseAsync(HttpStatusCode statusCode, HttpResponseMessage response, IEnumerable`1 handlers)
   at Docker.DotNet.DockerClient.MakeRequestAsync(IEnumerable`1 errorHandlers, HttpMethod method, String path, IQueryString queryString, IRequestContent body, IDictionary`2 headers, TimeSpan timeout, CancellationToken token)
   at Docker.DotNet.ContainerOperations.CreateContainerAsync(CreateContainerParameters parameters, CancellationToken cancellationToken)
   at DotNet.Testcontainers.Clients.DockerContainerOperations.RunAsync(ITestcontainersConfiguration configuration, CancellationToken ct)
   at DotNet.Testcontainers.Clients.TestcontainersClient.RunAsync(ITestcontainersConfiguration configuration, CancellationToken ct)
   at DotNet.Testcontainers.Containers.TestcontainersContainer.Create(CancellationToken ct)
   at DotNet.Testcontainers.Containers.TestcontainersContainer.StartAsync(CancellationToken ct)
   at Program.<Main>$(String[] args) in C:\d\nuke-testcontainers\src\Program.cs:line 33
   at Program.<Main>(String[] args)

Is I missing something to enable the Trace level log?

HofmeisterAn · 2022-11-10T17:25:22Z

Somehow it is not pulling images on your machine. Compare your log to my attached run: diagnostic.log. Unfortunately, the current Docker.DotNet version does not forward errors (building and pulling images). Can you run following test on your machine?

using var dockerClientConfiguration = TestcontainersSettings.OS.DockerEndpointAuthConfig.GetDockerClientConfiguration(ResourceReaper.DefaultSessionId);

using var dockerClient = dockerClientConfiguration.CreateClient();

await dockerClient.Images.CreateImageAsync(new ImagesCreateParameters { FromImage = "postgres", Tag = "11.14" }, new AuthConfig(), new Progress())
  .ConfigureAwait(false);

private sealed class Progress : IProgress<JSONMessage>
{
  public void Report(JSONMessage value)
  {
    Debug.WriteLine(value.ProgressMessage);
  }
}

lucasteles · 2022-11-11T12:26:59Z

@HofmeisterAn Your code works, it was able to pull the image 🤔

HofmeisterAn · 2022-11-11T12:37:13Z

Ah, I miss read your log - sorry.

Docker registry credential index.docker.io found

Can you check which credentials Testcontainers for .NET picks up? Looks like you are logged into a registry. I guess Testcontainers cannot authenticate.

lucasteles · 2022-11-11T12:43:41Z

@HofmeisterAn I see, If I log out of my docker hub account in Docker desktop it goes back to work

HofmeisterAn · 2022-11-11T12:51:57Z

Yep, can confirm. This looks like a bug. It picks up the Docker Hub credentials and probably tries to pull the image from the private Docker Hub registry.

HofmeisterAn · 2022-11-11T13:35:22Z

I know what is going "wrong", but I am not sure how to distinguish between private and public images yet (what is the correct approach for Docker Hub). I need to take a closer look at it. I won't make it today, expect a fix next week.

HofmeisterAn · 2022-11-11T18:58:36Z

Maybe this workaround helps in the meantime. Logout and then login via CLI (docker login), using an access token.

HofmeisterAn · 2022-11-11T19:50:47Z

Ok, I found the root cause. Docker.DotNet does not support the Docker Registry v2 Bearer token. Docker Desktop creates a JWT. Following test in Docker.DotNet works:

diff --git a/src/Docker.DotNet/Endpoints/ImageOperations.cs b/src/Docker.DotNet/Endpoints/ImageOperations.cs
index e9412d5..3ccf460 100644
--- a/src/Docker.DotNet/Endpoints/ImageOperations.cs
+++ b/src/Docker.DotNet/Endpoints/ImageOperations.cs
@@ -307,15 +307,18 @@ namespace Docker.DotNet
         {
             return new Dictionary<string, string>
             {
+                // {
+                //     RegistryAuthHeaderKey,
+                //     Convert.ToBase64String(
+                //         Encoding.UTF8.GetBytes(
+                //             this._client.JsonSerializer.SerializeObject(authConfig ?? new AuthConfig())))
+                //     .Replace("/", "_").Replace("+", "-") 
+                //     // This is not documented in Docker API but from source code (https://github.com/docker/docker-ce/blob/10e40bd1548f69354a803a15fde1b672cc024b91/components/cli/cli/command/registry.go#L47)
+                //     // and from multiple internet sources it has to be base64-url-safe. 
+                //     // See RFC 4648 Section 5. Padding (=) needs to be kept.
+                // },
                 {
-                    RegistryAuthHeaderKey,
-                    Convert.ToBase64String(
-                        Encoding.UTF8.GetBytes(
-                            this._client.JsonSerializer.SerializeObject(authConfig ?? new AuthConfig())))
-                    .Replace("/", "_").Replace("+", "-") 
-                    // This is not documented in Docker API but from source code (https://github.com/docker/docker-ce/blob/10e40bd1548f69354a803a15fde1b672cc024b91/components/cli/cli/command/registry.go#L47)
-                    // and from multiple internet sources it has to be base64-url-safe. 
-                    // See RFC 4648 Section 5. Padding (=) needs to be kept.
+                    "Authorization", "Bearer " + authConfig.Password
                 }
             };
         }

The suggested workaround in the previous comment will work. I need to double-check if we can add the bearer token support.

HofmeisterAn · 2022-11-14T11:49:38Z

docker-desktop-credentials returns a JWT for index.docker.io, but not for https://index.docker.io/v1/ (see #5347). The fix will query the Docker registry from the Docker client and passes it to (configuration.Image.GetHostname() ?? ${response_from_docker_client}):

testcontainers-dotnet/src/Testcontainers/Clients/TestcontainersClient.cs

Line 288 in 801ba3c

    
           ? this.registryAuthenticationProvider.GetAuthConfig(configuration.Image.GetHostname()) : configuration.DockerRegistryAuthConfig;

We will keep https://index.docker.io/v1/ as fallback value.

…sktop instead of the CLI

…sktop instead of the CLI (#677)

HofmeisterAn added the question Have you tried our Slack workspace (https://testcontainers.slack.com)? label Nov 10, 2022

HofmeisterAn added the bug Something isn't working label Nov 11, 2022

HofmeisterAn self-assigned this Nov 11, 2022

HofmeisterAn added this to the 2.3.0 milestone Nov 11, 2022

HofmeisterAn added a commit that referenced this issue Nov 16, 2022

fix(#666): DockerImageNotFoundException when logged in with Docker De…

a47202d

…sktop instead of the CLI

HofmeisterAn mentioned this issue Nov 16, 2022

DockerImageNotFoundException when logged in into Docker Desktop #677

Merged

HofmeisterAn added a commit that referenced this issue Nov 16, 2022

fix(#666): DockerImageNotFoundException when logged in with Docker De…

446f002

…sktop instead of the CLI (#677)

HofmeisterAn closed this as completed in #677 Nov 16, 2022

HofmeisterAn added a commit that referenced this issue Nov 21, 2022

fix(#666): DockerImageNotFoundException when logged in with Docker De…

89ff51e

…sktop instead of the CLI (#677)

HofmeisterAn mentioned this issue Dec 23, 2022

Remove path from default registry testcontainers/testcontainers-node#433

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DockerImageNotFoundException to pull images #666

DockerImageNotFoundException to pull images #666

lucasteles commented Nov 10, 2022

HofmeisterAn commented Nov 10, 2022

lucasteles commented Nov 10, 2022

HofmeisterAn commented Nov 10, 2022

lucasteles commented Nov 10, 2022 •

edited

Loading

HofmeisterAn commented Nov 10, 2022

lucasteles commented Nov 11, 2022

HofmeisterAn commented Nov 11, 2022 •

edited

Loading

lucasteles commented Nov 11, 2022 •

edited

Loading

HofmeisterAn commented Nov 11, 2022

HofmeisterAn commented Nov 11, 2022

HofmeisterAn commented Nov 11, 2022 •

edited

Loading

HofmeisterAn commented Nov 11, 2022

HofmeisterAn commented Nov 14, 2022

DockerImageNotFoundException to pull images #666

DockerImageNotFoundException to pull images #666

Comments

lucasteles commented Nov 10, 2022

HofmeisterAn commented Nov 10, 2022

lucasteles commented Nov 10, 2022

HofmeisterAn commented Nov 10, 2022

lucasteles commented Nov 10, 2022 • edited Loading

HofmeisterAn commented Nov 10, 2022

lucasteles commented Nov 11, 2022

HofmeisterAn commented Nov 11, 2022 • edited Loading

lucasteles commented Nov 11, 2022 • edited Loading

HofmeisterAn commented Nov 11, 2022

HofmeisterAn commented Nov 11, 2022

HofmeisterAn commented Nov 11, 2022 • edited Loading

HofmeisterAn commented Nov 11, 2022

HofmeisterAn commented Nov 14, 2022

lucasteles commented Nov 10, 2022 •

edited

Loading

HofmeisterAn commented Nov 11, 2022 •

edited

Loading

lucasteles commented Nov 11, 2022 •

edited

Loading

HofmeisterAn commented Nov 11, 2022 •

edited

Loading