[Bug]: Could not connect to a Mongo DB replica set with new Mongo DB driver 3.0.x

[Yorie]

Testcontainers version

3.10.0

Using the latest Testcontainers version?

Yes

Host OS

Windows 11

Host arch

x64

.NET version

.NET 8

Docker version

Client:
Version: 27.2.0
API version: 1.47
Go version: go1.21.13
Git commit: 3ab4256
Built: Tue Aug 27 14:17:17 2024
OS/Arch: windows/amd64
Context: desktop-linux

Server: Docker Desktop 4.34.2 (167172)
Engine:
Version: 27.2.0
API version: 1.47 (minimum version 1.24)
Go version: go1.21.13
Git commit: 3ab5c7d
Built: Tue Aug 27 14:15:15 2024
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.7.20
GitCommit: 8fc6bcff51318944179630522a095cc9dbf9f353
runc:
Version: 1.1.13
GitCommit: v1.1.13-0-g58aa920
docker-init:
Version: 0.19.0
GitCommit: de40ad0

Docker info

Client:
Version: 27.2.0
Context: desktop-linux
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.16.2-desktop.1
Path: C:\Program Files\Docker\cli-plugins\docker-buildx.exe
compose: Docker Compose (Docker Inc.)
Version: v2.29.2-desktop.2
Path: C:\Program Files\Docker\cli-plugins\docker-compose.exe
debug: Get a shell into any image or container (Docker Inc.)
Version: 0.0.34
Path: C:\Program Files\Docker\cli-plugins\docker-debug.exe
desktop: Docker Desktop commands (Alpha) (Docker Inc.)
Version: v0.0.15
Path: C:\Program Files\Docker\cli-plugins\docker-desktop.exe
dev: Docker Dev Environments (Docker Inc.)
Version: v0.1.2
Path: C:\Program Files\Docker\cli-plugins\docker-dev.exe
extension: Manages Docker extensions (Docker Inc.)
Version: v0.2.25
Path: C:\Program Files\Docker\cli-plugins\docker-extension.exe
feedback: Provide feedback, right in your terminal! (Docker Inc.)
Version: v1.0.5
Path: C:\Program Files\Docker\cli-plugins\docker-feedback.exe
init: Creates Docker-related starter files for your project (Docker Inc.)
Version: v1.3.0
Path: C:\Program Files\Docker\cli-plugins\docker-init.exe
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
Version: 0.6.0
Path: C:\Program Files\Docker\cli-plugins\docker-sbom.exe
scout: Docker Scout (Docker Inc.)
Version: v1.13.0
Path: C:\Program Files\Docker\cli-plugins\docker-scout.exe

Server:
Containers: 7
Running: 5
Paused: 0
Stopped: 2
Images: 17
Server Version: 27.2.0
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 nvidia runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 8fc6bcff51318944179630522a095cc9dbf9f353
runc version: v1.1.13-0-g58aa920
init version: de40ad0
Security Options:
seccomp
Profile: unconfined
Kernel Version: 5.15.153.1-microsoft-standard-WSL2
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 24
Total Memory: 31.22GiB
Name: docker-desktop
ID: 013b6fcb-be8b-43a8-b578-96ca0a4f9fdf
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Labels:
com.docker.desktop.address=npipe://\.\pipe\docker_cli
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5555
127.0.0.0/8
Live Restore Enabled: false

What happened?

Hi,
it seems that latest Mongo DB driver update have some specific changes about replica set connections. With MongoDB driver 3.0.0 it is impossible to connect to test-containers mongo using connection string provided by a test-containers builder.

Here is the test-containers and mongo client initialization:

var builder = new MongoDbBuilder().WithUsername("test").WithPassword("123").WithReplicaSet("rs1").Build();
await builder.StartAsync();
var client = new MongoClient(builder.GetConnectionString());
List<BsonDocument> collections = (await client.ListDatabasesAsync()).ToList();

Attemp to connect throws next exception:

A timeout occurred after 30000ms selecting a server using CompositeServerSelector{ Selectors = ReadPreferenceServerSelector{ ReadPreference = { Mode : Primary } }, LatencyLimitingServerSelector{ AllowedLatencyRange = 00:00:00.0150000 }, OperationsCountServerSelector }. Client view of cluster state is { ClusterId : "1", Type : "ReplicaSet", State : "Disconnected", Servers : [] }.
   at MongoDB.Driver.Core.Clusters.Cluster.ThrowTimeoutException(IServerSelector selector, ClusterDescription description)
   at MongoDB.Driver.Core.Clusters.Cluster.WaitForDescriptionChangedHelper.HandleCompletedTask(Task completedTask)
   at MongoDB.Driver.Core.Clusters.Cluster.WaitForDescriptionChangedAsync(IServerSelector selector, ClusterDescription description, Task descriptionChangedTask, TimeSpan timeout, CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Clusters.Cluster.SelectServerAsync(IServerSelector selector, CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Clusters.IClusterExtensions.SelectServerAndPinIfNeededAsync(IClusterInternal cluster, ICoreSessionHandle session, IServerSelector selector, IReadOnlyCollection`1 deprioritizedServers, CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Bindings.ReadPreferenceBinding.GetReadChannelSourceAsync(IReadOnlyCollection`1 deprioritizedServers, CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Operations.RetryableReadContext.InitializeAsync(CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Operations.RetryableReadContext.CreateAsync(IReadBinding binding, Boolean retryRequested, CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Operations.ReadCommandOperation`1.ExecuteAsync(IReadBinding binding, CancellationToken cancellationToken)
   at MongoDB.Driver.Core.Operations.ListDatabasesOperation.ExecuteAsync(IReadBinding binding, CancellationToken cancellationToken)
   at MongoDB.Driver.OperationExecutor.ExecuteReadOperationAsync[TResult](IReadBinding binding, IReadOperation`1 operation, CancellationToken cancellationToken)
   at MongoDB.Driver.MongoClient.ExecuteReadOperationAsync[TResult](IClientSessionHandle session, IReadOperation`1 operation, CancellationToken cancellationToken)
   at MongoDB.Driver.MongoClient.UsingImplicitSessionAsync[TResult](Func`2 funcAsync, CancellationToken cancellationToken)
...

I found it is possible to fix it in two ways:

1. Rollback to MongoDB driver 2.30.0;
2. Specify directConnection=true option manually in the connection string provided by test-containers builder. While it works fine on my local setup (Docker Desktop + Windows), I found it doesn't work on remote Linux server, throwing MongoDB.Driver.MongoNotPrimaryException : Server returned not primary error.

Either I'm doing something wrong, or it is undesired behavior with test-containers+mongodb:3.0.0

Relevant log output

No response

Additional information

No response

[Yorie]

Additional note: seems this issue is connected to recently posted Jira ticket in the MongoDB project that is yet unresolved.

[HofmeisterAn]

If I understand the issue (Jira ticket) correctly, this is the expected behavior for the new version. Do you know if directConnection=true is available in older driver versions as well? If so, we can simply add it to the connection string.

[Yorie]

Thank you for the response,
I mentioned earlier that, actually, directConnection=true works, but with an issue under Linux OS at list: during execution of db write operations I've got an error MongoDB.Driver.MongoNotPrimaryException : Server returned not primary error..
I cannot describe what commands were executed when I received this error - I have bunch of tests that use test-containers with mongo, but I'm almost sure it is some write operation.
If you need more details, I can investigate further and describe exact environment, test-containers configuration and operation that produces that error.

[Yorie]

Sorry, missed you asked for older version compatibility. Yes, MongoDB.Driver v2.21.0+ has directConnection option. It can be found here.

[HofmeisterAn]

I found it doesn't work on remote Linux server

What do you mean by that? Are you referring to a remote Docker host (container runtime) or a standalone Linux machine? I don't expect any difference regardless of where the container runs.

[maxventer-healthnow]

I have the same issue, with the same problem when running in a gitlab runner. Adding direct connection: true to the connection string solves the issue locally, but then results in a new error in gitlab pipeline MongoDB.Driver.MongoNotPrimaryException : Server returned not primary error.

[HofmeisterAn]

Did you ask the MongoDB team for help and guidance? We can adjust the configuration if we know what is necessary to support this type of environment (I assume both of you are referring to a remote container runtime). It would be interesting to see if we can reproduce the issue using a DinD configuration.

I do not think this is specifically a Testcontainers issue. I would expect the same behavior if you run the container manually and then try to connect.