Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add MongoDB replica set support #1196

Merged

Conversation

artiomchi
Copy link
Contributor

What does this PR do?

Adding an option to initialise MongoDb as a single node replica set.

To do this we need a keyfile generated with limited permissions, and add a wait strategy which will initialise the replica set on startup

Why is it important?

Some application would expect the Mongo server to be running in a replica set to improve resiliency, and would set read/write strategies on their commands, which will fail if a replica set is not initialised.

Related issues

There is already an open discussion about it, with some suggestions but no working solution. This PR provides a working implementation and closes #1154

How to test this PR

The single node replica set should behave mostly the same as a single mongo instance.

When running in a replica set, you can also call rs commands to get the status of the replica set.

e.g. the rs.status() command will display the status of the replica set.

Copy link

netlify bot commented Jun 12, 2024

Deploy Preview for testcontainers-dotnet ready!

Name Link
🔨 Latest commit 213f3c4
🔍 Latest deploy log https://app.netlify.com/sites/testcontainers-dotnet/deploys/66cf5dafb364d6000881f6c9
😎 Deploy Preview https://deploy-preview-1196--testcontainers-dotnet.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@artiomchi
Copy link
Contributor Author

The last test run should have been green, but somehow the container has crashed/stopped? Could someone try and re-run the last test run to see if it was an issue in the GitHub runner?

@HofmeisterAn HofmeisterAn added enhancement New feature or request module An official Testcontainers module labels Jun 13, 2024
@artiomchi
Copy link
Contributor Author

@HofmeisterAn I managed to re-run the tests- looks like an intermittent issue in GitHub actions. The tests are green now, and I've been using this code in my integration tests already

Let me know if anything else is needed to have this merged

@HofmeisterAn
Copy link
Collaborator

Thank you for the pull request 🙏. I will try to test and review it in the coming days. I am aware there are a couple of module pull requests waiting to be reviewed. I have not forgotten them; I am addressing them one by one, prioritizing potential bugs and features that benefit the base library.

@artiomchi
Copy link
Contributor Author

No rush, thank you very much @HofmeisterAn!

@artiomchi
Copy link
Contributor Author

Hi, it's been almost two months since the PR was created. Is there anything we can do to help speed up the process to approve and merge this?

Copy link
Collaborator

@HofmeisterAn HofmeisterAn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the late response. Work has been very busy over the last few days, and I just returned from parental leave. I'm trying to address the PRs, but it will take some time. The implementation looks good; I just have a question about whether it makes sense to move the initialization to WithReplicaSet(string). Apologies again.

src/Testcontainers.MongoDb/MongoDbBuilder.cs Outdated Show resolved Hide resolved
@HofmeisterAn HofmeisterAn changed the title Adding MongoDb replica set support feat: Add MongoDB replica set support Aug 28, 2024
Copy link
Collaborator

@HofmeisterAn HofmeisterAn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your patience, the PR, and the contribution. This is truly a great improvement 🚀 that many developers have requested — thank you!

@HofmeisterAn HofmeisterAn merged commit fffd384 into testcontainers:develop Aug 28, 2024
11 checks passed
@Zephyris94
Copy link

Zephyris94 commented Sep 4, 2024

Hi, I am using it now via
_mongoDbContainer = new MongoDbBuilder()
.WithReplicaSet()
.WithImage("mongo:latest")
.WithUsername("")
.WithPassword("")
.WithPortBinding(_mongoPort, true)
.WithWaitStrategy(Wait.ForUnixContainer()
.UntilPortIsAvailable(_mongoPort))
.Build();

The port is: private const int _mongoPort = 27017;

Because I've added transactions to my project, but running it this way makes mongodb fail on startup
the overall issue is
Message: 
System.TimeoutException : A timeout occurred after 30000ms selecting a server using CompositeServerSelector{ Selectors = WritableServerSelector, LatencyLimitingServerSelector{ AllowedLatencyRange = 00:00:00.0150000 }, OperationsCountServerSelector }. Client view of cluster state is { ClusterId : "2", Type : "ReplicaSet", State : "Connected", Servers : [{ ServerId: "{ ClusterId : 2, EndPoint : "127.0.0.1:52699" }", EndPoint: "127.0.0.1:52699", ReasonChanged: "Heartbeat", State: "Connected", ServerVersion: 7.0.0, TopologyVersion: { "processId" : ObjectId("66d8951c02314d40f9304eac"), "counter" : NumberLong(0) }, Type: "ReplicaSetGhost", WireVersionRange: "[0, 21]", LastHeartbeatTimestamp: "2024-09-04T17:13:22.0736177Z", LastUpdateTimestamp: "2024-09-04T17:13:22.0736185Z" }] }.

Version="3.10.0"

Here is a file with full log from start to end. Sadly I am not good enough with mongodb/docker to say what's going wrong with it, I can just state that if I remove the WithReplicaSet() - tests that don't do transactions will go green, and everything works as a charm with this config I stated above
mongodblog.txt

UPD: played around a bit, it didn't work with mongo:4.4 so eventually I ended up with
_mongoDbContainer = new MongoDbBuilder()
.WithImage("mongo:latest")
.WithUsername("")
.WithPassword("")
.WithPortBinding(_mongoPort, true)
.WithCommand("mongod", "--replSet", "rs0", "--bind_ip_all")
.WithWaitStrategy(Wait.ForUnixContainer()
.UntilPortIsAvailable(_mongoPort))
.Build();
and
await _mongoDbContainer.StartAsync();

await _mongoDbContainer.ExecScriptAsync("rs.initiate();");

This worked, and all tests worked (both with and without transactions inside them)

@HofmeisterAn
Copy link
Collaborator

Modules are pre-configured. If you are using them, it is not necessary to override the default configurations, such as wait strategies, etc. Doing so can even be a disadvantage and may break the module. Instead, use:

new MongoDbBuilder().WithReplicaSet().Build();

Override modules only if you are confident in what you are doing.

@Zephyris94
Copy link

it is not necessary to override the default configurations

Didn't know that. Actually this helped and worked, thank you very much, looks much cleaner

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request module An official Testcontainers module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Enhancement]: Add one node replica set support
3 participants