Another Reconnect Issue

[pjbrinck]

This issue is similar to but varies from that of #3370.

Im auto-reconnecting to mongodb thru mongoose, but when taking down (killing the process) and bringing mongodb back up again, often (about every fourth time) I don't get an immediate response from mongoose (or mongodb). Instead I only get the response upon a next kill/restart of mongodb.

I have constructed sample code to demonstrate my issue. This code will forever add more records to a mongodb. Upon err to save(), e.g. ECONNRESET, it retries forever until success. It also shows progress and throws an Error if any callback from save() to mongodb was skipped.

These are the steps to reproduce.

1. Build a node.js project and install mongoose and async, copy code below.
2. Let the code run and watch progress.
3. Kill the mongodb process and quickly restart it. Keep doing this several times with 5-10 seconds intervals.
4. Note how most often after restarting mongodb, progress continues. Also note how sometimes after restarting mongodb, progress does NOT continue.
5. When progress does not continue, wait a few minutes to note that it does not otherwise catch up by itself . This is where this issue differs from Reconnect Issue #3370.
6. After a few minutes of waiting, kill the mongodb process and quickly restart it again. Note how after restarting mongodb, progress continues. If not kill/restart the mongodb process repeatedly until progress does continue.

I am running mongodb 3.0 and mongoose ^4.1.8.

What may I be doing wrong? What may mongoose or mongodb not be telling me clearly?

My sample code looks like this:

var async = require('async');
var mongoose = require('mongoose');

var db = mongoose.createConnection();
db.open('mongodb://localhost:27017/test', { 
    server: { auto_connect: true }, 
    socketOptions: { keepAlive: 1, connectTimeoutMS: 30000 }
});

db.on('connected', function() {
    console.log('Now connected to mongo');
});
db.on('error', function(err) {
    console.log('Error using mongo', err);
});
db.on('disconnected', function() {
    console.log('Now disconnected from mongo');
});

var schema = mongoose.Schema({
    number: Number
}, { 
    autoIndex: true
});

var model = db.model('Numbers', schema);

var i = 0;
var prev = 0;
async.forever(function(next) {
    i++;
    async.forever(function(next) {
        new model({
            number: i
        }).save(function(err, result) {
            if (err) {
                console.log(err);
                setTimeout(function() {
                    next(); // continue inner forever - keep trying mongo operation until success 
                }, 10);
            } else {
                console.log(result);
                if (prev != result.number - 1) throw new Error('SEQUENCE NOT FOLLOWED')
                prev = result.number;
                setTimeout(function() {
                    next(true); // break inner forever - mongo operation succeeded
                }, 10);
            }
        }); 
    }, function() {
        next(); // continue outer forever - keep writing to mongo
    });
}, console.log);

[vkarpov15]

This is a pretty deep bug in the mongodb driver (about 2 steps above writing to the socket), and really tricky to repro :( However, looks like we have the general idea of what's wrong in the PR above. Hopefully we'll have a fix soon, thanks for your patience :)

[vkarpov15]

Fixed in 6c98eb4

[ryoqun]

We're are affected this issue too, maybe..

For the record, this is the changes introduced by this update: mongodb/node-mongodb-native@V2.0.48...V2.0.49 mongodb-js/mongodb-core@V1.2.21...V1.2.25

[vkarpov15]

So this issue was fixed in 4.2.7. Have you upgraded?

[pjbrinck]

I have now upgraded and tested and it works nicely... THANKS!!!