Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

topology was destroyed #4890

Closed
TrejGun opened this issue Jan 11, 2017 · 7 comments
Closed

topology was destroyed #4890

TrejGun opened this issue Jan 11, 2017 · 7 comments
Labels
help wanted needs clarification This issue doesn't have enough information to be actionable. Close after 14 days of inactivity

Comments

@TrejGun
Copy link
Contributor

TrejGun commented Jan 11, 2017

Do you want to request a feature or report a bug?
bug

What is the current behavior?

MongoError: topology was destroyed
at executeCommand (~/node_modules/mongoose/node_modules/mongodb/lib/db.js:289:76)
at Db.command (~/node_modules/mongoose/node_modules/mongodb/lib/db.js:334:44)
at count (~/node_modules/mongoose/node_modules/mongodb/lib/collection.js:2049:13)
at Collection.count (~/node_modules/mongoose/node_modules/mongodb/lib/collection.js:2004:44)
at NativeCollection.(anonymous function) [as count] (~/node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:126:28)
at NodeCollection.count (~/node_modules/mquery/lib/collection/node.js:46:19)
at model.Query.Query._count (~/node_modules/mongoose/lib/query.js:1391:20)
at ~/node_modules/kareem/index.js:239:8
at ~/node_modules/kareem/index.js:18:7
at _combinedTickCallback (internal/process/next_tick.js:67:7)
at process._tickDomainCallback (internal/process/next_tick.js:122:9)
From previous event:
	at model.Query.exec (~/node_modules/mongoose/lib/query.js:2294:17)
at model.Query.Query.then (~/node_modules/mongoose/lib/query.js:2341:15)
at AbstractModel.abstractModelWrapper [as count] (~/server/shared/models/abstract.js:138:5)
at VehicleController.wrapper [as count] (~/server/shared/controllers/abstract.js:107:10)
at paginate (~/server/shared/utils/response.js:5:20)
at VehicleController.list (~/server/office/controllers/vehicle.js:75:10)
at ~/server/shared/routes/wrapper.js:40:3
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at next (~/node_modules/express/lib/router/route.js:131:13)
at ~/server/shared/routes/utils.js:65:11
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at next (~/node_modules/express/lib/router/route.js:131:13)
at Route.dispatch (~/node_modules/express/lib/router/route.js:112:3)
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at ~/node_modules/express/lib/router/index.js:277:22
at Function.process_params (~/node_modules/express/lib/router/index.js:330:12)
at next (~/node_modules/express/lib/router/index.js:271:10)
at Function.handle (~/node_modules/express/lib/router/index.js:176:3)
at router (~/node_modules/express/lib/router/index.js:46:12)
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at trim_prefix (~/node_modules/express/lib/router/index.js:312:13)
at ~/node_modules/express/lib/router/index.js:280:7
at Function.process_params (~/node_modules/express/lib/router/index.js:330:12)
at next (~/node_modules/express/lib/router/index.js:271:10)
at ~/node_modules/express/lib/router/index.js:618:15
From previous event:
	at _passport2.default.deserializeUser (~/server/shared/auth/pathport.js:14:5)
at pass (~/node_modules/passport/lib/authenticator.js:347:9)
at Authenticator.deserializeUser (~/node_modules/passport/lib/authenticator.js:352:5)
at SessionStrategy.authenticate (~/node_modules/passport/lib/strategies/session.js:53:28)
at attempt (~/node_modules/passport/lib/middleware/authenticate.js:348:16)
at authenticate (~/node_modules/passport/lib/middleware/authenticate.js:349:7)
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at trim_prefix (~/node_modules/express/lib/router/index.js:312:13)
at ~/node_modules/express/lib/router/index.js:280:7
at Function.process_params (~/node_modules/express/lib/router/index.js:330:12)
at next (~/node_modules/express/lib/router/index.js:271:10)
at initialize (~/node_modules/passport/lib/middleware/initialize.js:53:5)
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at trim_prefix (~/node_modules/express/lib/router/index.js:312:13)
at ~/node_modules/express/lib/router/index.js:280:7
at Function.process_params (~/node_modules/express/lib/router/index.js:330:12)
at next (~/node_modules/express/lib/router/index.js:271:10)
at ~/node_modules/express-session/index.js:473:7
at Command.callback (~/node_modules/connect-redis/lib/connect-redis.js:160:14)
at normal_reply (~/node_modules/redis/index.js:714:21)
at RedisClient.return_reply (~/node_modules/redis/index.js:816:9)
at JavascriptRedisParser.returnReply (~/node_modules/redis/index.js:188:18)
at JavascriptRedisParser.execute (~/node_modules/redis-parser/lib/parser.js:491:12)
at Socket.<anonymous> (~/node_modules/redis/index.js:267:27)
at emitOne (events.js:96:13)

I'm doing nothing special but having this error at list once a day

Please mention your node.js, mongoose and MongoDB version.
mongoose 4.7.6
➜ node -v
v7.0.0
➜ npm -v
4.0.2
➜ mongo --version
MongoDB shell version v3.4.0

@sobafuchs
Copy link
Contributor

sobafuchs commented Jan 12, 2017

hey @TrejGun I get this sometimes too, usually during my tests because sometimes i forget to close my db connections in my tests. Couple of questions:

Can you give us more info on what environment this is happening in? Can you paste your mongoose configuration and how the connection is established, any options you pass?

@sobafuchs sobafuchs added the needs clarification This issue doesn't have enough information to be actionable. Close after 14 days of inactivity label Jan 12, 2017
@TrejGun
Copy link
Contributor Author

TrejGun commented Jan 16, 2017

on my localhost
I'm not using shards or replication

and using pm2, webpack, babel, react with hot reloading
but none of them can't cause destroyed topology

mongoose.Promise = bluebird.Promise;

const url = "mongodb://localhost/dbName";
const options = {
	server: {
		socketOptions: {
			keepalive: 1
		}
	}
}

const db = mongoose.createConnection();

db.on("disconnected", () => {
	db.open(url, Object.assign({}, options, {promiseLibrary: bluebird}));
});

db.on("error", error => {
	db.close();
});

db.open(url, Object.assign({}, options, {promiseLibrary: bluebird}));

@sobafuchs
Copy link
Contributor

Ok thanks. I was gonna ask if you were running this behind a load balancer but it looks like your db lives on the same instance as your app.

Do you have error logs for your application and can you cross reference them for when your db errors and your app issues db.close().

I wonder if your db is closing while a db call is ongoing or about to happen in your app. For instance, are there any db operations that happen on application startup?

Another theory is that your db error event listener is never triggered and the db maintains an open connection if an error crashes your server. I generally like to handle db connections in a SIGINT or SIGKILL or SIGTERM listener as well.

Either way, this might be difficult to debug unless you have some logs in place.

@TrejGun
Copy link
Contributor Author

TrejGun commented Jan 17, 2017

probably you are right this can be caused by webpack's --watch option which restarts dev server at any time, ie while db call is ongoing

not sure how to handle this gracefully, may be it make sense to add connectTimeoutMS: 30000 to connection options?

here are some more errors

MongoError: topology was destroyed
at executeCommand (~/node_modules/mongoose/node_modules/mongodb/lib/db.js:289:76)
at Db.command (~/node_modules/mongoose/node_modules/mongodb/lib/db.js:334:44)
at Collection.aggregate (~/node_modules/mongoose/node_modules/mongodb/lib/collection.js:2686:13)
at NativeCollection.(anonymous function) [as aggregate] (~/node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:126:28)
at ~/node_modules/mongoose/lib/aggregate.js:576:10
at Aggregate.exec (~/node_modules/mongoose/lib/aggregate.js:562:10)
at Function.aggregate (~/node_modules/mongoose/lib/model.js:2405:13)
at AbstractModel.aggregate (~/server/shared/models/abstract.js:111:10)
at VehicleController.wrapper [as aggregate] (~/server/shared/controllers/abstract.js:107:10)
at VehicleController.typeahead (~/server/office/controllers/vehicle.js:28:15)
at ~/server/shared/routes/wrapper.js:40:3
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at next (~/node_modules/express/lib/router/route.js:131:13)
at Route.dispatch (~/node_modules/express/lib/router/route.js:112:3)
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at ~/node_modules/express/lib/router/index.js:277:22
From previous event:
	at AbstractModel.aggregate (~/server/shared/models/abstract.js:111:10)
at VehicleController.wrapper [as aggregate] (~/server/shared/controllers/abstract.js:107:10)
at VehicleController.typeahead (~/server/office/controllers/vehicle.js:28:15)
at ~/server/shared/routes/wrapper.js:40:3
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at next (~/node_modules/express/lib/router/route.js:131:13)
at Route.dispatch (~/node_modules/express/lib/router/route.js:112:3)
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at ~/node_modules/express/lib/router/index.js:277:22
at Function.process_params (~/node_modules/express/lib/router/index.js:330:12)
at next (~/node_modules/express/lib/router/index.js:271:10)
at Function.handle (~/node_modules/express/lib/router/index.js:176:3)
at router (~/node_modules/express/lib/router/index.js:46:12)
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at trim_prefix (~/node_modules/express/lib/router/index.js:312:13)
at ~/node_modules/express/lib/router/index.js:280:7
at Function.process_params (~/node_modules/express/lib/router/index.js:330:12)
at next (~/node_modules/express/lib/router/index.js:271:10)
at ~/node_modules/express/lib/router/index.js:618:15
at next (~/node_modules/express/lib/router/index.js:256:14)
at Function.handle (~/node_modules/express/lib/router/index.js:176:3)
at router (~/node_modules/express/lib/router/index.js:46:12)
From previous event:
	at _passport2.default.deserializeUser (~/server/shared/auth/pathport.js:14:5)
at pass (~/node_modules/passport/lib/authenticator.js:347:9)
at Authenticator.deserializeUser (~/node_modules/passport/lib/authenticator.js:352:5)
at SessionStrategy.authenticate (~/node_modules/passport/lib/strategies/session.js:53:28)
at attempt (~/node_modules/passport/lib/middleware/authenticate.js:348:16)
at authenticate (~/node_modules/passport/lib/middleware/authenticate.js:349:7)
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at trim_prefix (~/node_modules/express/lib/router/index.js:312:13)
at ~/node_modules/express/lib/router/index.js:280:7
at Function.process_params (~/node_modules/express/lib/router/index.js:330:12)
at next (~/node_modules/express/lib/router/index.js:271:10)
at initialize (~/node_modules/passport/lib/middleware/initialize.js:53:5)
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at trim_prefix (~/node_modules/express/lib/router/index.js:312:13)
at ~/node_modules/express/lib/router/index.js:280:7
at Function.process_params (~/node_modules/express/lib/router/index.js:330:12)
at next (~/node_modules/express/lib/router/index.js:271:10)
at ~/node_modules/express-session/index.js:473:7
at Command.callback (~/node_modules/connect-redis/lib/connect-redis.js:160:14)
at normal_reply (~/node_modules/redis/index.js:714:21)
at RedisClient.return_reply (~/node_modules/redis/index.js:816:9)
at JavascriptRedisParser.returnReply (~/node_modules/redis/index.js:188:18)
at JavascriptRedisParser.execute (~/node_modules/redis-parser/lib/parser.js:491:12)
at Socket.<anonymous> (~/node_modules/redis/index.js:267:27)
at emitOne (events.js:96:13)
MongoError: Topology was destroyed
at nextFunction (~/node_modules/mongoose/node_modules/mongodb-core/lib/cursor.js:541:25)
at Cursor.next [as _next] (~/node_modules/mongoose/node_modules/mongodb-core/lib/cursor.js:693:3)
at nextObject (~/node_modules/mongoose/node_modules/mongodb/lib/cursor.js:671:8)
at Cursor.next (~/node_modules/mongoose/node_modules/mongodb/lib/cursor.js:261:12)
at findOne (~/node_modules/mongoose/node_modules/mongodb/lib/collection.js:1403:10)
at Collection.findOne (~/node_modules/mongoose/node_modules/mongodb/lib/collection.js:1389:44)
at NativeCollection.(anonymous function) [as findOne] (~/node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:126:28)
at NodeCollection.findOne (~/node_modules/mquery/lib/collection/node.js:38:19)
at model.Query.Query.findOne (~/node_modules/mquery/lib/mquery.js:1787:20)
at model.Query.Query._findOne (~/node_modules/mongoose/lib/query.js:1260:22)
at ~/node_modules/kareem/index.js:239:8
at ~/node_modules/kareem/index.js:18:7
at _combinedTickCallback (internal/process/next_tick.js:67:7)
at process._tickDomainCallback (internal/process/next_tick.js:122:9)
From previous event:
	at model.Query.exec (~/node_modules/mongoose/lib/query.js:2294:17)
at AbstractModel.enchant (~/server/shared/models/abstract.js:30:16)
at AbstractModel.findOne (~/server/shared/models/abstract.js:62:15)
at VehicleController.wrapper (~/server/shared/controllers/abstract.js:107:10)
at VehicleController.getByUId (~/server/shared/controllers/stateful.js:20:15)
at VehicleController.getById (~/server/office/controllers/vehicle.js:92:16)
at ~/server/shared/routes/wrapper.js:40:3
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at next (~/node_modules/express/lib/router/route.js:131:13)
at Route.dispatch (~/node_modules/express/lib/router/route.js:112:3)
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at ~/node_modules/express/lib/router/index.js:277:22
at param (~/node_modules/express/lib/router/index.js:349:14)
at paramCallback (~/node_modules/express/lib/router/index.js:401:21)
at ~/server/shared/routes/private.js:15:5
at paramCallback (~/node_modules/express/lib/router/index.js:404:7)
at param (~/node_modules/express/lib/router/index.js:384:5)
at Function.process_params (~/node_modules/express/lib/router/index.js:410:3)
at next (~/node_modules/express/lib/router/index.js:271:10)
at Function.handle (~/node_modules/express/lib/router/index.js:176:3)
at router (~/node_modules/express/lib/router/index.js:46:12)
From previous event:
	at _passport2.default.deserializeUser (~/server/shared/auth/pathport.js:14:5)
at pass (~/node_modules/passport/lib/authenticator.js:347:9)
at Authenticator.deserializeUser (~/node_modules/passport/lib/authenticator.js:352:5)
at SessionStrategy.authenticate (~/node_modules/passport/lib/strategies/session.js:53:28)
at attempt (~/node_modules/passport/lib/middleware/authenticate.js:348:16)
at authenticate (~/node_modules/passport/lib/middleware/authenticate.js:349:7)
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at trim_prefix (~/node_modules/express/lib/router/index.js:312:13)
at ~/node_modules/express/lib/router/index.js:280:7
at Function.process_params (~/node_modules/express/lib/router/index.js:330:12)
at next (~/node_modules/express/lib/router/index.js:271:10)
at initialize (~/node_modules/passport/lib/middleware/initialize.js:53:5)
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at trim_prefix (~/node_modules/express/lib/router/index.js:312:13)
at ~/node_modules/express/lib/router/index.js:280:7
at Function.process_params (~/node_modules/express/lib/router/index.js:330:12)
at next (~/node_modules/express/lib/router/index.js:271:10)
at ~/node_modules/express-session/index.js:473:7
at Command.callback (~/node_modules/connect-redis/lib/connect-redis.js:160:14)
at normal_reply (~/node_modules/redis/index.js:714:21)
at RedisClient.return_reply (~/node_modules/redis/index.js:816:9)
at JavascriptRedisParser.returnReply (~/node_modules/redis/index.js:188:18)
at JavascriptRedisParser.execute (~/node_modules/redis-parser/lib/parser.js:491:12)
at Socket.<anonymous> (~/node_modules/redis/index.js:267:27)
at emitOne (events.js:96:13)
WriteError({"code":11000,"index":0,"errmsg":"E11000 duplicate key error index: DB.users.$email-duplicate dup key: { : \"trejgun@gmail.com\" }","op":{"email":"trejgun@gmail.com"}})
at Function.MongoError.create (~/node_modules/mongoose/node_modules/mongodb-core/lib/error.js:31:11)
at toError (~/node_modules/mongoose/node_modules/mongodb/lib/utils.js:114:22)
at ~/node_modules/mongoose/node_modules/mongodb/lib/collection.js:658:23
at handleCallback (~/node_modules/mongoose/node_modules/mongodb/lib/utils.js:95:56)
at ~/node_modules/mongoose/node_modules/mongodb/lib/bulk/unordered.js:465:9
at handleCallback (~/node_modules/mongoose/node_modules/mongodb/lib/utils.js:95:56)
at resultHandler (~/node_modules/mongoose/node_modules/mongodb/lib/bulk/unordered.js:413:5)
at ~/node_modules/mongoose/node_modules/mongodb-core/lib/connection/pool.js:457:18
at _combinedTickCallback (internal/process/next_tick.js:67:7)
at process._tickDomainCallback (internal/process/next_tick.js:122:9)
From previous event:
	at Function.create (~/node_modules/mongoose/lib/model.js:1852:17)
at AbstractModel.abstractModelWrapper [as create] (~/server/shared/models/abstract.js:137:10)
at UserController.wrapper [as create] (~/server/shared/controllers/abstract.js:107:10)
at UserController.insert (~/server/oauth2/controllers/user.js:80:15)
at ~/server/shared/routes/wrapper.js:40:3
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at next (~/node_modules/express/lib/router/route.js:131:13)
at Route.dispatch (~/node_modules/express/lib/router/route.js:112:3)
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at ~/node_modules/express/lib/router/index.js:277:22
at Function.process_params (~/node_modules/express/lib/router/index.js:330:12)
at next (~/node_modules/express/lib/router/index.js:271:10)
at Function.handle (~/node_modules/express/lib/router/index.js:176:3)
at router (~/node_modules/express/lib/router/index.js:46:12)
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at trim_prefix (~/node_modules/express/lib/router/index.js:312:13)
at ~/node_modules/express/lib/router/index.js:280:7
at Function.process_params (~/node_modules/express/lib/router/index.js:330:12)
at next (~/node_modules/express/lib/router/index.js:271:10)
at ~/node_modules/express/lib/router/index.js:618:15
at next (~/node_modules/express/lib/router/index.js:256:14)
at Function.handle (~/node_modules/express/lib/router/index.js:176:3)
at router (~/node_modules/express/lib/router/index.js:46:12)
at Layer.handle [as handle_request] (~/node_modules/express/lib/router/layer.js:95:5)
at trim_prefix (~/node_modules/express/lib/router/index.js:312:13)

@sobafuchs
Copy link
Contributor

Ah I was under the impression this was happening in your production env too. Is this only in dev mode when you run webpack dev server?

Anyway, you can try increasing the connection timeout, that might work. But this is kind of outside mongoose's scope to handle since throwing an error when you try to make a db call when the db isn't accessible is probably a good thing and it is actually a mongo node driver issue moreso than it is a mongoose issue.

That being said, you could probably easily replicate the bug to confirm by making a query that is guaranteed to take a few seconds and then change some server side code that would force a server restart. If you can make it reproducible, I'd recommend you open an issue on https://github.com/mongodb/node-mongodb-native. Sorry if this seems like a cop out but mongoose doesn't do much in the way of db connection besides simply exposing the underlying driver connection to the db.

@TrejGun
Copy link
Contributor Author

TrejGun commented Jan 19, 2017

@varunjayaraman yes sure, just tell me please who is responsible for reconnection? because this is the main issue, after that Topology was destroyed mongoose can't reconnect and I should manually restart server

@vkarpov15
Copy link
Collaborator

So if mongoose reports a connection error then it's your job to try to reconnect. Mongoose will try to reconnect on its own but if it gets Topology was destroyed then that's up to you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted needs clarification This issue doesn't have enough information to be actionable. Close after 14 days of inactivity
Projects
None yet
Development

No branches or pull requests

3 participants