Memory Leak

[gothammm]

I started using log4js-node-mongodb, to move logs to my mongodb.

Later I started noticing, the increase in the memory consumption, whenever I switch to log4js-node-mongo appender, this is not the same when I use the console appender.

Any help is appreciated.

[4kochi]

We will check this.

[gothammm]

For Example:

I'm trying to hit an end-point, that does nothing but log a single statement - hitting that endpoint with around 200 req/s, the memory consumption shoots up like crazy, while logging onto mongo, but not with console.

[4kochi]

I can reproduce it. Will check if it's a problem with the code of our module or the mongdb driver module.

[gothammm]

👍 Thanks!

[4kochi]

I have done some tests which writes a lot of log entries in the mongo db. The memory usage grows until a certain amount an then the v8 Garbage Collector kicks in. So there is no leak i can found. But at which point the Garbage Collector starts to freeing up memory differs in each node.js version. And in general if there is enough memory in your system then the garbage collection might not even run.

You can find more informations about the garbage collection in node.js here:

nodejs/node#2738
nodejs/node#3370 (comment)

[gothammm]

Can you tell me, to how much did it grow until the GC kicks in?

Also I wanted to know why is there so much growth in memory while writing to db? Any idea?

[4kochi]

On my machine with 8 GB RAM the memory growths up to 1 GB for the node app. Then it was cleaned during the following operations and at the end the memory usage dropped to ~150 MB.

In MongoDB the data is first written to the memory and later to the disk. So if you have a large amount of write operations then the memory increases fast and is freed up later when the data was saved to the disk.