Skip to content
This repository has been archived by the owner on Apr 2, 2019. It is now read-only.

streaming large files maxes out the call stack #26

Open
epmenard opened this issue Apr 4, 2017 · 1 comment
Open

streaming large files maxes out the call stack #26

epmenard opened this issue Apr 4, 2017 · 1 comment

Comments

@epmenard
Copy link

epmenard commented Apr 4, 2017

Processing files with millions of records result in a crash on maximum call stack exceeded.

I believe returning queueData recursively just adds to the function stack. Deferring the recursive call to the next tick avoids this issue.

` function queueData (data, number) {
...
// If the next value is in the queue, write it
if (writeQueue.hasOwnProperty(nextToWrite)) {
var dataToWrite = writeQueue[nextToWrite]
delete writeQueue[nextToWrite]
process.nextTick(() => {
return queueData(dataToWrite, nextToWrite);
});
}
...

}`

@epmenard
Copy link
Author

epmenard commented Apr 4, 2017

actually turns out my proposition fails.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant