Throttle (pause/resume) file upload

[pixelzor]

...Synchronicity for unique file names is killing me.

I'm having a hard time, with the not being able to perform any async operation before: res.onData while receiving a post.

I'm setting up a file upload server, and I need to get a unique filename for my stream in an async way, there's just too much concurrency to do it any other way, is this really not feasible? :(

If I perform an async operation to get fileName, then res.onData just won't perform its callback, like there's nothing there.

const fileName = 'jesusChrist';
let writeStream = fs.createWriteStream(fileName);
 await new Promise((resolve, reject) => {
    const timeout = setTimeout(reject, 60 * 1000);
    writeStream.on('error', reject);
    let byteSize = 0;
      res.onData((chunk, isLast) => {
        byteSize += chunk.byteLength;
        console.log('byteSize', byteSize);
        writeStream.write(Buffer.from(chunk.slice(0)));
        isLast && resolve() && clearTimeout(timeout);
      });
  });`

[uNetworkingAB]

You can pause receiving with https://unetworking.github.io/uWebSockets.js/generated/interfaces/HttpResponse.html#pause.pause-1

But you need to buffer up spurious data either way (up to 512kb but for correctness you need to handle all spurious data). So call onData first, then call pause and when you have your filename call resume then write first your buffered up then from onData.

This library does not hide or fake behavior that would be non-technical so for this reason there still is spurious data like mentioned above (it's impossible to know, efficiently)

[pavulon]

FWIW, if you use this to implement throttling, even if you call .pause() immediately after receiving a chunk, you might still get a bunch more that were already buffered by the system. On Linux, the size of this buffer can be controlled with net.ipv4.tcp_rmem

[uNetworkingAB]

Yes this is the spurious data

[pixelzor]

Gotcha, thanks.