Memory leak when using fetch and not consuming the response body

[mcat95]

Version

v21.4.0

Platform

Linux 6.6.7-arch1-1 #1 SMP PREEMPT_DYNAMIC Thu, 14 Dec 2023 03:45:42 +0000 x86_64 GNU/Linux

Subsystem

fetch API

What steps will reproduce the bug?

Execute the following sample script with node --expose-gc:

const repro = async () => {
  for (let i = 0; i < 1000; i++) {
    await fetch('https://raw.githubusercontent.com/nodejs/node/main/doc/changelogs/CHANGELOG_V20.md');
    console.log(process.memoryUsage().rss);
  }
}

repro()
  .then(() => console.log('Finished'))
  .catch(err => console.log('Error', err));

How often does it reproduce? Is there a required condition?

It always happen

What is the expected behavior? Why is that the expected behavior?

The logged memory usage of the process should be consistent and not increase over every execution of the fetch

What do you see instead?

The memory usage of the process increases with every request, and it's not ever garbage collected (Even with calling global.gc() on every execution.

Additional information

If you add a .then(r => r.text()) or something else that uses the body, the memory leak goes away. I found a similar issue in the node-fetch library (node-fetch/node-fetch#83), but I wasn't able to find this leak in nodejs itself.

For a workaround, I found that using a Finalization Register, I can create a wrapper function that detects evictions of unneeded response objects, and, if the body was unused, empty the body like this:

const registry = new FinalizationRegistry(response => {
  if (!response || response.bodyUsed) return;
  response.arrayBuffer().catch(console.log);
});

[marco-ippolito]

cc @nodejs/undici

[ronag]

This is expected. https://undici.nodejs.org/#/?id=garbage-collection

[cxcorp]

In case anyone else bumps into this, it appears that this behavior has changed and since v20.16.0 the body stream gets closed when the Response is garbage collected. This change was in Undici v6.16.0 via nodejs/undici#3199