Large files (<2GB) cause user data download to fail/loop leading to extreme slowness and out of heap.

[likewhatevs]

Description:

Steps to reproduce:

Have 1TB+ of free disk space.
Raise file upload limit to 2gb.
Use s3 as backing blob store.
Upload 10GB worth of files (this much might not be necessary) to channel user has access to.
Attempt user data download.

Expected behavior:

User data download works.

Actual behavior:

User data download fails.

Server Setup Information:

• Version of Rocket.Chat Server: 3.12.1
• Operating System: ubuntu
• Deployment Method: docker
• Number of Running Instances: 1
• DB Replicaset Oplog: default
• NodeJS Version: default
• MongoDB Version: 4.4.3

Client Setup Information

• Desktop App or Browser Version: any, backend issue
• Operating System: any, backend issue

Additional context

Deleting the large files and running the following:

#15663 (comment)

and removing the container and redeploying rocket.chat fixes this issue.

Adding some keywords to simplify finding this for when this issue is automatically closed and someone else runs into this issue:
user data download
crashing
extreme slowness
large files
unusably slow
cronProcessDownloads.js
OOM
out of heap

Relevant logs:

Progressive slowness until eventual out of heap error, messages about failed cron.

[tetesh]

UP

[magenbrot]

still not working with version 6.6.1 running on docker

[benedikt-wegmann]

Exports are stuck here, too, additionally there is still no way to "prune" old exports. Version 6.6.7.