Large upload stuck at 100%, then file is deleted on server

[Jierr]

I am using version 6.2.0

I am uploading a file ~4.9GB to my raspberry Pi. The files are stored to an USB stick (working directory of fileshelter). The max file limit is set to 32GB. I already successfully uploaded a file of size ~600MB.

In the logfile, I think the upload starts at ~Oct 24 22:17:17 and probably finishes around ~Oct 24 23:03:01. Is this a filesystem (NTFS) syncing issue? The download seems to be finished the 4 spawned threads that were active during upload have terminated and cpu load is reducing again. It's just, that I do not see the result on the website and the file is deleted after a few minutes.

fileshelter.log.

The file that is affected is the wtQ5nzHd (snapshot from during the upload process):

rest@raspberrypi:/media/rest/xxx/fileshelter-share/uploaded-files $ du -sch *
2.8M    wt4e9gPn
1.1M    wtknGnX6
592M    wtlPRQRS
180K    wto2STOT
**2.1G    wtQ5nzHd**
2.3M    wttZnCBS
4.0K    wtuuqCQ2
2.7G    total

Do you need additional information to analyze this issue?

Update:

• Seems to be related to Progress stucked on 100% while uploading big files (eg. 6GB) #31 (comment) Unfortunately the file is additionally lost.
• Also getting WebController: timeout: expiring, Wt: signal from dead session, sending reload. messages in logfile

[Jierr]

Further Research

Wt seems to create a copy of that file. Copying the damnt file is incredibly slow:
It seem like the Wt File object is deleted without calling stealSpoolFile, deleting the associated file automatically.

2.8M wt4e9gPn
2.5G wtER6x1m
2.6G wtgkaLLq
1.1M wtknGnX6
592M wtlPRQRS
5.6M wtmuZaFl
180K wto2STOT
2.3M wttZnCBS
4.0K wtuuqCQ2
5.6G total

2.8M wt4e9gPn
1.1M wtknGnX6
592M wtlPRQRS
5.6M wtmuZaFl
180K wto2STOT
2.3M wttZnCBS
4.0K wtuuqCQ2
603M total

• Although copying completes, the file is deleted without any error message in the log (I prolly have to increase the WT Log level)
• I did receive a 502 Error as response though: No change in the webinterface though:
  
• There is also a parameter in the wt config I may need to try out, The config is convoluted so there may be another ineresting param in there...:

            <!-- Session timeout (seconds).

               When a session remains inactive for this amount of time, it is
               cleaned up.
              -->
            <timeout>600</timeout>

[Jierr]

Seems to be a WT problem (https://redmine.emweb.be/boards/2/topics/18414). Which is unfortunate, rendering the purpose of this app ineffective/useless:

There is an proposed workaround, using chunk based uploads for large files.

I researched it by adding additional log messages to fileshelter. The uploaded event of WT is never triggered. Trying version 4.11.0 of WT instead of 4.9.1. Hopefully fileshelter compiles with dat...

---

I wonder if we can steal the spooled file after progress is == 100% and awaiting the secondary copy of the uploaded file. This would be a workaround that is based on polling the copy process.