Large files (>10MB) on S3 primary storage are lost when uploaded to group folder #48481

infra-monkey · 2024-10-01T05:55:47Z

How to use GitHub

Please use the 👍 reaction to show that you are affected by the same issue.
Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
Subscribe to receive notifications on status change and new comments.

Steps to reproduce

Upload a large file to the group folder
Try to access the file
Get an error saying that the object cannot be found

Expected behaviour

Object should be present

Actual behaviour

I can see that The file is written on the backend

Nextcloud logs show

 Exception Failed to read object urn:oid:769058
Could not get object urn:oid:769058 for file __groupfolders/6/24-01-01 14-59-28 1225.mov 
fopen(httpseek://): Failed to open stream: &quot;OC\Files\Stream\SeekableHttpStream::stream_open&quot; call failed at /var/www/html/lib/private/Files/Stream/SeekableHttpStream.php#49 
fopen(http://s3.empire.lan:9000/nextcloud-data/urn%3Aoid%3A769058): Failed to open stream: HTTP request failed! HTTP/1.1 416 Requested Range Not Satisfiable at /var/www/html/lib/private/Files/ObjectStore/S3ObjectTrait.php#72 
ServiceUnavailable Could not open file

Uploading to a regular user folder works perfectly well.
Moving a large file from a user folder to a group folder looses the file also.
However, copying a large file from a user folder to a group folder works. The file is accessible from the group folder.
Files <10MB don't seem to show the same behavior.

Server configuration

Operating system:
docker container

Web server:
nginx

Database:
postgresql 15

PHP version:
8.2

Nextcloud version: (see Nextcloud admin page)
29.0.7 & 30.0.0

Group folders version:
17.0.4 & 18.0.2

Updated from an older Nextcloud/ownCloud or fresh install:
Updated from older versions

Where did you install Nextcloud from:

Are you using external storage, if yes which one: local/s3/smb/sftp/...
primary storage is S3 minio

Are you using encryption: no

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/Saml/...
OAuth2

Client configuration

Browser:
Firefox/chrome/brave

Operating system:
Linux Leap or Fedora

Logs

No error during upload, but when trying to access the file I get this :

{"reqId":"AH29rbeboMLyOpfwxiEM","level":3,"time":"2024-10-01T05:48:57+00:00","remoteAddr":"10.27.1.1","user":"mattermost-bnrda-ix5z9ksay38488pmnefdehnmer","app":"PHP","method":"GET","url":"/remote.php/dav/files/mattermost-bnrda-ix5z9ksay38488pmnefdehnmer/BNRDA%20Video%20Upload/24-01-01%2014-59-28%201225.mov","message":"fopen(http://s3.empire.lan:9000/nextcloud-data/urn%3Aoid%3A769064): Failed to open stream: HTTP request failed! HTTP/1.1 416 Requested Range Not Satisfiable\r\n at /var/www/html/lib/private/Files/ObjectStore/S3ObjectTrait.php#72","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:130.0) Gecko/20100101 Firefox/130.0","version":"30.0.0.14","data":{"app":"PHP"}}
{"reqId":"s6MMhsgognRjj6U8YDz8","level":3,"time":"2024-10-01T05:48:57+00:00","remoteAddr":"10.27.1.1","user":"mattermost-bnrda-ix5z9ksay38488pmnefdehnmer","app":"PHP","method":"GET","url":"/remote.php/dav/files/mattermost-bnrda-ix5z9ksay38488pmnefdehnmer/BNRDA%20Video%20Upload/24-01-01%2014-59-28%201225.mov","message":"fopen(http://s3.empire.lan:9000/nextcloud-data/urn%3Aoid%3A769064): Failed to open stream: HTTP request failed! HTTP/1.1 416 Requested Range Not Satisfiable\r\n at /var/www/html/lib/private/Files/ObjectStore/S3ObjectTrait.php#72","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:130.0) Gecko/20100101 Firefox/130.0","version":"30.0.0.14","data":{"app":"PHP"}}
{"reqId":"AH29rbeboMLyOpfwxiEM","level":3,"time":"2024-10-01T05:48:57+00:00","remoteAddr":"10.27.1.1","user":"mattermost-bnrda-ix5z9ksay38488pmnefdehnmer","app":"PHP","method":"GET","url":"/remote.php/dav/files/mattermost-bnrda-ix5z9ksay38488pmnefdehnmer/BNRDA%20Video%20Upload/24-01-01%2014-59-28%201225.mov","message":"fopen(httpseek://): Failed to open stream: &quot;OC\\Files\\Stream\\SeekableHttpStream::stream_open&quot; call failed at /var/www/html/lib/private/Files/Stream/SeekableHttpStream.php#49","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:130.0) Gecko/20100101 Firefox/130.0","version":"30.0.0.14","data":{"app":"PHP"}}
{"reqId":"s6MMhsgognRjj6U8YDz8","level":3,"time":"2024-10-01T05:48:57+00:00","remoteAddr":"10.27.1.1","user":"mattermost-bnrda-ix5z9ksay38488pmnefdehnmer","app":"PHP","method":"GET","url":"/remote.php/dav/files/mattermost-bnrda-ix5z9ksay38488pmnefdehnmer/BNRDA%20Video%20Upload/24-01-01%2014-59-28%201225.mov","message":"fopen(httpseek://): Failed to open stream: &quot;OC\\Files\\Stream\\SeekableHttpStream::stream_open&quot; call failed at /var/www/html/lib/private/Files/Stream/SeekableHttpStream.php#49","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:130.0) Gecko/20100101 Firefox/130.0","version":"30.0.0.14","data":{"app":"PHP"}}
{"reqId":"AH29rbeboMLyOpfwxiEM","level":3,"time":"2024-10-01T05:48:57+00:00","remoteAddr":"10.27.1.1","user":"mattermost-bnrda-ix5z9ksay38488pmnefdehnmer","app":"objectstore","method":"GET","url":"/remote.php/dav/files/mattermost-bnrda-ix5z9ksay38488pmnefdehnmer/BNRDA%20Video%20Upload/24-01-01%2014-59-28%201225.mov","message":"Could not get object urn:oid:769064 for file __groupfolders/6/24-01-01 14-59-28 1225.mov","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:130.0) Gecko/20100101 Firefox/130.0","version":"30.0.0.14","exception":{"Exception":"Exception","Message":"Failed to read object urn:oid:769064","Code":0,"Trace":[{"file":"/var/www/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php","line":308,"function":"readObject","class":"OC\\Files\\ObjectStore\\S3","type":"->","args":["urn:oid:769064"]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php","line":283,"function":"fopen","class":"OC\\Files\\ObjectStore\\ObjectStoreStorage","type":"->","args":["__groupfolders/6/24-01-01 14-59-28 1225.mov","r"]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/Jail.php","line":288,"function":"fopen","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["__groupfolders/6/24-01-01 14-59-28 1225.mov","r"]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/Quota.php","line":153,"function":"fopen","class":"OC\\Files\\Storage\\Wrapper\\Jail","type":"->","args":["24-01-01 14-59-28 1225.mov","r"]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php","line":283,"function":"fopen","class":"OC\\Files\\Storage\\Wrapper\\Quota","type":"->","args":["24-01-01 14-59-28 1225.mov","r"]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/PermissionsMask.php","line":97,"function":"fopen","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["24-01-01 14-59-28 1225.mov","r"]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php","line":283,"function":"fopen","class":"OC\\Files\\Storage\\Wrapper\\PermissionsMask","type":"->","args":["24-01-01 14-59-28 1225.mov","r"]},{"file":"/var/www/html/lib/private/Files/View.php","line":1134,"function":"fopen","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["24-01-01 14-59-28 1225.mov","r"]},{"file":"/var/www/html/lib/private/Files/View.php","line":962,"function":"basicOperation","class":"OC\\Files\\View","type":"->","args":["fopen","/BNRDA Video Upload/24-01-01 14-59-28 1225.mov",["read"],"r"]},{"file":"/var/www/html/apps/dav/lib/Connector/Sabre/File.php","line":457,"function":"fopen","class":"OC\\Files\\View","type":"->","args":["BNRDA Video Upload/24-01-01 14-59-28 1225.mov","r"]},{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":85,"function":"get","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->","args":[]},{"file":"/var/www/html/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpGet","class":"Sabre\\DAV\\CorePlugin","type":"->","args":[["Sabre\\HTTP\\Request"],["Sabre\\HTTP\\Response"]]},{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->","args":["method:GET",[["Sabre\\HTTP\\Request"],["Sabre\\HTTP\\Response"]]]},{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->","args":[["Sabre\\HTTP\\Request"],["Sabre\\HTTP\\Response"]]},{"file":"/var/www/html/apps/dav/lib/Server.php","line":370,"function":"start","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/html/apps/dav/appinfo/v2/remote.php","line":19,"function":"exec","class":"OCA\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/html/remote.php","line":146,"args":["/var/www/html/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/html/lib/private/Files/ObjectStore/S3ObjectTrait.php","Line":75,"message":"Could not get object urn:oid:769064 for file __groupfolders/6/24-01-01 14-59-28 1225.mov","exception":{},"CustomMessage":"Could not get object urn:oid:769064 for file __groupfolders/6/24-01-01 14-59-28 1225.mov"}}
{"reqId":"s6MMhsgognRjj6U8YDz8","level":3,"time":"2024-10-01T05:48:57+00:00","remoteAddr":"10.27.1.1","user":"mattermost-bnrda-ix5z9ksay38488pmnefdehnmer","app":"objectstore","method":"GET","url":"/remote.php/dav/files/mattermost-bnrda-ix5z9ksay38488pmnefdehnmer/BNRDA%20Video%20Upload/24-01-01%2014-59-28%201225.mov","message":"Could not get object urn:oid:769064 for file __groupfolders/6/24-01-01 14-59-28 1225.mov","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:130.0) Gecko/20100101 Firefox/130.0","version":"30.0.0.14","exception":{"Exception":"Exception","Message":"Failed to read object urn:oid:769064","Code":0,"Trace":[{"file":"/var/www/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php","line":308,"function":"readObject","class":"OC\\Files\\ObjectStore\\S3","type":"->","args":["urn:oid:769064"]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php","line":283,"function":"fopen","class":"OC\\Files\\ObjectStore\\ObjectStoreStorage","type":"->","args":["__groupfolders/6/24-01-01 14-59-28 1225.mov","r"]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/Jail.php","line":288,"function":"fopen","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["__groupfolders/6/24-01-01 14-59-28 1225.mov","r"]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/Quota.php","line":153,"function":"fopen","class":"OC\\Files\\Storage\\Wrapper\\Jail","type":"->","args":["24-01-01 14-59-28 1225.mov","r"]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php","line":283,"function":"fopen","class":"OC\\Files\\Storage\\Wrapper\\Quota","type":"->","args":["24-01-01 14-59-28 1225.mov","r"]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/PermissionsMask.php","line":97,"function":"fopen","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["24-01-01 14-59-28 1225.mov","r"]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php","line":283,"function":"fopen","class":"OC\\Files\\Storage\\Wrapper\\PermissionsMask","type":"->","args":["24-01-01 14-59-28 1225.mov","r"]},{"file":"/var/www/html/lib/private/Files/View.php","line":1134,"function":"fopen","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["24-01-01 14-59-28 1225.mov","r"]},{"file":"/var/www/html/lib/private/Files/View.php","line":962,"function":"basicOperation","class":"OC\\Files\\View","type":"->","args":["fopen","/BNRDA Video Upload/24-01-01 14-59-28 1225.mov",["read"],"r"]},{"file":"/var/www/html/apps/dav/lib/Connector/Sabre/File.php","line":457,"function":"fopen","class":"OC\\Files\\View","type":"->","args":["BNRDA Video Upload/24-01-01 14-59-28 1225.mov","r"]},{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":85,"function":"get","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->","args":[]},{"file":"/var/www/html/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpGet","class":"Sabre\\DAV\\CorePlugin","type":"->","args":[["Sabre\\HTTP\\Request"],["Sabre\\HTTP\\Response"]]},{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->","args":["method:GET",[["Sabre\\HTTP\\Request"],["Sabre\\HTTP\\Response"]]]},{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->","args":[["Sabre\\HTTP\\Request"],["Sabre\\HTTP\\Response"]]},{"file":"/var/www/html/apps/dav/lib/Server.php","line":370,"function":"start","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/html/apps/dav/appinfo/v2/remote.php","line":19,"function":"exec","class":"OCA\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/html/remote.php","line":146,"args":["/var/www/html/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/html/lib/private/Files/ObjectStore/S3ObjectTrait.php","Line":75,"message":"Could not get object urn:oid:769064 for file __groupfolders/6/24-01-01 14-59-28 1225.mov","exception":{},"CustomMessage":"Could not get object urn:oid:769064 for file __groupfolders/6/24-01-01 14-59-28 1225.mov"}}
{"reqId":"s6MMhsgognRjj6U8YDz8","level":3,"time":"2024-10-01T05:48:57+00:00","remoteAddr":"10.27.1.1","user":"mattermost-bnrda-ix5z9ksay38488pmnefdehnmer","app":"webdav","method":"GET","url":"/remote.php/dav/files/mattermost-bnrda-ix5z9ksay38488pmnefdehnmer/BNRDA%20Video%20Upload/24-01-01%2014-59-28%201225.mov","message":"Could not open file","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:130.0) Gecko/20100101 Firefox/130.0","version":"30.0.0.14","exception":{"Exception":"Sabre\\DAV\\Exception\\ServiceUnavailable","Message":"Could not open file","Code":0,"Trace":[{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":85,"function":"get","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->","args":[]},{"file":"/var/www/html/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpGet","class":"Sabre\\DAV\\CorePlugin","type":"->","args":[["Sabre\\HTTP\\Request"],["Sabre\\HTTP\\Response"]]},{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->","args":["method:GET",[["Sabre\\HTTP\\Request"],["Sabre\\HTTP\\Response"]]]},{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->","args":[["Sabre\\HTTP\\Request"],["Sabre\\HTTP\\Response"]]},{"file":"/var/www/html/apps/dav/lib/Server.php","line":370,"function":"start","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/html/apps/dav/appinfo/v2/remote.php","line":19,"function":"exec","class":"OCA\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/html/remote.php","line":146,"args":["/var/www/html/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/html/apps/dav/lib/Connector/Sabre/File.php","Line":463,"message":"Could not open file","exception":{},"CustomMessage":"Could not open file"}}
{"reqId":"AH29rbeboMLyOpfwxiEM","level":3,"time":"2024-10-01T05:48:57+00:00","remoteAddr":"10.27.1.1","user":"mattermost-bnrda-ix5z9ksay38488pmnefdehnmer","app":"webdav","method":"GET","url":"/remote.php/dav/files/mattermost-bnrda-ix5z9ksay38488pmnefdehnmer/BNRDA%20Video%20Upload/24-01-01%2014-59-28%201225.mov","message":"Could not open file","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:130.0) Gecko/20100101 Firefox/130.0","version":"30.0.0.14","exception":{"Exception":"Sabre\\DAV\\Exception\\ServiceUnavailable","Message":"Could not open file","Code":0,"Trace":[{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":85,"function":"get","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->","args":[]},{"file":"/var/www/html/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpGet","class":"Sabre\\DAV\\CorePlugin","type":"->","args":[["Sabre\\HTTP\\Request"],["Sabre\\HTTP\\Response"]]},{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->","args":["method:GET",[["Sabre\\HTTP\\Request"],["Sabre\\HTTP\\Response"]]]},{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->","args":[["Sabre\\HTTP\\Request"],["Sabre\\HTTP\\Response"]]},{"file":"/var/www/html/apps/dav/lib/Server.php","line":370,"function":"start","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/html/apps/dav/appinfo/v2/remote.php","line":19,"function":"exec","class":"OCA\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/html/remote.php","line":146,"args":["/var/www/html/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/html/apps/dav/lib/Connector/Sabre/File.php","Line":463,"message":"Could not open file","exception":{},"CustomMessage":"Could not open file"}}

Web server error log

Insert your webserver log here

Nextcloud log (data/nextcloud.log)

Nextcloud log

Insert your Nextcloud log here

Browser log

Insert your browser log here, this could for example include:

a) The javascript console log
b) The network log
c) ...

The text was updated successfully, but these errors were encountered:

provokateurin · 2024-10-01T06:13:45Z

Not groupfolder specific, this is an S3 issue.

fcharlaix-opendsi · 2024-10-01T09:20:27Z

I have the same issue with Nextcloud 29.0.7 on Debian 12, PHP 8.2 without Docker, using OVH Object Storage (their backend is not MinIO as far as I know).

On a Nextcloud 28.0.9 with Group Folders 16.0.10 and MinIO, the issue is not present.

kesselb · 2024-10-01T12:28:10Z

@fcharlaix-opendsi would you mind opening a support ticket?

If the problem occurs with 29.0.7, but not 28.0.9 then a relevant change is: #46013

A way to test that could be to comment the moveFromStorage method in ObjectStoreStorage to use, as before the patch, just the default implemention. Note this will reintroduce the issue that were fixed by it.

infra-monkey · 2024-10-01T14:13:11Z

@kesselb in v30.0.0 commenting out the method and restarting the container seems to fix this issue.
I'll do more thorough tests tonight

Mika-6-9 · 2024-10-04T06:43:05Z

Hi @infra-monkey
Have you been able to do more tests?
And especially conclusive tests? 😄

Thanks

infra-monkey · 2024-10-04T09:57:56Z

@Mika-6-9 I have uploaded successfully 40+ big files since. No issue.
As I didn't have any problem prior to updating to 29.0.7, the change seems indeed to be the culprit in my case.
I don't use other external storage so I was not impacted.

Any particular use case to test?

infra-monkey · 2024-10-04T10:39:51Z

I still have a weird issue.
Upload file to root folder. Then move it to a group folder.
I get an error notification but the file is available in the group folder. After the move, the file was still listed in the root folder and disappeared after a refresh.
No problem with a copy tho.
The error is:

Error	webdav	
TypeError
OCA\Files_Versions\Listener\VersionStorageMoveListener::handleMoveOrCopy(): Argument #3 ($source) must be of type OCP\Files\File, null given, called in /var/www/html/apps/files_versions/lib/Listener/VersionStorageMoveListener.php on line 101
Error	PHP	
Undefined array key 769579 at /var/www/html/apps/files_versions/lib/Listener/VersionStorageMoveListener.php#97

moving between subfolders of the same group folder works correctly.
moving from the group folder to the group folder fails in the same way.
moving from a group folder to another group folder works also.

Mika-6-9 · 2024-10-07T17:33:05Z

Thanks for your feedback, but I'm not sure this is viable for a production release.

infra-monkey · 2024-10-08T05:17:36Z

Not sure to understand what you mean. Thanks for your feedback, but I'm not sure this is viable for a production release. — Reply to this email directly, view it on GitHub , or unsubscribe . You are receiving this because you were mentioned. Message ID: <nextcloud/server/issues/48481/2397508653 @ github . com>

PeteKersker · 2024-10-09T18:49:42Z

My setup includes:
Nextcloud: 29.0.7
Group Folders: 17.0.3
OS: Ubuntu 22.04
Webserver: Apache
PHP: 8.2
Storage: S3 (Primary)

My errors (after upgrade from Nextcloud 28.0.9) include:

@infra-monkey's original issue at the top:
large file upload to a group folder appears to work, but the file cannot be accessed because could not get object urn:oid
More Information: the same error occurs with a direct (web browser) upload or windows app sync upload.
More Information: the same large file uploads and is accessible in the user's root directory (which is S3 storage)
More Information: small files upload and are accessible in the group folder
@infra-monkey's wierd issue five days ago:
a large file upload to the user's root directory works and is accessible. Moving it to a group folder fails.
More Information: Copying the file from the root directory to a group folder works correctly and is accessible.

I'm willing to test any potential fixes with my configuration if that would help.

PeteKersker · 2024-10-12T13:49:18Z

If the problem occurs with 29.0.7, but not 28.0.9 then a relevant change is: #46013

A way to test that could be to comment the moveFromStorage method in ObjectStoreStorage to use, as before the patch, just the default implemention. Note this will reintroduce the issue that were fixed by it.

I ran the test you suggested and commented out the changes from #46013. Here are my results:
Large File Upload to Group Folder (Sync): Success
Large File Upload to Group Folder (Browser): Success
Large File Upload to Group Folder (Android App): Success
Large File Move from Group Folder to Group Folder (Browser): Success
Large File Copy from Group Folder to Group Folder (Browser): Success

Large File Upload to User Root Directory (Sync): Success (as before)
Large File Move from Root to Group Folder (Browser): Success (but Notification reports Fail).
Large File Copy from Root to Group Folder (Browser): Success (as before but now Notification reports Fail).

I'm leaving this code commented out on my server for now, since the incorrect notification is not a breaking change.
Let me know if there's any other way I can help with testing.

PeteKersker · 2024-10-14T22:23:34Z

Update: I just upgraded to Nextcloud 29.0.8 and ran most of the tests above. It is all working correctly without any manual code changes on my part. I no longer need to comment out the #46013 code!

infra-monkey · 2024-10-19T15:47:58Z

Seems to be fixed in 30.0.1 also.

infra-monkey added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels Oct 1, 2024

provokateurin transferred this issue from nextcloud/groupfolders Oct 1, 2024

szaimen added the 29-feedback label Oct 1, 2024

joshtrichards added the feature: object storage label Oct 1, 2024

Mika-6-9 mentioned this issue Oct 3, 2024

[Bug]: "Connection closed" message when syncing files larger then +- 10 Mb nextcloud/desktop#7257

Closed

8 tasks

joshtrichards added feature: filesystem feature: versions integration labels Oct 12, 2024

infra-monkey closed this as completed Oct 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Large files (>10MB) on S3 primary storage are lost when uploaded to group folder #48481

Large files (>10MB) on S3 primary storage are lost when uploaded to group folder #48481

infra-monkey commented Oct 1, 2024 •

edited by joshtrichards

Loading

provokateurin commented Oct 1, 2024

fcharlaix-opendsi commented Oct 1, 2024 •

edited

Loading

kesselb commented Oct 1, 2024

infra-monkey commented Oct 1, 2024

Mika-6-9 commented Oct 4, 2024

infra-monkey commented Oct 4, 2024

infra-monkey commented Oct 4, 2024 •

edited

Loading

Mika-6-9 commented Oct 7, 2024

infra-monkey commented Oct 8, 2024 via email

PeteKersker commented Oct 9, 2024 •

edited

Loading

PeteKersker commented Oct 12, 2024 •

edited

Loading

PeteKersker commented Oct 14, 2024 •

edited

Loading

infra-monkey commented Oct 19, 2024

Large files (>10MB) on S3 primary storage are lost when uploaded to group folder #48481

Large files (>10MB) on S3 primary storage are lost when uploaded to group folder #48481

Comments

infra-monkey commented Oct 1, 2024 • edited by joshtrichards Loading

How to use GitHub

Steps to reproduce

Expected behaviour

Actual behaviour

Server configuration

Client configuration

Logs

Web server error log

Nextcloud log (data/nextcloud.log)

Browser log

provokateurin commented Oct 1, 2024

fcharlaix-opendsi commented Oct 1, 2024 • edited Loading

kesselb commented Oct 1, 2024

infra-monkey commented Oct 1, 2024

Mika-6-9 commented Oct 4, 2024

infra-monkey commented Oct 4, 2024

infra-monkey commented Oct 4, 2024 • edited Loading

Mika-6-9 commented Oct 7, 2024

infra-monkey commented Oct 8, 2024 via email

PeteKersker commented Oct 9, 2024 • edited Loading

PeteKersker commented Oct 12, 2024 • edited Loading

PeteKersker commented Oct 14, 2024 • edited Loading

infra-monkey commented Oct 19, 2024

infra-monkey commented Oct 1, 2024 •

edited by joshtrichards

Loading

fcharlaix-opendsi commented Oct 1, 2024 •

edited

Loading

infra-monkey commented Oct 4, 2024 •

edited

Loading

PeteKersker commented Oct 9, 2024 •

edited

Loading

PeteKersker commented Oct 12, 2024 •

edited

Loading

PeteKersker commented Oct 14, 2024 •

edited

Loading