Cannot set mtime in file upload #1248

dpakach · 2020-09-11T03:54:12Z

Steps to reproduce

Upload a file with mtime timestamp for "Thu, 08 Aug 2019 04:18:13 GMT"

curl -XPUT https://135.181.40.111:9200/remote.php/webdav/file.txt einstein:relativity -d "hello" -H "X-OC-Mtime: 1565237893" -vk

Get the mtime with PROPFIND

Expected result

The mtime should match with the one set in the request

Actual result

The mtime is set automatically from upload time

❯ curl -XPROPFIND https://135.181.40.111:9200/remote.php/webdav/file.txt -u einstein:relativity -d '<?xml version="1.0"?>
                <d:propfind
                   xmlns:d="DAV:"
                   xmlns:oc="http://owncloud.org/ns"
                   xmlns:ocs="http://open-collaboration-services.org/ns"
                   > \
                    <d:prop><d:getetag/><d:getlastmodified/></d:prop>
                </d:propfind>'  -k | xmllint --format -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   649  100   415  100   234    387    218  0:00:01  0:00:01 --:--:--   605
<?xml version="1.0" encoding="utf-8"?>
<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:oc="http://owncloud.org/ns">
  <d:response>
    <d:href>/remote.php/webdav/file.txt</d:href>
    <d:propstat>
      <d:prop>
        <d:getetag>"2452963196928:062c0215"</d:getetag>
        <d:getlastmodified>Fri, 11 Sep 2020 03:43:45 +0000</d:getlastmodified>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
</d:multistatus>

The text was updated successfully, but these errors were encountered:

ScharfViktor · 2021-09-01T09:10:02Z

tested with local server oCIS version 1.11.0
The issue is checked: the time is the same. I suggest to close

@dpakach maybe there is a different configuration?

dpakach · 2021-09-02T07:28:54Z

thanks @ScharfViktor for looking into it. Works indeed. We can close this issue.

gennaios · 2023-06-18T00:14:22Z

By any chance might this bug have resurfaced? I’ve started using OCIS 3.0 and using the latest macOS desktop client, all my files show upload time for file date. Full sync hasn’t completed yet so perhaps such is synced after, I’m not sure, though I think if it were syncing date it would be correct on upload.

micbar · 2023-06-19T07:43:46Z

@gennaios Thanks for the report.

@ScharfViktor Can we try to confirm that?

ScharfViktor · 2023-06-19T09:36:29Z

Can confirm that it is relevant again.
We have an api test for that https://github.com/owncloud/ocis/blob/master/tests/acceptance/features/coreApiWebdavUpload1/uploadFile.feature#L221

I'll figure out why the test didn't catch it

michaelstingl · 2023-06-19T10:18:05Z

In web UI, works same as in oC10:

folder ➡️ upload time
files ➡️ mtime ✅

But calculation seems to be done different?

[Log]  ownCloud Web UI 7.0.0  (index.html-55a12d73.mjs, line 7)
[Log]  Infinite Scale 3.1.0-next.1+7a2afee44 Community  (index.html-55a12d73.mjs, line 7)

ScharfViktor · 2023-06-19T10:29:49Z

It happens with uploading zero file so api test was passed

Upload file with zero bytes:
curl -XPUT https://localhost:9200/remote.php/webdav/file1.txt -ueinstein:relativity -H "X-OC-Mtime: 1565237893" -vk

Upload file with content:
curl -XPUT https://localhost:9200/remote.php/webdav/file2.txt -ueinstein:relativity -d "content" -H "X-OC-Mtime: 1565237893" -vk

result:

gennaios · 2023-06-19T10:38:17Z

All my uploads were done with the macOS desktop client.

ScharfViktor · 2023-06-19T10:58:50Z

All my uploads were done with the macOS desktop client.

Yes, and that too

Screen.Recording.2023-06-19.at.12.57.43.mov

michaelstingl · 2023-06-19T13:16:21Z

Desktop client only sends Upload-Metadata: filename:

23-06-19 15:03:47:322 [ info sync.httplogger ]:	"a429b677-89f8-4d3c-a955-605de0331f63: 
Request: POST https://ocis.ocis-traefik.daily.owncloud.works/dav/spaces/0f1c87f7-92e5-4549-8197-c7ece53e2e87$4c510ada-c86b-4815-8820-42cdf82c3d51 
Header: { X-OC-Mtime: 1685710240, 
Content-Type: application/offset+octet-stream, 
Content-Length: 1040524, 
Upload-Offset: 0, 
Tus-Resumable: 1.0.0, 
Upload-Metadata: filename L1Bob3Rvcy0wMDEuemlw,checksum U0hBMSBiYjg4ZTY1ZTRkMzNlZTVhMWNjY2QxMjViNzY2OGY1ZWM3NDlkNjNj, 
Upload-Length: 1040524, 
Authorization: Bearer [redacted], 
User-Agent: Mozilla/5.0 (Macintosh) mirall/5.0.0.11270-daily20230619 (testpilotcloud, 
macos-22.5.0 ClientArchitecture: arm64 OsArchitecture: arm64), 
Accept: */*, 
X-Request-ID: a429b677-89f8-4d3c-a955-605de0331f63, 
Original-Request-ID: a429b677-89f8-4d3c-a955-605de0331f63, 
} 
Data: [1040524 bytes of application/offset+octet-stream data]"

oC web sends more in the 'Upload-Metadata::

curl 'https://ocis.ocis-traefik.daily.owncloud.works/remote.php/dav/spaces/0f1c87f7-92e5-4549-8197-c7ece53e2e87$4c510ada-c86b-4815-8820-42cdf82c3d51' \
-X 'POST' \
-H 'Sec-Fetch-Mode: cors' \
-H 'Referer: https://ocis.ocis-traefik.daily.owncloud.works/files/spaces/personal/einstein?fileId=0f1c87f7-92e5-4549-8197-c7ece53e2e87%244c510ada-c86b-4815-8820-42cdf82c3d51%214c510ada-c86b-4815-8820-42cdf82c3d51&items-per-page=500&view-mode=resource-table&tiles-size=1' \
-H 'Cache-Control: no-cache' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.5 Safari/605.1.15' \
-H 'Host: ocis.ocis-traefik.daily.owncloud.works' \
-H 'Pragma: no-cache' \
-H 'Origin: https://ocis.ocis-traefik.daily.owncloud.works' \
-H 'Sec-Fetch-Dest: empty' \
-H 'Sec-Fetch-Site: same-origin' \
-H 'Content-Length: 1040524' \
-H 'Connection: keep-alive' \
-H 'Authorization: Bearer REDACTED' \
-H 'Accept-Language: en' \
-H 'Accept: */*' \
-H 'Content-Type: application/offset+octet-stream' \
-H 'Accept-Encoding: gzip, deflate, br' \
-H 'Tus-Resumable: 1.0.0' \
-H 'Upload-Length: 1040524' \
-H 'X-Request-ID: ba359508-b4c5-4add-a3bc-c55d53668820' \
-H 'Upload-Metadata: spaceId MGYxYzg3ZjctOTJlNS00NTQ5LTgxOTctYzdlY2U1M2UyZTg3JDRjNTEwYWRhLWM4NmItNDgxNS04ODIwLTQyY2RmODJjM2Q1MQ==,
spaceName UGVyc29uYWw=,
driveAlias cGVyc29uYWwvZWluc3RlaW4=,
driveType cGVyc29uYWw=,
currentFolder Lw==,
currentFolderId MGYxYzg3ZjctOTJlNS00NTQ5LTgxOTctYzdlY2U1M2UyZTg3JDRjNTEwYWRhLWM4NmItNDgxNS04ODIwLTQyY2RmODJjM2Q1MSE0YzUxMGFkYS1jODZiLTQ4MTUtODgyMC00MmNkZjgyYzNkNTE=,
uppyId dXBweS1waG90b3MvMDAyL3ppcC0xZC0xZS1hcHBsaWNhdGlvbi96aXAtMTA0MDUyNC0xNjg1NzEwMjQwMDAw,
relativeFolder ,
relativePath dW5kZWZpbmVk,
tusEndpoint aHR0cHM6Ly9vY2lzLm9jaXMtdHJhZWZpay5kYWlseS5vd25jbG91ZC53b3Jrcy9yZW1vdGUucGhwL2Rhdi9zcGFjZXMvMGYxYzg3ZjctOTJlNS00NTQ5LTgxOTctYzdlY2U1M2UyZTg3JDRjNTEwYWRhLWM4NmItNDgxNS04ODIwLTQyY2RmODJjM2Q1MQ==,
uploadId ZWM1NDdjNjEtM2IzZi00ZjZhLTgzNjgtOTEzMTJlOThiZDU5,
topLevelFolderId dW5kZWZpbmVk,
routeName ZmlsZXMtc3BhY2VzLWdlbmVyaWM=,
routeDriveAliasAndItem cGVyc29uYWwvZWluc3RlaW4=,
routeShareId ,
name UGhvdG9zLTAwMi56aXA=,
type YXBwbGljYXRpb24vemlw,
mtime MTY4NTcxMDI0MA==,
filetype YXBwbGljYXRpb24vemlw,
filename UGhvdG9zLTAwMi56aXA='

According to @micbar 's comment, this was always the case?

x-oc-mtime header is not being respected in tus requests #4559 (comment)

micbar · 2023-06-19T13:49:54Z

I see the two different Protocols

TUS -> reads mtime from Upload-Metadata
Plain PUT - > reads mtime from X-OC-Mtime

my expectation would be that we do either 1 OR 2, no mixed protocol.

@michaelstingl @dragotin @TheOneRing Can I assume that the clients do only One of both protocols, or do we need to expect and handle "mixed" scenarios on the server side?

michaelstingl · 2023-06-19T14:58:44Z

I see the two different Protocols

TUS -> reads mtime from Upload-Metadata

Plain PUT - > reads mtime from X-OC-Mtime

my expectation would be that we do either 1 OR 2, no mixed protocol.

iOS app doesn't send any mtime 🤔

2023-06-19 16:54:32.897000+0200 ownCloud[39393:2447844] [dbug] | [HTTP, Request, …] Sending request:
# REQUEST ---------------------------------------------------------
URL:         https://ocis.ocis-traefik.daily.owncloud.works/dav/spaces/0f1c87f7-92e5-4549-8197-c7ece53e2e87$4c510ada-c86b-4815-8820-42cdf82c3d51/
Error:       -
Req Signals: coreOnline, authAvailable
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
POST /dav/spaces/0f1c87f7-92e5-4549-8197-c7ece53e2e87$4c510ada-c86b-4815-8820-42cdf82c3d51 HTTP/1.1
Host: ocis.ocis-traefik.daily.owncloud.works
Content-Length: 1040524
Tus-Resumable: 1.0.0
X-Request-ID: D29DB304-0371-48F4-967D-D71F3936E489
Original-Request-ID: D29DB304-0371-48F4-967D-D71F3936E489
Content-Type: application/offset+octet-stream
User-Agent: ownCloudApp/12.0.1 (App/268; iPadOS/16.5; iPad)
Authorization: Bearer [redacted:2]
Accept-Language: en-gb,en
Upload-Metadata: filename UGhvdG9zLTAwMy56aXA=,checksum U0hBMSBiYjg4ZTY1ZTRkMzNlZTVhMWNjY2QxMjViNzY2OGY1ZWM3NDlkNjNj
Upload-Length: 1040524
Upload-Offset: 0

[Contents from /Users/michaelstingl/Library/GroupContainersAlias/group.com.owncloud.ios-app/Vaults/03511470-51A1-4F5E-A70A-28FD24144F7C/TUS/C082436E-7899-4754-8539-061D507C2455/BDD9FB3C-8B0A-444E-A613-57012AD1520F (1040524 bytes)]
----------------------------------------------------------------- [… POST, RequestID:D29DB304-0371-48F4-967D-D71F3936E489, URLSessionTaskID:1] [OCHTTPPipeline.m:1183|FULL]

gennaios · 2023-06-19T22:50:28Z

Thanks for looking into it.

Perhaps such would be out of scope yet who knows. Might it also be possible to revisit the issue of folders also having correct mtime? I’m not too familiar with the issue though I have the impression such has been the case for a long time and has never been addressed.

TheOneRing · 2023-06-20T08:13:16Z

I see the two different Protocols

TUS -> reads mtime from Upload-Metadata

Plain PUT - > reads mtime from X-OC-Mtime

my expectation would be that we do either 1 OR 2, no mixed protocol.

iOS app doesn't send any mtime 🤔

2023-06-19 16:54:32.897000+0200 ownCloud[39393:2447844] [dbug] | [HTTP, Request, …] Sending request:
# REQUEST ---------------------------------------------------------
URL:         https://ocis.ocis-traefik.daily.owncloud.works/dav/spaces/0f1c87f7-92e5-4549-8197-c7ece53e2e87$4c510ada-c86b-4815-8820-42cdf82c3d51/
Error:       -
Req Signals: coreOnline, authAvailable
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
POST /dav/spaces/0f1c87f7-92e5-4549-8197-c7ece53e2e87$4c510ada-c86b-4815-8820-42cdf82c3d51 HTTP/1.1
Host: ocis.ocis-traefik.daily.owncloud.works
Content-Length: 1040524
Tus-Resumable: 1.0.0
X-Request-ID: D29DB304-0371-48F4-967D-D71F3936E489
Original-Request-ID: D29DB304-0371-48F4-967D-D71F3936E489
Content-Type: application/offset+octet-stream
User-Agent: ownCloudApp/12.0.1 (App/268; iPadOS/16.5; iPad)
Authorization: Bearer [redacted:2]
Accept-Language: en-gb,en
Upload-Metadata: filename UGhvdG9zLTAwMy56aXA=,checksum U0hBMSBiYjg4ZTY1ZTRkMzNlZTVhMWNjY2QxMjViNzY2OGY1ZWM3NDlkNjNj
Upload-Length: 1040524
Upload-Offset: 0

[Contents from /Users/michaelstingl/Library/GroupContainersAlias/group.com.owncloud.ios-app/Vaults/03511470-51A1-4F5E-A70A-28FD24144F7C/TUS/C082436E-7899-4754-8539-061D507C2455/BDD9FB3C-8B0A-444E-A613-57012AD1520F (1040524 bytes)]
----------------------------------------------------------------- [… POST, RequestID:D29DB304-0371-48F4-967D-D71F3936E489, URLSessionTaskID:1] [OCHTTPPipeline.m:1183|FULL]

Desktop neither https://github.com/owncloud/client/blob/master/src/libsync/propagateuploadtus.cpp#L86-L86

michaelstingl · 2023-06-20T08:51:51Z

Desktop neither https://github.com/owncloud/client/blob/master/src/libsync/propagateuploadtus.cpp#L86-L86

Desktop sends it, but not included in the TUS metadata:

Header: { X-OC-Mtime: 1685710240,

michaelstingl · 2023-06-21T09:30:23Z

Client issues:

micbar · 2023-06-22T11:30:13Z

Result from chat with @TheOneRing and @JammingBen

We need an ADR which covers the stuff that is not expicitly covered by the TUS protocol like the property names in the Upload metadata and which of them are required for oCIS.

micbar · 2023-06-22T11:32:13Z

@kobergj @dragonchaser I put this on our docs board. IMO it is more a DEV team task. But like to track it.

2403905 · 2023-06-23T10:57:04Z

@micbar Are we still have to fix the X-OC-Mtime handing for zero sizes files?

micbar · 2023-06-23T12:26:43Z

Are we still have to fix the X-OC-Mtime handing for zero sizes files?

Yes on PUT Uploads
No on TUS Uploads

2403905 · 2023-07-07T11:38:13Z

The related fix has been merged cs3org/reva#4012
@ScharfViktor Please validate
@micbar FYI

ScharfViktor · 2023-07-07T13:29:08Z

Please validate

I re-tested it on fresh master
Upload file with zero bytes works fine:
curl -XPUT https://localhost:9200/remote.php/webdav/file1.txt -ueinstein:relativity -H "X-OC-Mtime: 1565237893" -vk

Uploading file via macOS desktop client works correctly for me

Should we close ticket?

ScharfViktor · 2023-12-07T11:04:40Z

close as completed

butonic transferred this issue from owncloud/ocis-reva Jan 18, 2021

refs added the Storage:EOS label Jan 18, 2021

refs changed the title ~~[eos] Cannot set mtime in file upload~~ Cannot set mtime in file upload Jan 18, 2021

refs added the Type:Bug label Jan 18, 2021

settings bot removed the Storage:EOS label Jan 29, 2021

exalate-issue-sync bot added the p3-medium label Feb 10, 2021

settings bot removed the p3-medium label Apr 7, 2021

dpakach closed this as completed Sep 2, 2021

micbar reopened this Jun 19, 2023

micbar added Priority:p2-high Escalation, on top of current planning, release blocker Priority:p3-medium Normal priority and removed Priority:p2-high Escalation, on top of current planning, release blocker labels Jun 19, 2023

micbar added this to Infinite Scale Team Board Jun 19, 2023

github-project-automation bot moved this to Qualification in Infinite Scale Team Board Jun 19, 2023

micbar moved this from Qualification to In progress in Infinite Scale Team Board Jun 19, 2023

ScharfViktor mentioned this issue Jun 19, 2023

[test-only][full-ci]ApiTest. added test for uploading zero file with mtime #6552

Merged

This was referenced Jun 20, 2023

[BUG] TUS -H 'Upload-Metadata: mtime missing owncloud/ios-app#1227

Closed

[BUG] TUS -H 'Upload-Metadata: mtime missing owncloud/client#10950

Closed

micbar moved this from In progress to Prio 3 or less in Infinite Scale Team Board Jun 22, 2023

micbar added Topic:Documentation Topic:API labels Jun 22, 2023

2403905 assigned 2403905 and unassigned 2403905 Jun 22, 2023

2403905 moved this from Prio 3 or less to Needs Tests in Infinite Scale Team Board Jul 7, 2023

ScharfViktor moved this from Needs Tests to Done in Infinite Scale Team Board Jul 10, 2023

ScharfViktor closed this as completed Dec 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot set mtime in file upload #1248

Cannot set mtime in file upload #1248

dpakach commented Sep 11, 2020

ScharfViktor commented Sep 1, 2021

dpakach commented Sep 2, 2021

gennaios commented Jun 18, 2023

micbar commented Jun 19, 2023

ScharfViktor commented Jun 19, 2023

michaelstingl commented Jun 19, 2023

ScharfViktor commented Jun 19, 2023 •

edited

Loading

gennaios commented Jun 19, 2023

ScharfViktor commented Jun 19, 2023

michaelstingl commented Jun 19, 2023 •

edited

Loading

micbar commented Jun 19, 2023 •

edited

Loading

michaelstingl commented Jun 19, 2023

gennaios commented Jun 19, 2023

TheOneRing commented Jun 20, 2023

michaelstingl commented Jun 20, 2023

michaelstingl commented Jun 21, 2023

micbar commented Jun 22, 2023

micbar commented Jun 22, 2023

2403905 commented Jun 23, 2023

micbar commented Jun 23, 2023

2403905 commented Jul 7, 2023 •

edited

Loading

ScharfViktor commented Jul 7, 2023

ScharfViktor commented Dec 7, 2023

Cannot set mtime in file upload #1248

Cannot set mtime in file upload #1248

Comments

dpakach commented Sep 11, 2020

Steps to reproduce

Expected result

Actual result

ScharfViktor commented Sep 1, 2021

dpakach commented Sep 2, 2021

gennaios commented Jun 18, 2023

micbar commented Jun 19, 2023

ScharfViktor commented Jun 19, 2023

michaelstingl commented Jun 19, 2023

ScharfViktor commented Jun 19, 2023 • edited Loading

gennaios commented Jun 19, 2023

ScharfViktor commented Jun 19, 2023

michaelstingl commented Jun 19, 2023 • edited Loading

micbar commented Jun 19, 2023 • edited Loading

michaelstingl commented Jun 19, 2023

gennaios commented Jun 19, 2023

TheOneRing commented Jun 20, 2023

michaelstingl commented Jun 20, 2023

michaelstingl commented Jun 21, 2023

micbar commented Jun 22, 2023

micbar commented Jun 22, 2023

2403905 commented Jun 23, 2023

micbar commented Jun 23, 2023

2403905 commented Jul 7, 2023 • edited Loading

ScharfViktor commented Jul 7, 2023

ScharfViktor commented Dec 7, 2023

ScharfViktor commented Jun 19, 2023 •

edited

Loading

michaelstingl commented Jun 19, 2023 •

edited

Loading

micbar commented Jun 19, 2023 •

edited

Loading

2403905 commented Jul 7, 2023 •

edited

Loading