Two pushes of the same file at about the same time result in no changes in the etag.

[ogoffart]

Owncloud version: stable6 branch

As shown on this capture, two PUT request done in the same second lead to the same Etag.
(This bug make the client test fail from time to time)

PUT /~owncloud/remote.php/webdav/t1-0736/remoteToLocal1/kernelcrash.txt HTTP/1.1
Content-Type: application/octet-stream
OC-Total-Length: 0
X-OC-Mtime: 1395236757
Authorization: Basic ****=
User-Agent: Mozilla/5.0 (Linux) mirall/1.6.0
Content-Length: 0
Connection: Keep-Alive
Accept-Encoding: gzip, deflate
Accept-Language: en-US,*
Host: localhost



HTTP/1.1 201 Created
Date: Wed, 19 Mar 2014 13:45:57 GMT
Server: Apache/2.2.26 (Unix) mod_ssl/2.2.26 OpenSSL/1.0.1f DAV/2 PHP/5.5.9
X-Powered-By: PHP/5.5.9
Set-Cookie: oca639a7e5c4=supirjc4f7am0fc9i85s5s81j2; path=/~owncloud; HttpOnly
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-OC-MTime: accepted
OC-FileId: 00007464oca639a7e5c4
Content-Length: 0
ETag: "53299f9583680"
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html




PUT /~owncloud/remote.php/webdav/t1-0736/remoteToLocal1/kernelcrash.txt HTTP/1.1
TE: deflate,gzip;q=0.3
Connection: TE, close
Authorization: Basic ****=
Host: localhost:80
User-Agent: DAV.pm/v0.44
Content-Length: 57

This is more stuff
This is more stuff
This is more stuff


HTTP/1.1 204 No Content
Date: Wed, 19 Mar 2014 13:45:57 GMT
Server: Apache/2.2.26 (Unix) mod_ssl/2.2.26 OpenSSL/1.0.1f DAV/2 PHP/5.5.9
X-Powered-By: PHP/5.5.9
Set-Cookie: oca639a7e5c4=th31snq6m7eajm8495cqls0p67; path=/~owncloud; HttpOnly
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
OC-FileId: 00007464oca639a7e5c4
Content-Length: 0
ETag: "53299f9583680"
Connection: close
Content-Type: text/html

[karlitschek]

Hmm. I´m not sure the server guarantees that etags are unique between files. @dragotin What do you think?

[ogoffart]

I Forgot to mention that it is the same file.

[dragotin]

The same etag for two subsequente PUTs on the same file is a bug of death. For two different files, it would probably be ok, but still not nice.

[DeepDiver1975]

The etag is generated using uniqid() - http://php.net/manual/en/function.uniqid.php

And there is a big red warning on the documentation:

Warning

This function does not create random nor unpredictable string. This function must not be used for 
security purposes. Use cryptographically secure random function/generator and cryptographically
 secure hash functions to create unpredictable secure ID.

[DeepDiver1975]

Gets a prefixed unique identifier based on the current time in microseconds. 

[DeepDiver1975]

Does that mean if one executes uniqid() in the same microsecond on two different php processes it can return the same value?

@ogoffart this occurred while testing parallel upload I assume?

[ogoffart]

No, this was running the client tests perl scripts.
the perl script is running some commands to upload files and then we sync.
There must be more than a µs between the two calls.
(about 400ms according to the wireshark timing)

[DeepDiver1975]

in this case I assume a server side issue where we use a not-yet-updated etag

[DeepDiver1975]

I'll take care of this

[DeepDiver1975]

uniqid() has some know limitation - we shall replace it with e.g. \OC_Util::generateRandomBytes(13)

But this is not the root of this issue.

[PVince81]

Not directly related but it reminds me of a time where our unit tests were using the current time() timestamp value and when the machine was too fast, the same timestamp would come out and cause trouble.

Will generateRandomBytes() affect performance in any way (compared to uniqid) ?
I see that it first uses openssl if available, else /dev/urandom (on Linux), else a custom algorithm.

[PVince81]

@DeepDiver1975 did you find further useful information ?

[karlitschek]

@DeepDiver1975 Is this still valid?

[DeepDiver1975]

@ogoffart still an issue? we changed the etag generation function in OC8+ - I'd assume this issue is solved by now.

[PVince81]

@jnfrmarks this should be testable somehow with smashbox (and a bit of timing luck)

[PVince81]

Hopefully solvable with high level file locking: #16237

[MorrisJobke]

Please reopen this if this still happens with high level file locking enabled.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.