litmus cleanup issues with OC storage

[PVince81]

An old classic has returned:

2020-05-08T11:56:04+02:00 ERR home/vincent/Private/Work/workspace/reva/internal/grpc/services/storageprovider/storageprovider.go:391 > error deleting file: path:"/home/home/litmus/ccdest"  error="ocfs: could not restore item: rename /var/tmp/reva/data/einstein/files/home/litmus/ccdest /var/tmp/reva/data/einstein/files_trashbin/files/ccdest.d1588931764: file exists" pkg=rgrpc service=reva traceid=0bca41988c683777979bd62760d724e7

basically: the tests are deleting the folder so fast that it gets the same timestamp value in the trashbin... we never had a solution for this... ref owncloud/core#23151

from the litmus copymove:

docker run -e LITMUS_URL="http://172.17.0.1:9140/remote.php/webdav/home/" -e LITMUS_USERNAME="einstein" -e LITMUS_PASSWORD="relativity" -e TESTS="copymove" owncloud/litmus:latest
-> running `copymove':
 0. init.................. pass
 1. begin................. pass
 2. copy_init............. pass
 3. copy_simple........... pass
 4. copy_overwrite........ pass
 5. copy_nodestcoll....... pass
 6. copy_cleanup.......... pass
 7. copy_coll............. pass
 8. copy_shallow.......... WARNING: Could not delete csrc
WARNING: Could not clean up cdest
    ...................... pass (with 2 warnings)
 9. move.................. FAIL (MOVE `/remote.php/webdav/home/litmus/move' to `/remote.php/webdav/home/litmus/movedest': 500 Internal Server Error)
10. move_coll............. FAIL (collection MOVE `/remote.php/webdav/home/litmus/mvsrc/' to `/remote.php/webdav/home/litmus/mvdest/': 500 Internal Server Error)
11. move_cleanup.......... pass
12. finish................ pass
<- summary for `copymove': of 13 tests run: 11 passed, 2 failed. 84.6%
-> 2 warnings were issued.
See debug.log for network/debug traces.

[PVince81]

ok, my quick workaround idea would be to cheat and make it retry once with the timestamp + 100 ms or so...

[PVince81]

PR here: cs3org/reva#730

[phil-davis]

In Oc10 acceptance test implementation we get around this stuff by making sure to sleep(1) if it has not been 1 second since the last upload or delete or share action:

• quick delete sequences create the trashbin problem mentioned here
• quick repeated upload (upload a file to create it, then upload again with different content to test overwriting, and again...) creates multiple versions (when versions are enabled) and those also do not like to have more than 1 version per second
• creating 2 shares with the same stime means that the order of output when listing "shared with you", "shared by you" can be annoying in tests (some output is ordered by share creation time), so it is easier to make sure the tests do not "accidentally" create 2 shares with the same stime

Those are the things we have done to avoid "random fails" due to this problem. Various ocis tests could have that sort of logic added.

But of course if the system-under-test can be made to correctly handle multiple upload/multiple delete per second, then that is even better.

[PVince81]

@phil-davis in any case I don't want to patch the litmus C code to add sleep there... that code is already extremely old...

[PVince81]

solved in ocis dd985d90b2dd9b4afb25cce1a8947741fe9a4e81