Folder move not propagated to server when any error occurs during the sync

[gabi18]

Seen with client ownCloud-2.10.0-rc1.6315.x64.msi on Windows 10 20H2

Steps to reproduce:

• On server: create a subfolder EmptyFolder inside folder Test-Data, create 2 files having a file name clash
  Result: the client shows red error message "...file name clash..." for a while then shows the 'i' icon, 'Not synced' tab shows the file clash. The Explorer icon of file NeuesFile.txt and folder Test-Data remains spinning.

• move EmptyFolder to a new location (outside snyc root)
  Result: the folder is moved locally but is not deleted on server

...
20211220_1822_owncloud.log.0:12-20 18:23:43:182 [ warning sync.vfs.win ]:       void __cdecl OCC::VfsWin::fileStatusChanged(const class QString &,class OCC::SyncFileStatus) : couldn't CfSetInSyncState for "C:/Users/gabi/ownCloud26/Test-Data/EmptyFolder"  status  OCC::SyncFileStatus::StatusSync : "WindowsError: ffffffff80070178: Die Datei ist keine Clouddatei."
20211220_1822_owncloud.log.0:12-20 18:23:45:205 [ debug sync.localdiscoverytracker ]    [ OCC::LocalDiscoveryTracker::startSyncPartialDiscovery ]:      partial discovery with paths:  ("Test-Data", "Test-Data/123344888889999999999", "Test-Data/123344888889999999999/Neues Textdokument.txt", "Test-Data/EmptyFolder")
20211220_1822_owncloud.log.0:12-20 18:23:45:429 [ info sync.discovery ]:        Processing "Test-Data/EmptyFolder" | valid: false/true/false | mtime: 0/1640014730/0 | size: 0/0/0 | etag: ""//"" | checksum: ""//"" | perm: ""//"" | fileid: ""//"" | inode: 0/621914/ | type: CSyncEnums::ItemTypeSkip/CSyncEnums::ItemTypeDirectory/CSyncEnums::ItemTypeFile
20211220_1822_owncloud.log.0:12-20 18:23:45:429 [ info sync.discovery ]:        Discovered "Test-Data/EmptyFolder" SyncInstruction(CSYNC_INSTRUCTION_NEW) OCC::SyncFileItem::Up CSyncEnums::ItemTypeDirectory
20211220_1822_owncloud.log.0:12-20 18:23:45:429 [ info sync.discovery ]:        STARTING "Test-Data/EmptyFolder" OCC::ProcessDirectoryJob::ParentDontExist "Test-Data/EmptyFolder" OCC::ProcessDirectoryJob::NormalQuery
20211220_1822_owncloud.log.0:12-20 18:23:45:429 [ debug sync.database.sql ]     [ OCC::SqlQuery::bindValue ]:   SQL bind 1 "Test-Data/EmptyFolder"
20211220_1822_owncloud.log.0:12-20 18:23:45:429 [ debug sync.statustracker ]    [ OCC::SyncFileStatusTracker::slotAboutToPropagate ]:   Investigating "Test-Data/EmptyFolder" OCC::SyncFileItem::NoStatus SyncInstruction(CSYNC_INSTRUCTION_NEW)
20211220_1822_owncloud.log.0:12-20 18:23:45:429 [ warning sync.vfs.win ]:       void __cdecl OCC::VfsWin::fileStatusChanged(const class QString &,class OCC::SyncFileStatus) : couldn't CfSetInSyncState for "C:/Users/gabi/ownCloud26/Test-Data/EmptyFolder"  status  OCC::SyncFileStatus::StatusSync : "WindowsError: ffffffff80070178: Die Datei ist keine Clouddatei."
...
20211220_1827_owncloud.log.1:12-20 18:29:32:575 [ warning sync.filesystem ]:    Could not get modification time for "C:/Users/gabi/ownCloud26/Test-Data/EmptyFolder" with csync, using QFileInfo: -3600
20211220_1827_owncloud.log.1:12-20 18:29:34:628 [ debug sync.localdiscoverytracker ]    [ OCC::LocalDiscoveryTracker::startSyncPartialDiscovery ]:      partial discovery with paths:  ("Test-Data/EmptyFolder", "Test-Data/NeuesFile.txt", "Test-Data/Neuesfile.txt")

20211220_1821_owncloud.log.1.gz
20211220_1822_owncloud.log.0.gz
20211220_1823_owncloud.log.0.gz
20211220_1823_owncloud.log.1.gz
20211220_1826_owncloud.log.0.gz
20211220_1826_owncloud.log.1.gz
20211220_1827_owncloud.log.0.gz
20211220_1827_owncloud.log.1.gz

[gabi18]

After deleting the file Newfile.txt which causes the clash on server, the client automatically resyncs, shows the green check mark and the message disappears from 'Not synced' tab. Also EmptyFolder is deleted on server -> OK

20211221_1100_owncloud.log.0.gz
20211221_1220_owncloud.log.0.gz

The icon for Test-Data and NewFile.txt isn't shown correctly, also not after 'Force sync', reopen the Explorer. Edit, save/exit the file let the file icon change to full file but icon for folder is still circling -> NOT ok

Test-Data icon finally changed to cloud icon after 'Free up space' for the file NewFile.txt is done -> OK

[TheOneRing]

Does that only occur with empty folders?

[TheOneRing]

Also this seems to be unrelated to a case clash

[TheOneRing]

12-22 13:30:11:652 [ info sync.discovery ]:	STARTING "New folder/aaaaaa" OCC::ProcessDirectoryJob::ParentNotChanged "New folder/aaaaaa" OCC::ProcessDirectoryJob::ParentDontExist

Looks like we get OCC::ProcessDirectoryJob::ParentDontExist for the folder it self and probably don't propagate the removal of aaaaaa to the server.

[gabi18]

Tested removal of an empty folder without having a clash with ownCloud-2.10.0-rc3.6417.x64.msi - VFS ON

• create empty subfolder 'Emptyfolder' in 'Test-Data' folder on the server
• move 'Empytfolder' to another location outside the sync root in Explorer

Result: 'Emptyfolder' is moved locally and deleted on server

20220105_0818_owncloud.log.0.gz
20220105_0820_owncloud.log.0.gz
20220105_0822_owncloud.log.0.gz

[gabi18]

Retested initial scenario with ownCloud-2.10.0-rc3.6417.x64.msi

• create 'Emptyfolder' in 'Test-Data' on server

• create 'Newfile.txt' and 'NewFile.txt' in 'Test-Data' on server
  -> client first shows red error message '... clash ...' then the 'i' and error in 'Not synced'

• move 'Emptyfolder' in Explorer

Result is reproducible: 'Emptyfolder' is moved to new locaction locally but not deleted on server

20220105_0938_owncloud.log.0.gz
20220105_0939_owncloud.log.0.gz
20220105_0939_owncloud.log.1.gz
20220105_0939_owncloud.log.2.gz
20220105_0941_owncloud.log.0.gz

• delete the file causing the clash on server:

Result is reproducible: the file name clash disappears from 'Not synced' tab, the client shows green checkmark, 'Emptyfolder' is deleted on server.

The icons for folder 'Test-Data' and 'Newfile.txt' remain spinning.

20220105_0945_owncloud.log.0.gz
20220105_0955_owncloud.log.0.gz

Also after a 'Force sync'

20220105_0956_owncloud.log.0.gz
20220105_1006_owncloud.log.0.gz

[gabi18]

This time a cannot do a 'Free up' space on file Newfile.txt but a 'Always keep...' and with the result that the icons are shown correctly.

[TheOneRing]

So for some unknown reason someone decided that folder deletes should not happen during the normal propagation but at the end.
Now if any error happened in any item the root job is aborted:

client/src/libsync/owncloudpropagator.cpp

Line 1156 in bfa9887

if (status != SyncFileItem::Success

and with it the remove jobs.

client/src/libsync/owncloudpropagator.cpp

Line 1131 in bfa9887

_dirDeletionJobs.abort(abortType);

So any sync error in will prevent the removal of empty folders...

[TheOneRing]

See 1e652e1

Note that this means that if there are errors in subJobs the dirDeletionJobs won't get executed.

[dragotin]

Here:
https://central.owncloud.org/t/not-syncing-folder-deletions-from-web-to-computer/39318
somebody reports something very similar: The local delete of an empty folder is not done after a detected case clash.

[amrita-shrestha]

tests in ownCloud 4.0.0.10524-daily20230329 f5ecc9

Test 1: #9311 (comment) ✔️

• deleting empty folders outside of the clash error folder tree ✔️

Test 2:
Steps

1. create subfolder testData/blueprint on the server
2. create two same case-sensitive files 'hello.txt' and 'Hello.txt' inside the folder 'blueprint' on the server
3. sync the account to the desktop client
4. one file sync Hello.txt and clash error for another file hello.txt ✔️
5. move the blueprint folder outside of the sync root
6. only the file Hello.txt was removed from the server and desktop ❓
7. another file hello.txt sync to the desktop inside the blueprint folder ❓ (file sync overlay icon remains 🔄 )

cc @TheOneRing Is step 6 and 7 expected behavior ❓

[TheOneRing]

Test scenario:

• Create Folder A, B, C
• Create a case clash or an arbitrary sync error in folder A
• Remove folder C on the server
• C is deleted locally independent of the error in A

In previous releases the C was not deleted due to the unrelated error in A

I'm pretty sure there are more combinations we should test....

[amrita-shrestha]

Test scenario:

• Create Folder A, B, C
• Create a case clash or an arbitrary sync error in folder A
• Remove folder C on the server
• C is deleted locally independent of the error in A

In previous releases the C was not deleted due to the unrelated error in A

I'm pretty sure there are more combinations we should test....

Test Result :

C is deleted locally independent of the error in A