Catch and ignore UniqueConstraintViolationException in file cache insert

[MorrisJobke]

• caused by a concurrect insert that happens between the INSERT and it's sub-SELECT which was there to actually avoid it within insertIfNotExists - sub selects are not atomic and allow this
• see also Bug 6160: Deadlock removed #12315
• fixes Doctrine\DBAL\Exception\UniqueConstraintViolationException: An exception occurred while executing 'INSERT INTO "oc_filecache"  #6160
• improved version of Bug 6160: Deadlock removed #12315
• part of UniqueConstraintViolationException overview ticket #12369

Avoids an error that has this stack trace:

Doctrine\DBAL\Exception\UniqueConstraintViolationException: An exception occurred while executing 'INSERT INTO "oc_filecache" ("mimepart","mimetype","mtime","size","etag","storage_mtime","permissions","checksum","path_hash","path","parent","name","storage") SELECT ?,?,?,?,?,?,?,?,?,?,?,?,? FROM "oc_filecache" WHERE "storage" = ? AND "path_hash" = ? HAVING COUNT(*) = 0' with params [1, 2, 1502911047, -1, "59949a47cabae", 1502911047, 31, "", "fb66dca5f27af6f15c1d1d81e6f8d28b", "files_trashbin", 1742774, "files_trashbin", 136, 136, "fb66dca5f27af6f15c1d1d81e6f8d28b"]:

SQLSTATE[23505]: Unique violation: 7 ERROR: duplicate key value violates unique constraint "fs_storage_path_hash"
DETAIL: Key (storage, path_hash)=(136, fb66dca5f27af6f15c1d1d81e6f8d28b) already exists.

3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php - line 128: Doctrine\DBAL\Driver\AbstractPostgreSQLDriver->convertException('An exception oc...', Object(Doctrine\DBAL\Driver\PDOException))
3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php - line 1015: Doctrine\DBAL\DBALException driverExceptionDuringQuery(Object(Doctrine\DBAL\Driver\PDOPgSql\Driver), Object(Doctrine\DBAL\Driver\PDOException), 'INSERT INTO "oc...', Array)
lib/private/DB/Connection.php - line 213: Doctrine\DBAL\Connection->executeUpdate('INSERT INTO "oc...', Array, Array)
lib/private/DB/Adapter.php - line 114: OC\DB\Connection->executeUpdate('INSERT INTO "oc...', Array)
lib/private/DB/Connection.php - line 251: OC\DB\Adapter->insertIfNotExist('*PREFIX*filecac...', Array, Array)
lib/private/Files/Cache/Cache.php - line 273: OC\DB\Connection->insertIfNotExist('*PREFIX*filecac...', Array, Array)
lib/private/Files/Cache/Cache.php - line 230: OC\Files\Cache\Cache->insert('files_trashbin', Array)
lib/private/Files/Cache/Scanner.php - line 279: OC\Files\Cache\Cache->put('files_trashbin', Array)
lib/private/Files/Cache/Scanner.php - line 216: OC\Files\Cache\Scanner->addToCache('files_trashbin', Array, -1)
lib/private/Files/Cache/Scanner.php - line 175: OC\Files\Cache\Scanner->scanFile('files_trashbin')
lib/private/Files/Cache/Scanner.php - line 322: OC\Files\Cache\Scanner->scanFile('files_trashbin/...', 3, -1, NULL, false)
lib/private/Files/Cache/Updater.php - line 124: OC\Files\Cache\Scanner->scan('files_trashbin/...', false, 3, false)
lib/private/Files/View.php - line 321: OC\Files\Cache\Updater->update('files_trashbin/...', 1502911047)
lib/private/Files/View.php - line 1151: OC\Files\View->writeUpdate(Object(OCA\Files_Trashbin\Storage), 'files_trashbin/...')
lib/private/Files/View.php - line 269: OC\Files\View->basicOperation('mkdir', '/files_trashbin...', Array)
apps/files_trashbin/lib/Trashbin.php - line 154: OC\Files\View->mkdir('files_trashbin/...')
apps/files_trashbin/lib/Trashbin.php - line 225: OCA\Files_Trashbin\Trashbin setUpTrash('klas')
apps/files_trashbin/lib/Storage.php - line 247: OCA\Files_Trashbin\Trashbin move2trash('Photos', false)
apps/files_trashbin/lib/Storage.php - line 189: OCA\Files_Trashbin\Storage->doDelete('files/Photos', 'rmdir')
lib/private/Files/View.php - line 1136: OCA\Files_Trashbin\Storage->rmdir('files/Photos')
lib/private/Files/View.php - line 348: OC\Files\View->basicOperation('rmdir', '/Photos', Array)
apps/dav/lib/Connector/Sabre/Directory.php - line 303: OC\Files\View->rmdir('/Photos')
3rdparty/sabre/dav/lib/DAV/Tree.php - line 179: OCA\DAV\Connector\Sabre\Directory->delete()
3rdparty/sabre/dav/lib/DAV/CorePlugin.php - line 287: Sabre\DAV\Tree->delete('Photos')
[internal function] Sabre\DAV\CorePlugin->httpDelete(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))
3rdparty/sabre/event/lib/EventEmitterTrait.php - line 105: call_user_func_array(Array, Array)
3rdparty/sabre/dav/lib/DAV/Server.php - line 479: Sabre\Event\EventEmitter->emit('method DELETE', Array)
3rdparty/sabre/dav/lib/DAV/Server.php - line 254: Sabre\DAV\Server->invokeMethod(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))
apps/dav/appinfo/v1/webdav.php - line 71: Sabre\DAV\Server->exec()
remote.php - line 162: require_once('/var/www/nextcl...')
{main}

@obel1x Have a look. It's similar to #12366

[MorrisJobke]

@enoch85 @mxschmitt @tx7 @j-ed @tsueri @protree @MeiRos @SchoolGuy @urbenlegend @danut2006 @quicktrick @fizzgig314 @karottenbaum @ak1n @jcklpe Could you have a look at this one? This should solve the issues you experienced in #6160 ;)

Thanks

[MorrisJobke]

@fractos You ran into this as well in #6899 (comment) - please have a look at this.

[MorrisJobke]

@mduller You ran into this as well in #6899 (comment)

@Redsandro You mentioned this one in #6899 (comment)

[protree]

Thanks! Did you do some basic testing so you don‘t expect major things to break?

[blizzz]

I prefer #12371 (less code duplication, cannot be forgotten on new/changed code or db structure)

[SchoolGuy]

@MorrisJobke I have currently not the setup to test it, but when my weekend plans do not change I will try to test it then.

[kesselb]

I would prefer insert instead of insertIfNotExists here.

1. There should be an unique index

   server/core/Migrations/Version13000Date20170718121200.php

   Line 213 in b82cbe0

   $table->addUniqueIndex(['storage', 'path_hash'], 'fs_storage_path_hash');

2. https://dev.mysql.com/doc/refman/8.0/en/insert-select.html

The target table of the INSERT statement may appear in the FROM clause of the SELECT part of the query. However, you cannot insert into a table and select from the same table in a subquery. When selecting from and inserting into the same table, MySQL creates an internal temporary table to hold the rows from the SELECT and then inserts those rows into the target table. However, you cannot use INSERT INTO t ... SELECT ... FROM t when t is a TEMPORARY table, because TEMPORARY tables cannot be referred to twice in the same statement.

I guess a temporary table with a few records is not that bad. But for the use case (as far as i understand it) you dont need it?

Maybe I'm missing something but when you catch the UniqueConstraintViolationException anyway this "fake" atomic insert is not required anymore?

[MorrisJobke]

Let's go for #12371, which addresses the first comment of @danielkesselberg and was also the feedback from other developers on IRC.