Error while scanning music (mysql issue) #217

Ark74 · 2017-06-27T00:39:42Z

Expected behavior

Complete scan
sudo -u www-data php occ audioplayer:scan --all

Under Nextcloud 12

Actual behavior

Won't complete scan.
It thows the following error:

/User/files//My song.mp3/

[Doctrine\DBAL\Exception\UniqueConstraintViolationException]                 
  An exception occurred while executing 'INSERT INTO `oc_audioplayer_tracks`   
  (`user_id`,`title`,`number`,`artist_id`,`album_id`,`length`,`file_id`,`bitr  
  ate`,`mimetype`,`genre_id`,`year`,`folder_id`,`disc`,`composer`,`subtitle`)  
   VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)' with params ["User", "My song", 
   0, 2, 2, "5:07", 1218, 128000, "audio\/mpeg", 1, 2016, 145,  1, "", ""]:                                                                  
  SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1218  
  -User' for key 'aa_tracks_file_user_id_idx'                              
                                                                               

                                                                               
  [Doctrine\DBAL\Driver\PDOException]                                          
  SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1218  
  -User' for key 'aa_tracks_file_user_id_idx'                              
                                                                               

                                                                               
  [PDOException]                                                               
  SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1218  
  -User' for key 'aa_tracks_file_user_id_idx'

Steps to reproduce the behavior

Enable app
Run scan from cli:
sudo -u www-data php occ audioplayer:scan --all
Waint until fail

Possible solution (optional)

Additional information (optional)

Tried deleting/droping all the audioplayer tables from db - won't change behavior
Debug shows that i scans until failing at certain file.

Server configuration

Operating system:
Ubuntu 16.04

Web server:
Apache2

Database:
MySQL

PHP version:
7.0.18

Nextcloud/ownCloud version: (see /status.php)
12.0.0.29

Updated from an older Nextcloud/ownCloud or fresh install:
11.0.3

Audio Player version: (see /settings/apps)
2.0.2

Where did you install Audio Player from:
Nextcloud appstore

Are you using external storage, if yes which one: local/smb/sftp/...
Yes.
Dropbox

Are you using encryption: yes/no
No

The text was updated successfully, but these errors were encountered:

Rello · 2017-06-30T19:33:21Z

Hello,
thank you for the report. we are analyzing how this can happen at all

Ark74 · 2017-06-30T19:41:59Z

Is there any software that you could recommend to check tag integrity, maybe there could be a malformed tag.
That's the only reason I could think of.

Or maybe something on the js(?) parser?

Ark74 · 2017-07-01T07:22:09Z

Could this be related with iTunNORM / iTunSMPB values?

Ark74 · 2017-07-01T16:27:57Z

I disabled the folder (.noaudio) that seemed to be the problem.
Now the scan completes, and tracks show on the player.

If i re-enable it, and rescan i find this issue on a different song (than the frst reported),


Show You Better.wav

                                                                                                                                                                                               
  [Doctrine\DBAL\Exception\DriverException]                                                                                                                                                    
  An exception occurred while executing 'INSERT INTO `oc_audioplayer_tracks` (`user_id`,`title`,`number`,`artist_id`,`album_id`,`length`,`file_id`,`bitrate`,`mimetype`,`genre_id`,`year`,`fo  
  lder_id`,`disc`,`composer`,`subtitle`) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)' with params ["Afriking", "\xc2\xb9\x00\x00\x00\xc2\xb6\x00\x27\xc3\x85\xc2\xb0\xc2\x93\x3c\xc3\xaa\xc2\x9f\x4  
  5\x38\xc3\xaa\xc2\x9f\x45\x00\x00\x00\x00\x00\x00\xc2\x90\x45\x02\x00\xc2\xb9\x45\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x4e\xc3\xbf\x1a\x00\x00\x0e\x0c\xc2\x98\x03\x00\xc2\xa2\x68\xc2\x  
  9a\x45\x00\x35\xc3\x8a\x02\x14\x00\x00\x00\x00\x00\x00\x00\x40\x20\x4b\x54\xc2\xa0\x68\xc2\x9a\x45\xc3\xbe\x01\x00\x00\x00\x00\x00\x00\xc2\x88\xc2\xbf\xc3\xbf\xc2\xbf\x65\xc3\x84\xc2\xb0\  
  xc2\x93\xc3\xbe\x01\x00\x00\x00\x00\x00\x00\x3a\xc3\x83\x0e\xc2\x98\xc3\xa7\xc3\x8f\xc3\xb2\xc2\x93\xc2\x80\x1d\x00\x03\xc2\x80\x1d\x00\x02\xc2\x94\xc3\x93\xc2\x9c\x45\x0b\xc3\x96\xc2\xb1  
  \x02\xc3\x84\xc2\xbf\xc3\xbf\xc2\xbf\xc3\x98\xc3\x80\xc3\xbf\xc2\xbf\xc2\xa8\xc2\xbf\xc3\xbf\xc2\xbf\xc2\xa0\x68\xc2\x9a\x45\xc2\x88\xc2\xa0\x53\xc2\xa0\x45\x0a\x06\xc2\x94\x00\x00\xc2\x9  
  0\x45\x00\x69\xc2\x9a\x45\x00\x00\xc3\x8a\x02\x00\x00\xc2\x90\x45\x74\x00\x00\x00\x02\xc3\x95\x0e\xc2\x98\x00\x00\x00\x00\x00\x00\xc3\x8a\x02\x08\xc3\x80\xc3\xbf\xc2\xbf\xc3\xa8\xc3\x2e\x  
  2e\x2e", 0, 1, 1, "3:56", 2570, 1411200, "audio\/wav", 1, 0, 687, 1, "", ""]:                                                                                                                
  SQLSTATE[HY000]: General error: 1366 Incorrect string value: '\xC3...' for column 'title' at row 1                                                                                           
                                                                                                                                                                                               

Exception trace:
 () at /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php:116
 Doctrine\DBAL\Driver\AbstractMySQLDriver->convertException() at /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php:128
 Doctrine\DBAL\DBALException::driverExceptionDuringQuery() at /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Statement.php:177
 Doctrine\DBAL\Statement->execute() at /var/www/nextcloud/apps/audioplayer/lib/Controller/ScannerController.php:834
 OCA\audioplayer\Controller\ScannerController->writeTrackToDB() at /var/www/nextcloud/apps/audioplayer/lib/Controller/ScannerController.php:664
 OCA\audioplayer\Controller\ScannerController->scanForAudios() at /var/www/nextcloud/apps/audioplayer/lib/Command/Scan.php:77
 OCA\audioplayer\Command\Scan->execute() at /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php:256
 Symfony\Component\Console\Command\Command->run() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:818
 Symfony\Component\Console\Application->doRunCommand() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:186
 Symfony\Component\Console\Application->doRun() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:117
 Symfony\Component\Console\Application->run() at /var/www/nextcloud/lib/private/Console/Application.php:170
 OC\Console\Application->run() at /var/www/nextcloud/console.php:92
 require_once() at /var/www/nextcloud/occ:11

                                                                                                      
  [Doctrine\DBAL\Driver\PDOException (HY000)]                                                         
  SQLSTATE[HY000]: General error: 1366 Incorrect string value: '\xC3...' for column 'title' at row 1  
                                                                                                      

Exception trace:
 () at /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php:93
 Doctrine\DBAL\Driver\PDOStatement->execute() at /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Statement.php:168
 Doctrine\DBAL\Statement->execute() at /var/www/nextcloud/apps/audioplayer/lib/Controller/ScannerController.php:834
 OCA\audioplayer\Controller\ScannerController->writeTrackToDB() at /var/www/nextcloud/apps/audioplayer/lib/Controller/ScannerController.php:664
 OCA\audioplayer\Controller\ScannerController->scanForAudios() at /var/www/nextcloud/apps/audioplayer/lib/Command/Scan.php:77
 OCA\audioplayer\Command\Scan->execute() at /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php:256
 Symfony\Component\Console\Command\Command->run() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:818
 Symfony\Component\Console\Application->doRunCommand() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:186
 Symfony\Component\Console\Application->doRun() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:117
 Symfony\Component\Console\Application->run() at /var/www/nextcloud/lib/private/Console/Application.php:170
 OC\Console\Application->run() at /var/www/nextcloud/console.php:92
 require_once() at /var/www/nextcloud/occ:11

                                                                                                      
  [PDOException (HY000)]                                                                              
  SQLSTATE[HY000]: General error: 1366 Incorrect string value: '\xC3...' for column 'title' at row 1  
                                                                                                      

Exception trace:
 () at /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php:91
 PDOStatement->execute() at /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php:91
 Doctrine\DBAL\Driver\PDOStatement->execute() at /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Statement.php:168
 Doctrine\DBAL\Statement->execute() at /var/www/nextcloud/apps/audioplayer/lib/Controller/ScannerController.php:834
 OCA\audioplayer\Controller\ScannerController->writeTrackToDB() at /var/www/nextcloud/apps/audioplayer/lib/Controller/ScannerController.php:664
 OCA\audioplayer\Controller\ScannerController->scanForAudios() at /var/www/nextcloud/apps/audioplayer/lib/Command/Scan.php:77
 OCA\audioplayer\Command\Scan->execute() at /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php:256
 Symfony\Component\Console\Command\Command->run() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:818
 Symfony\Component\Console\Application->doRunCommand() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:186
 Symfony\Component\Console\Application->doRun() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:117
 Symfony\Component\Console\Application->run() at /var/www/nextcloud/lib/private/Console/Application.php:170
 OC\Console\Application->run() at /var/www/nextcloud/console.php:92
 require_once() at /var/www/nextcloud/occ:11

There are some files with iTunNORM and iTunSMPB tags in that folder, funny thing that specific file doesn't show those tags.

Rello · 2017-07-01T22:20:59Z

Hello,
thank you for the detail. we have 2 issues here:

Duplicate entry: this relates to a file existing twice in your library but not as a double file - it´s the same file_id (from filecache table) in 2 different places. could you check if the folder you blocked with .noaudio is being used twice - e.g. shared double with that account? because it tries to add the combination of file_id+user_id twice and for some reason, your filecache_table is delivering the file twice.
nevertheless, we will try to catch this case better
Incorrect string value: I expect this to be related with Scanner truncates long multiple title properly #203 because your file is multibyte and we had an issue with an incorrect split there.
feel free to send me the file to audioplayer -at- scherello -dot- de to verify, that your case is handled with the fix

Ark74 · 2017-07-02T18:25:05Z

Duplicate entry

I have gone under the users and it doesn't seem to have shared that folder.
The dropbox mount thoug is available for all the users, nevertheless the trouble folders is a subfolder from dropbox.
Maybe the issue comes at the Dropbox tree and not the NC one, but that might take time to check.

Incorrect string value

I've send an email about it, thanks.
I did move that sole wav/music file to a new instance and it wont recognize any music file after the file scan.
So maybe the issue is the file itself.

the check for folder_id was removed because in sharing contexts, the same file could be available in different folders #217

Rello · 2017-07-02T19:08:42Z

Incorrect string: this could not be reproduced with the new version so it should be correct
duplicate file: the check was corrected to not validate against folder_id

Ark74 · 2017-07-02T19:31:56Z

Would master be a good branch to test here?

Rello · 2017-07-02T20:05:54Z

hold on. I need to fix something with the filehook first. I will let you know thank you for testing! On 02.07.2017, at 21:31, Luis Guzman <notifications@github.com> wrote: Would master be a good branch to test here? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#217 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AMw9n2nCOCsAokKPzQS-G5sGFi3wh7NKks5sJ_AtgaJpZM4OGBCe>.

Ark74 · 2017-07-02T20:07:37Z

Made a test with 2.1.0-beta.2 and there is no longer any issue when scanning.
We'll wait the oficial release to retest and close this up if you don't mind.

Cheers! 🍻

Rello added bug needs info feedback from requester required scanner labels Jul 1, 2017

Rello added this to the 2.1.0 milestone Jul 1, 2017

Rello added a commit that referenced this issue Jul 2, 2017

remove folder_id from duplicate check

a1412a0

the check for folder_id was removed because in sharing contexts, the same file could be available in different folders #217

Rello added testing and removed needs info feedback from requester required labels Jul 2, 2017

Rello added pending release and removed testing labels Jul 3, 2017

ghost pushed a commit that referenced this issue Jul 4, 2017

#217

44e6eb9

ghost removed the pending release label Aug 29, 2017

ghost closed this as completed Aug 29, 2017

This issue was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error while scanning music (mysql issue) #217

Error while scanning music (mysql issue) #217

Ark74 commented Jun 27, 2017

Rello commented Jun 30, 2017

Ark74 commented Jun 30, 2017

Ark74 commented Jul 1, 2017

Ark74 commented Jul 1, 2017

Rello commented Jul 1, 2017

Ark74 commented Jul 2, 2017

Rello commented Jul 2, 2017

Ark74 commented Jul 2, 2017

Rello commented Jul 2, 2017 via email

Ark74 commented Jul 2, 2017 •

edited

Loading

Error while scanning music (mysql issue) #217

Error while scanning music (mysql issue) #217

Comments

Ark74 commented Jun 27, 2017

Expected behavior

Actual behavior

Steps to reproduce the behavior

Possible solution (optional)

Additional information (optional)

Server configuration

Rello commented Jun 30, 2017

Ark74 commented Jun 30, 2017

Ark74 commented Jul 1, 2017

Ark74 commented Jul 1, 2017

Rello commented Jul 1, 2017

Ark74 commented Jul 2, 2017

Duplicate entry

Incorrect string value

Rello commented Jul 2, 2017

Ark74 commented Jul 2, 2017

Rello commented Jul 2, 2017 via email

Ark74 commented Jul 2, 2017 • edited Loading

Ark74 commented Jul 2, 2017 •

edited

Loading