Scanner duplicate MAME entries bug.

[i30817]

First and foremost consider this:

• Only RetroArch bugs should be filed here. Not core bugs or game bugs
• This is not a forum or a help section, this is strictly developer oriented

Description

This happens on a recent mame split set, but it doesn't seem to be caused by the set being split.
What happened is that after scanning several bogus entries with 'interesting' playlist entries appear:

/media/i30817/backup/Documents/Games/MAME/cyberlip.zip
Cyber-Lip (NGM-010)
DETECT
DETECT
E51609CA|crc
MAME.lpl
/media/i30817/backup/Documents/Games/MAME/punisher.zip
D. D. Crew (World, 3 Player, FD1094 317-0190)
DETECT
DETECT
3C566638|crc
MAME.lpl
/media/i30817/backup/Documents/Games/MAME/mslug2t.zip
D. D. Crew (World, 3 Player, FD1094 317-0190)
DETECT
DETECT
3C566638|crc
MAME.lpl

etc.This repeats for quite a while but it does eventually stop (with the actual right game, isn't that interesting?)

/media/i30817/backup/Documents/Games/MAME/ddcrew.zip
D. D. Crew (World, 3 Player, FD1094 317-0190)
DETECT
DETECT
3C566638|crc
MAME.lpl
/media/i30817/backup/Documents/Games/MAME/dariusg.zip
Darius Gaiden - Silver Hawk (Ver 2.5O 1994/09/19)
DETECT
DETECT
F577041D|crc
MAME.lpl

Different files being attributed the same, wrong CRC.

Since i have a copy of the libretro-database repository for my hack commits i did a grep of it using grep -Ric "3C566638" . | grep -v ":0" . This is 'from a recursive, case insensitive search of crc that shows the number of matches, only show the lines that don't have zero matches'.

It had this result:

grep -Ric "3C566638" . | grep -v ":0" 
./metadat/mame/MAME.dat:1
./metadat/mame-nonmerged/MAME 2014.dat:1
./metadat/mame-nonmerged/MAME 2010.dat:1
./metadat/mame-split/MAME 2014.dat:1
./metadat/mame-split/MAME 2010.dat:1
./metadat/fba-split/FB Alpha - Arcade Games.dat:1

My rdb runtime database also says there is only one entry with the crc:

/media/i30817/Huggin/Documents/retroarch/database/rdb$ ./libretrodb_tool MAME.rdb list | grep -i "3C566638"
{"sha1": "0C808E07B5C8AF067F3DC6DA3A7F34E18EE2D732", "md5": "5C770AD3DD80DAF84D92627DEEC3C43B", "crc": "3C566638", "developer": "Sega", "size": 3201535, "rom_name": "ddcrew.zip", "name": "D. D. Crew (World, 3 Player, FD1094 317-0190)"}

Obvious i know, but i wanted to show that it isn't a libretro-database bug.

There seems to be yet another bug on the scanner where it keeps a stale match 'from the future' somehow and keeps giving it to files which it shouldn't.

Expected behavior

Whatever freaky algorithm bug this is wouldn't happen.

Actual behavior

Weirdness already shown.

Steps to reproduce the bug

1. Scan a mame dir

Bisect Results

[Try to bisect and tell us when this started happening]

Version/Commit

You can find this information under Information/System Information

• RetroArch: [version/commit]

Environment information

• OS: [The operating system you're running]
• Compiler: [In case you are running local builds]

[i30817]

Funny enough this can be reproduced with just two roms on a test directory:

crc32 *
3c566638	ddcrew.zip
1ec4856d	punisher.zip

Delete the mame playlist or even the whole system directory, update the databases from RA and try out directory scan.

BTW: ./libretrodb_tool MAME.rdb list | grep -i "1ec4856d"
shows nothing, so this is one of the roms that updated meanwhile that RA can't find, which shows it's the 'fail' case which is keeping outdated info around in a loop. Why the real 'ddcrew' entry was the last though, i have no clue since it logically should have been found and set first. Maybe it's a stack when it writes? Makes sense.

This also makes me think that any file could have been classified as 'ddcrew' even if it was not a game (or empty. In fact lets test that).
edit: surprisingly no. Must be filtering by name first?

Lets test that.
edit2: weirdness. A 'punisher.zip' with a empty 0 length file 'a' causes both ddcrew.zip and punisher.zip not to appear on the playlist.
edit3: actually it writes the 'ddcrew.zip' to file but the search for the non-existent 'punisher.zip' never writes out the second, and the GUI never updates, so i can't tell if the thread is hanging or it finished but never sent the notification 'everything is finished, display the new playlist'.

The scanner really needs unit tests.

[i30817]

fixed by #8333