What to do in case when written file has wrong CRC?

[veraciousnottaken]

Assuming:
./igir move zip test report --input ./RomsUnsorted/nes-test --output "./No-Intro-Nes-Test" --overwrite --dat "./Nintendo - Nintendo Entertainment System (Headerless) (Parent-Clone) (20230318-220804).zip" --single --prefer-language EN --prefer-region USA,EUR,WORLD,JPN --prefer-verified --prefer-good --prefer-revision-newer --dir-dat-name -vv

No-Intro-Nes-Test is empty at the begining and nes-test is

nes-test\Zombie Nation.zip 262160 bytes
nes-test\English\Zombie Nation (USA).zip 262160 bytes

Files inside are named like .zip file and both have CRC 29685F24

 ______   ______   ______  _______
|      \ /      \ |      \|       \
 \$$$$$$|  $$$$$$\ \$$$$$$| $$$$$$$\
  | $$  | $$ __\$$  | $$  | $$__| $$
  | $$  | $$|    \  | $$  | $$    $$   ROM collection manager
  | $$  | $$ \$$$$  | $$  | $$$$$$$\
 _| $$_ | $$__| $$ _| $$_ | $$  | $$   v1.4.1
|   $$ \ \$$    $$|   $$ \| $$  | $$
 \$$$$$$  \$$$$$$  \$$$$$$ \$$   \$$


INFO:  DATScanner: Scanning DAT files
DEBUG: DATScanner: Found 1 DAT file
DEBUG: DATScanner: Enumerating DAT archives
DEBUG: DATScanner: Deserializing DAT XML to objects
DEBUG: DATScanner: Parsing DAT files
INFO:  DATScanner: Done scanning DAT files
✓ Scanning for DATs ············ | 1 unique DAT found
INFO:  ROMScanner: Scanning ROM files
DEBUG: ROMScanner: Found 2 ROM files
INFO:  ROMScanner: Done scanning ROM files
✓ Scanning for ROMs ············ | 1 unique ROM found
INFO:  HeaderProcessor: Processing file headers
INFO:  HeaderProcessor: Done processing file headers
✓ Detecting ROM headers ········ | 1 ROM processed
INFO:  Processing 1 DAT
INFO:  CandidateFilter: Nintendo - Nintendo Entertainment System (Headerless): Generating candidates
DEBUG: CandidateFilter: Nintendo - Nintendo Entertainment System (Headerless): 2 unique ROMs found
DEBUG: CandidateFilter: Nintendo - Nintendo Entertainment System (Headerless): generated 256KiB of 1 candidate for 2,365 parents
INFO:  CandidateFilter: Nintendo - Nintendo Entertainment System (Headerless): Done generating candidates
INFO:  PatchCandidateGenerator: Nintendo - Nintendo Entertainment System (Headerless): Generating patched candidates
DEBUG: PatchCandidateGenerator: Nintendo - Nintendo Entertainment System (Headerless): 0 unique patches found
INFO:  PatchCandidateGenerator: Nintendo - Nintendo Entertainment System (Headerless): Done generating patched candidates
INFO:  CandidateFilter: Nintendo - Nintendo Entertainment System (Headerless): Filtering candidates
DEBUG: CandidateFilter: Nintendo - Nintendo Entertainment System (Headerless): filtered to 256KiB of 1 candidate for 2,365 parents
INFO:  CandidateFilter: Nintendo - Nintendo Entertainment System (Headerless): Done filtering candidates
INFO:  ROMWriter: Nintendo - Nintendo Entertainment System (Headerless): Writing candidates
ERROR: ROMWriter: Nintendo - Nintendo Entertainment System (Headerless): No-Intro-Nes-Test/Nintendo - Nintendo Entertainment System (Headerless)/Zombie Nation (USA).zip: written zip has the file Zombie Nation (USA).nes with the CRC 03fb57b6, expected 29685f24
INFO:  ROMWriter: Nintendo - Nintendo Entertainment System (Headerless): Done writing candidates
INFO:  StatusGenerator: Nintendo - Nintendo Entertainment System (Headerless): Generating ROM statuses
INFO:  StatusGenerator: Nintendo - Nintendo Entertainment System (Headerless): Done generating ROM statuses
✓ Nintendo - Nintendo Entertainm | 0/5 BIOSes, 1/2,919 retail releases written
INFO:  Done processing 1 DAT
INFO:  ReportGenerator: Generating report
DEBUG: ReportGenerator: No-Intro-Nes-Test/igir_2023-03-26T13;19;34+00;00.csv: wrote 1 status
INFO:  ReportGenerator: Done generating report
✓ Generating report ············ | No-Intro-Nes-Test/igir_2023-03-26T13;19;34+00;00.csv

Output directory contains: Nintendo - Nintendo Entertainment System (Headerless)\Zombie Nation (USA).zip with CRC 03fb57b6 and size 262144 bytes.

My question is how to interpret this error? I'm assuming it's something with headers

Content of headerless .dat for this rom is:

<game name="Zombie Nation (USA)">
	<description>Zombie Nation (USA)</description>
	<release name="Zombie Nation (USA)" region="USA"/>
	<rom name="Zombie Nation (USA).nes" size="262144" crc="03fb57b6" md5="e83eefc4ae17b50c3d6910922eb3e1ed" sha1="c26eeeefb90f857df8134fd267106dee2f5a1caa"/>
</game>

and headered is:

<game name="Zombie Nation (USA)">
	<description>Zombie Nation (USA)</description>
	<release name="Zombie Nation (USA)" region="USA"/>
	<rom name="Zombie Nation (USA).nes" size="262160" crc="b842b58c" md5="86b97f846b1eed035aeef669d00b8728" sha1="ea63840b39029b57e249be2a83624d0b9dc94ea7"/>
</game>

[emmercm]

Is the Zombie Nation (USA).zip zip file's CRC 03fb57b6, or is the Zombie Nation (USA).nes entry within the zip that CRC? Some kind of GUI such as 7zip or WinRAR should be able to display it pretty quickly.

I don't think it would be header-related actually, though that's a good thought.

Invalid written files should be unexpected, and it likely indicates a file I/O or filesystem issue. I wouldn't be surprised if this is related to #341 which I'm still working on.

[veraciousnottaken]

Testing zip itself file indicates no errors.

[emmercm]

I'm hoping this is just another bug around continuing processing before the zip has been fully written.

[emmercm]

Per #341, this is likely not a failure to flush the file to disk issue.

[emmercm]

Reading this again @veraciousnottaken, the log message indicated that Zombie Nation (USA).nes was indeed the headerless 03fb57b6 (good), but it was expecting 29685f24 for some reason (bad).

Searching 29685f24 on the internet, I definitely see it referenced as Zombie Nation (U) [!].nes in a few places.

Does your DAT contain 29685f24 anywhere? I can't find it in any of the No-Intro DATs.

[veraciousnottaken]

I'm using No-Intro Love Pack (PC XML) (2023-03-19).zip and there's no 29685f24 string inside any dat file.

[emmercm]

Never mind, I'm able to reproduce it on my end. Looking into this.

[emmercm]

It appears that 29685f24 is a valid headered NES ROM, just not the header that No-Intro is expecting.

The issue here is the auto-removal of the header, since you're supplying a headerless DAT. igir is expecting the output to match the input, but that isn't right, it should be expecting the headerless CRC of 03fb57b6 - resulting in no error message.

I converted this to the issue #368.