Update: README

README.md

@@ -1,6 +1,6 @@
 # igir
 
-`igir` (pronounced "eager") is a platform-independent ROM collection manager to help sort collections and make one game, one rom (1G1R) sets.
+`igir` (pronounced "eager") is a platform-independent ROM collection manager to help filter, sort, patch, and archive ROM collections.
 
 ![CLI:Windows,macOS,Linux](https://badgen.net/badge/icon/Windows,%20macOS,%20Linux?icon=terminal&label=CLI&color=grey)
 [![npm:igir](https://badgen.net/npm/v/igir?icon=npm&label=igir&color=red)](https://www.npmjs.com/package/igir)
@@ -17,54 +17,15 @@ A video of an example use case:
 
 [![asciicast](https://asciinema.org/a/uVZpMCas3SQIA0q6sCh5rYqdI.svg)](https://asciinema.org/a/uVZpMCas3SQIA0q6sCh5rYqdI)
 
-With a large ROM collection it can be difficult to:
+With `igir` you can manage a ROM collection of any size:
 
-- 📂 Organize ROM files by console
-- 🪄 Name ROM files consistently, including the right extension
+- 🔍 Scan DAT, ROM, and patch files & archives (see [archive docs](docs/advanced-topics.md#supported-archive-formats))
+- 📂 Organize ROM files by console (with [DATs](docs/dats.md))
+- 🪄 Name ROM files consistently, including the right extension (with [DATs](docs/dats.md))
+- ✂️ Filter out duplicate ROMs, or ROMs in languages you don't understand (see [filtering docs](docs/rom-filtering.md))
 - 🗜️ Archive ROMs individually in mass
-- ✂️ Filter out duplicate ROMs, or ROMs in languages you don't understand
-- 🩹 Patch ROMs automatically in mass
-- 🔍 Know what ROMs are missing for each console
-
-`igir` helps solve all of these problems!
-
-## What does `igir` need?
-
-**`igir` needs an input set of ROMs, of course!**
-
-Those ROMs can be in archives (`.001`, `.7z`, `.gz`, `.rar`, `.tar.gz`, `.z01`, `.zip`, `.zipx`, and more!) or on their own. They can also contain a header or not (see [docs](docs/rom-headers.md)).
-
-**`igir` works best with a set of DATs as well.**
-
-Though not required, DATs can provide a lot of information for ROMs such as their correct name, and which ROMs are duplicates of others. See the [docs](docs/dats.md) for more information on DATs and some "_just tell me what to do_" instructions.
-
-**`igir` then needs one or more commands:**
-
-- `copy`: copy ROMs from input directories to an output directory
-- `move`: move ROMs from input directories to an output directory
-- `zip`: create zip archives of output ROMs
-- `test`: test all written ROMs for accuracy
-- `clean`: recycle all unknown files in an output directory
-- `report`: generate a report on ROMs found and processed
-
-The `igir --help` command shown below includes examples of how to use multiple commands together.
-
-## How does `igir` work?
-
-`igir` runs these steps in the following order:
-
-1. Scans the DAT input path for every file and parses them, if provided
-2. Scans each ROM input path for every file
-   - Then detects headers in those files, if applicable (see [docs](docs/rom-headers.md))
-3. Scans each patch input path for every file (see [docs](docs/rom-patching.md))
-4. ROMs are matched to the DATs, if provided
-   - Then ROMs are matched to any applicable patches, creating multiple versions from the same ROM
-   - Then filtering and sorting options are applied (see [docs](docs/rom-filtering.md))
-   - Then ROMs are written to the output directory, if specified (`copy`, `move`)
-   - Then written ROMs are tested for accuracy, if specified (`test`)
-   - Then input ROMs are deleted, if specified (`move`)
-5. Unknown files are recycled from the output directory, if specified (`clean`)
-6. An output report is written to the output directory, if specified (`report`)
+- 🩹 Patch ROMs automatically in mass (see [patching docs](docs/rom-patching.md))
+- 🔮 Know what ROMs are missing for each console (with [DATs](docs/dats.md))
 
 ## How do I run `igir`?
 
@@ -188,11 +149,7 @@ Examples:
     igir copy --input **/*.smc --output Headerless/ --dir-mirror --remove-headers .smc
 ```
 
-## How do I obtain ROMs?
-
-Emulators are generally _legal_, as long as they don't include copyrighted software such as a console BIOS. Downloading ROM files that you do not own is piracy which is _illegal_ in many countries.
-
-See the [Dumping ROMs](docs/rom-dumping.md) page for more information.
+See the [advanced examples](docs/advanced-examples.md) page for even more examples.
 
 ## Why choose `igir`?
 
@@ -213,14 +170,14 @@ Each manager has its own pros, but many have the same drawbacks or limitations:
 - Output report formats that are difficult to parse or filter
 - Limited archive extraction support
 - Limited folder management options
-- No ROM header support
-- No ROM header removal functionality
+- No ROM header detection & removal support
+- No ROM patching functionality
 - Limited parent/clone, region, language, version, and ROM type filtering
 - No ability to prioritize parent/clones when creating a 1G1R set
 
 ## Additional documentation
 
-See the [docs](/docs) page for more in-depth information!
+See the [docs](/docs) page for in-depth information on multiple topics!
 
 ## Feature requests, bug reports, and contributing
 

docs/advanced-examples.md

@@ -0,0 +1,86 @@
+# Advanced Examples
+
+## Personal usage
+
+`igir` has many options available to fit almost any use case, but the number of options can be overwhelming. So that begs a question: _how do I, the author of `igir`, use `igir` in the real world?_
+
+I have a 4TiB external hard drive that I use as my source of truth where I store all of my DATs, ROMs, and patches. In general, I'm more interested in cartridge-based consoles. Optical-based ROMs can take up a significant amount of space.
+
+The file tree in that hard drive looks like this:
+
+```text
+/Volumes/WDPassport4
+├── FBNeo
+│   ├── Arcade
+│   └── Neogeo
+├── No-Intro
+│   ├── Nintendo - Game Boy
+│   ├── Nintendo - Game Boy Advance
+│   ├── Nintendo - Game Boy Advance (Multiboot)
+│   ├── Nintendo - Game Boy Advance (Video)
+│   ├── Nintendo - Game Boy Advance (e-Reader)
+│   ├── Nintendo - Game Boy Color
+│   └── etc...
+├── No-Intro Love Pack (PC XML) (2022-10-21).zip
+├── Patches
+│   ├── gb
+│   ├── gba
+│   └── gbc
+├── Redump (2022-10-22).zip
+├── TOSEC - DAT Pack - Complete (3530) (TOSEC-v2022-07-10).zip
+└── igir_library_sync.sh
+```
+
+The root folder has a DAT zip and sub-folder for each [DAT](dats.md) release group. This helps separate differing quality of DATs and different DAT group ROM naming schemes. I then have one sub-folder for each game console, using the `--dir-dat-name` option.
+
+The `igir_library_sync.sh` script helps me keep this collection organized and merge new ROMs into it. The complete source is:
+
+```bash
+#!/usr/bin/env bash
+# @param {...string} $@ Input directories to merge into this collection
+set -euo pipefail
+
+# Treat every CLI argument as an input directory
+INPUTS=()
+for INPUT in "$@"; do
+  INPUTS+=(--input "${INPUT}")
+done
+
+npx igir@latest move zip test clean report \
+  --dat "./No-Intro*.zip" \
+  --input "./No-Intro/" \
+  "${INPUTS[@]}" \
+  --patch "./Patches/" \
+  --output "./No-Intro/" \
+  --dir-dat-name
+```
+
+I then copy ROMs to other devices from this source of truth.
+
+For example, I have this script `igir_pocket_sync.sh` at the root of my [Analogue Pocket](https://www.analogue.co/pocket)'s SD card:
+
+```bash
+#!/usr/bin/env bash
+set -euo pipefail
+
+SOURCE=/Volumes/WDPassport4
+
+npx igir@latest copy test clean \
+  --dat "${SOURCE}/No-Intro*.zip" \
+  --input "${SOURCE}/No-Intro/" \
+  --input-exclude "${SOURCE}/No-Intro/Nintendo - Game Boy Advance (e-Reader)/" \
+  --patch "${SOURCE}/Patches/" \
+  --output "./Assets/{pocket}/common/" \
+  --dir-letter \
+  `# Leave BIOS files alone` \
+  --clean-exclude "./Assets/*/common/*.*" \
+  --no-bios \
+  --no-bad \
+  --single \
+  --prefer-language EN \
+  --prefer-region USA,EUR,JPN \
+  --prefer-revision-newer \
+  --prefer-retail
+```
+
+That lets me create an EN+USA preferred 1G1R set for my Pocket on the fly, making sure I don't delete BIOS files needed for each core. This command will cause a lot of warning spam for the `{pocket}` output token because not every ROM of mine is playable on the Pocket, but this command will make sure every playable one is copied over.

docs/advanced-topics.md

@@ -0,0 +1,48 @@
+# Advanced Topics
+
+Information about the inner workings of `igir`.
+
+## Order of operations
+
+`igir` runs these steps in the following order:
+
+1. Scans each DAT input path for every file and parses them, if provided (`--dat`)
+2. Scans each ROM input path for every file (`--input`)
+   - Then detects headers in those files, if applicable (see [header docs](docs/rom-headers.md))
+3. Scans each patch input path for every file (`--patch`) (see [patching docs](docs/rom-patching.md))
+4. ROMs are matched to the DATs, if provided
+   - Then ROMs are matched to any applicable patches, creating multiple versions from the same ROM
+   - Then filtering and sorting options are applied (see [filtering docs](docs/rom-filtering.md))
+   - Then ROMs are written to the output directory, if specified (`copy`, `move`)
+   - Then written ROMs are tested for accuracy, if specified (`test`)
+   - Then input ROMs are deleted, if specified (`move`)
+5. Unknown files are recycled from the output directory, if specified (`clean`)
+6. An output report is written to the output directory, if specified (`report`)
+
+## Supported archive formats
+
+`igir` supports most common archive formats:
+
+| Extension                | Includes file CRC32 | Can extract natively |
+|--------------------------|---------------------|----------------------|
+| `.7z`                    | ✅                   | ❌                    |
+| `.gz`, `.gzip`           | ❌ CRC16             | ❌                    |
+| `.rar`                   | ✅                   | ❌                    |
+| `.tar`                   | ❌                   | ✅                    |
+| `.tar.gz`, `.tgz`        | ❌                   | ✅                    |
+| `.z01`                   | ✅                   | ❌                    |
+| `.zip` (including zip64) | ✅                   | ✅                    |
+| `.zip.001`               | ✅                   | ❌                    |
+| `.zipx`                  | ✅                   | ❌                    |
+
+**You should prefer archive formats that have CRC32 checksum information for each file.**
+
+`igir` uses CRC32 information to match ROMs to DAT entries. If an archive already contains CRC32 information for each file, then `igir` won't need to extract each file and compute its CRC32 itself. This can save a lot of time on large files especially.
+
+This is why you should use the `igir zip` command when organizing your primary ROM collection. It is much faster to scan archives with CRC32 information, speeding up actions such as merging new ROMs into an existing collection.
+
+**You should prefer archive formats that `igir` can extract natively.**
+
+Somewhat proprietary archive formats such as `.7z` and `.rar` require `igir` to use an external tool to enumerate and extract files. This can greatly slow down processing speed.
+
+This is why `igir` uses `.zip` as its output archive of choice, `.zip` files are easy and fast to read, even if they can't offer as high of compression as other formats.

docs/dats.md

@@ -27,7 +27,7 @@ igir [commands..] --dat No-Intro*.zip --input <input>
 ## Just tell me what to do
 
 1. Go to the No-Intro DAT-o-MATIC [daily download page](https://datomatic.no-intro.org/index.php?page=download&s=64&op=daily)
-2. Select the "P/C XML" option (as opposed to "standard DAT") and download the `.zip` to wherever you store your ROMs
+2. Select the "P/C XML" dropdown option (as opposed to "standard DAT") and download the `.zip` to wherever you store your ROMs
 3. Every time you run `igir`, specify the `.zip` file you downloaded with the `--dat` option:
 
   ```shell
@@ -50,7 +50,7 @@ And some less popular release groups are:
 
 ## Parent/clone (P/C)
 
-DATs that include "parent" and "clone" information help `igir` understand what game releases are actually the same game ("clones"). Frequently a game will be released in many regions or with different revisions, usually with only language translations and minor bug fixes. For example, No-Intro has 6+ "clones" of Pokémon Blue cataloged.
+DATs that include "parent" and "clone" information help `igir` understand what game releases are actually the same game ("clones"). Frequently a game will be released in many regions or with different revisions, usually with only language translations and minor bug fixes. For example, No-Intro has 6+ "clones" of Pokemon Blue cataloged.
 
 Being able to know that many releases are actually the same game gives `igir` the ability to produce "one game, one rom" (1G1R) sets with the `--single` option. 1G1R sets include only one of these "clone" releases, usually filtered to a language and region, because many people don't care about ROMs they can't understand.
 

docs/rom-dumping.md

@@ -1,5 +1,9 @@
 # ROM Dumping
 
+⚠️ An obligatory warning about downloading ROMs:
+
+> Emulators are generally _legal_, as long as they don't include copyrighted software such as a console BIOS. Downloading ROM files that you do not own is piracy which is _illegal_ in many countries.
+
 [Dumping.Guide](https://dumping.guide/start) and [Emulation General Wiki](https://emulation.gametechwiki.com/index.php/Ripping_games) are some of the best resources for legally creating ROM files from games you own.
 
 Here is a condensed version that isn't guaranteed to be up-to-date.
@@ -27,7 +31,7 @@ Here is a condensed version that isn't guaranteed to be up-to-date.
 | Sony - PSP              | [PSP Filer](http://wiki.redump.org/index.php?title=PlayStation_Portable_Dumping_Guide)                                                                                                                                          |
 | Sony - PlayStation Vita | [psvgamesd](https://github.com/motoharu-gosuto/psvgamesd)                                                                                                                                                                       |
 
-## CD/DVD/BluRay/etc. consoles
+## Optical-based consoles
 
 | Optical-based consoles     | [Media Preservation Frontend (MPF)](https://github.com/SabreTools/MPF) (w/ PC) | Software for native hardware                                                                                  |
 |----------------------------|--------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------|

docs/rom-headers.md

@@ -52,4 +52,4 @@ igir [commands..] --dat <dats> --input <input> --remove-headers
 
 Some DAT groups such as No-Intro publish "headered" and "headerless" DATs for the same console, such as NES. `igir` will treat these DATs differently, automatically removing headers (if present) for "headerless" DATs, and leaving the header intact for "headered" DATs (regardless of CLI parameters).
 
-As explained above, you almost always want the "headered" version. It's only in very specific circumstances that you would want the "headerless" version.
+As explained above, you almost always want the "headered" version. It's only in very specific circumstances that you might need the "headerless" version.

docs/rom-patching.md

@@ -8,7 +8,7 @@ Games and their ROMs are protected under copyrights, so patches are used in orde
 
 There are many, _many_ patch types that ROM hackers use to distribute their changes on the internet ([xkcd "Standards"](https://xkcd.com/927/)). Typically, a patch will only be distributed in one format, so gamers are entirely at the mercy of the ROM hacker's choice.
 
-Not all patch types are created equal. Here are some tables of some existing formats, whether `igir` supports them, and what they support.
+Not all patch types are created equal. Here are some tables of some existing formats, whether `igir` supports them, and what the patch supports.
 
 **Common patch types:**
 
@@ -38,15 +38,15 @@ Not all patch types are created equal. Here are some tables of some existing for
 | `.rup` (NINJA 2.0)  | ❌         | ❌ uses MD5              |                                                                                                                                       |
 | `.rxl`              | ❌         | ❓                       |                                                                                                                                       |
 
-If you have a choice in patch format, choose one that does contain CRC32 checksums in the patch file contents, otherwise see any notes above.
+If you have a choice in patch format, choose one that contains CRC32 checksums in the patch file contents.
 
 ## ROM checksums
 
 `igir` needs to be able to know what source ROM each patch file applies to, and it does this using CRC32 checksums.
 
-A few patch formats include the source ROM's CRC32 checksum in the patch's file contents. This is the most accurate and therefore the best way to get this information. `.bps` is a great example of an efficient and simple patch format that includes this information.
+A few patch formats include the source ROM's CRC32 checksum in the patch's file contents. This is the most accurate and therefore the best way to get source ROM information. `.bps` is a great example of an efficient and simple patch format that includes this information.
 
-Most patch formats do not include the source ROM's CRC32 checksum. `.ips` patches are some of the most likely you will come across. For those patches, you need to put the source ROM's CRC32 checksum in the patch's filename, either at the beginning or end, like this:
+Most patch formats _do not_ include the source ROM's CRC32 checksum. `.ips` patches are some of the most likely you will come across. For those patches, you need to put the source ROM's CRC32 checksum in the patch's filename, either at the beginning or end, like this:
 
 ```text
 Source ROM filename:

src/modules/argumentsParser.ts

@@ -74,7 +74,7 @@ export default class ArgumentsParser {
       .command('clean', 'Recycle unknown files in the output directory', (yargsSubObj) => {
         addCommands(yargsSubObj);
       })
-      .command('report', 'Generate a CSV report on the known ROM files found in the input directories', (yargsSubObj) => {
+      .command('report', 'Generate a CSV report on the known ROM files found in the input directories (requires --dat)', (yargsSubObj) => {
         addCommands(yargsSubObj);
       });
 

src/modules/patchScanner.ts

@@ -28,7 +28,7 @@ export default class PatchScanner extends Scanner {
       try {
         return await PatchFactory.patchFrom(file);
       } catch (e) {
-        await this.progressBar.logError(`Failed to parse patch ${file.toString()} : ${e}`);
+        await this.progressBar.logError(`${file.toString()}: Failed to parse patch : ${e}`);
         return undefined;
       }
     }))).filter((file) => file) as Patch[];

src/modules/scanner.ts

@@ -59,11 +59,11 @@ export default abstract class Scanner extends Module {
     try {
       const files = await FileFactory.filesFrom(filePath);
       if (!files.length) {
-        await this.progressBar.logWarn(`Found no files in path: ${filePath}`);
+        await this.progressBar.logWarn(`${filePath}: Found no files in path`);
       }
       return files;
     } catch (e) {
-      await this.progressBar.logError(`Failed to parse file ${filePath} : ${e}`);
+      await this.progressBar.logError(`${filePath}: Failed to parse file : ${e}`);
       return [];
     }
   }

src/types/archives/fileFactory.ts

@@ -1,3 +1,5 @@
+import path from 'path';
+
 import File from '../files/file.js';
 import Archive from './archive.js';
 import Rar from './rar.js';
@@ -27,7 +29,7 @@ export default class FileFactory {
       return new SevenZip(filePath);
     }
 
-    throw new Error(`Unknown archive type: ${filePath}`);
+    throw new Error(`Unknown archive type: ${path.extname(filePath)}`);
   }
 
   static isArchive(filePath: string): boolean {