Corrupted Images When Using --cbz

[Fenikkusu]

Documenting this as I didn't see any specific details on why it was occurring.

I am running Gallery DL in a K8S CronJob. I kept getting images that were incomplete and corrupted (Identical to #29). I found that if I kept the raw files, the raw download was fine. I tried a number of different things debugging, including with and without postprocessors, along with --cbz. This may be an issue that limits itself to a K8S Empty Dir environment, but I'm not 100% sure.

I noticed two things "off" about the cbz file. The first was that the first image was missing from the archive. The second was that I could see the size of the cbz increasing as the download progressed, indicating that --cbz is adding the files to the archive immediately. I don't think using post-processors does this, but removing the --cbz and using postprocessors exclusively resolved the issue, allowing the images to not only be correct, but also contain the missing first image.

[mikf]

This may be due to the archive getting corrupted when the gallery-dl process gets killed without being able to do a proper shutdown sequence, leaving the archive file with an incomplete central directory file header - see zip.mode.

but removing the --cbz and using postprocessors exclusively resolved the issue

--cbz does the exact same as

        "postprocessors": [
            {
                "name": "zip",
                "extension": "cbz"
            }
        ]

exclusively

Maybe you were using two zip post processors at the same time, both writing to the same archive file?

[Fenikkusu]

I was using two at one point, yes. I had both --cbz and postprocessors enabled initially. While trying to identify the issue, I disabled the postprocessors section by renaming it #postprocessors, but found I was getting the corrupted archives. I didn't try with postprocessor you've listed above, though. I am currently using

          "postprocessors": [{
              "name": "zip",
              "compression": "store",
              "extension": "cbz",
              "filter": "extension not in ('zip', 'rar', 'cbz', 'sqlite')",
              "keep-files": false
            }]

It is working as expected so far.

[Fenikkusu]

It occurs to me that I need to retry. I'm currently using a docker image based on the gallery-dl image from mid-June, when I was last working on this project. I just got back to a few days ago and haven't yet rebuilt the image (My custom image installs rsync). I will get the image updated and retry to see if the issue persists on the latest version.