Add option for Zopfli iteration count

[LegionMammal978]

This PR extends the --zopfli argument with an optional iteration count. In my case, I have a bunch of very small images (a few kB or less), and I often like to use hundreds of iterations to squeeze off the last several bytes. (I know that this crate isn't intended for brute-force optimization, but I've found that some of its transformations and filter strategies can be more creative than zopflipng.) But this is also useful in the opposite direction, for allowing Zopfli compression on large images where 15 iterations would be prohibitive.

[AlexTMjugador]

Nice PR, thanks! I tested the changes and they work as described, so after the Clippy lints are addressed, I think we can move forward with merging it. The ARM CI build failures are unrelated to this PR and thus not a concern for it.

[andrews05]

Thanks for doing this @LegionMammal978, it does feel a bit overdue 😅

I always thought about adding this but never did because a) nobody had asked for it and b) I couldn't decide how best to implement it. As you mention, optional parameters are a bit weird and I'm concerned the require_equals will inevitably trip some users up. Overloading --zc may indeed be a better solution, though care would have to be taken to ensure it's capped at 12 for libdeflate only. Or of course a new parameter such as --zi. Do you have any thoughts @AlexTMjugador?

[ace-dent]

+1 for being able to control iterations. Nice :-)

Is the use of NonZeroU64 in place of NonZeroU8 necessary? Does it impact win32 compatibility?

[andrews05]

@ace-dent win32 should be fine, zopfli-rs is already using U64 so the change is just bringing oxipng in line.

That said, it is technically a breaking change. I don't think we're ready to jump to v10 at this point, @LegionMammal978 could we hold off on that part for now?

[ace-dent]

@andrews05 - thanks.
It's a shame that -i is taken, or we could have had parity with advdef.
I'm not sure why adding a value after -Z / -zopfli is a breaking change?... If we ensure the absence of the parameter keeps the historic default value, would that work?

Alternatively, I prefer --zi than overloading --zc. :-)

[andrews05]

The breaking change is the API changing from U8 to U64.

Adding a value to --zopfli isn't breaking, but the fact that the value is optional means that it can only be used in the form --zopfli=100 and not --zopfli 100, otherwise the value is ambiguous. I worry that the inconsistency with the syntax of all other arguments, coupled with the = syntax being less commonly used anyway, could cause confusion with users.

I'm also leaning towards --zi over --zc.

[LegionMammal978]

Alright, I've backed out the NonZeroU64 change and added a separate --zi argument which requires --zopfli.

[andrews05]

Looks good to me, thanks @LegionMammal978. I'll just wait to hear @AlexTMjugador's thoughts on the changes (he may also need to fix the failing workflow before we can merge).

[AlexTMjugador]

Overloading --zc may indeed be a better solution, though care would have to be taken to ensure it's capped at 12 for libdeflate only. Or of course a new parameter such as --zi. Do you have any thoughts @AlexTMjugador?

After giving this some thought, I concur it's probably best to go with --zi for specifying iterations 👍

Newer versions of Zopfli can also operate in a "do iterations until no compression gains are achieved" mode, which can be useful for people looking into brute-forcing their way to marginally better compression, and that is more straightforward to expose in OxiPNG through additional dedicated command-line switches than by overloading already existing ones. So setting this precedent here is a good idea.

Is the use of NonZeroU64 in place of NonZeroU8 necessary?

I don't think this is necessary for OxiPNG at the moment either. Upstream Zopfli began accepting such high iteration counts to support the mode outlined above (after all, there is little difference in real usage between unbounded iterations and 2⁶⁴ - 1 iterations), but I don't think there's any practical purpose for them besides that. 255 iterations is more than enough room to brute-force higher compression for PNGs.

(he may also need to fix the failing workflow before we can merge).

Yeah, I will need to get down to this sooner than later... Linux musl AArch64 nightly Rust builds have an annoying habit of breaking down due to esoteric, supposedly-fixed symbol resolution heisenbugs every some months. <opinion>I guess this is the sort of things one can expect when foundational software tools are maintained with scarce resources by skilled volunteers, while billions of dollars are poured into dubious "industry-changing" "AI-powered" projects by stakeholders who neglect to understand that "AI" also needs reliable and ongoing maintenance on toolchains and other "solved problems" to even work.</opinion>

[LegionMammal978]

Newer versions of Zopfli can also operate in a "do iterations until no compression gains are achieved" mode

Would you happen to have links to these? The main Zopfli repo hasn't really changed in functionality since 2016, and it's hard to figure out whether any reimplementations have improved on the basic algorithm.

255 iterations is more than enough room to brute-force higher compression for PNGs.

One might think so. But for the small PNGs I mentioned above, I can often recover another dozen bytes or so by bumping it up to 500 or 1000 iterations. Past that, every order of magnitude in the iteration count tends to recover 1–5 bytes. So huge iteration counts can be useful if you really want to squeeze an image.

[AlexTMjugador]

Would you happen to have links to these? The main Zopfli repo hasn't really changed in functionality since 2016, and it's hard to figure out whether any reimplementations have improved on the basic algorithm.

Sure! OxiPNG uses the most popular Rust library that reimplements Zopfli, whose docs can be read here (the specific documentation for the option that allows such behavior is here). The compression code is very similar to the original Zopfli C code, but it adds some API niceties on top of what's available in the C version, such as support for streaming compression and the aforementioned compression mode.

One might think so. But for the small PNGs I mentioned above, I can often recover another dozen bytes or so by bumping it up to 500 or 1000 iterations. Past that, every order of magnitude in the iteration count tends to recover 1–5 bytes. So huge iteration counts can be useful if you really want to squeeze an image.

That's interesting, I definitely wasn't patient enough to notice that emergent behavior at very high iteration counts. It looks like there's a reasonable use case for it then. Thanks for sharing!

[andrews05]

We can certainly make the change to U64 in v10. Along with adding support for iterations_without_improvement.