7z can’t decompress archives created with lbzip2 with -9 settings

[MaxPower85]

7z can’t decompress archives created with lbzip2 with -9 settings (900k block size), even though they open just fine with the native Archive Utility on macOS or The Unarchiver or unar CLI utility… and after checking the checksum of the decompressed file, md5 checks out… so it doesn’t seem that the archive was damaged during compression, but for some reason 7z rejects to decompress it.

Keka also shows an error, since it uses 7z to decompress archives.

Large bzip2 archives created with pbzip2 with -9 settings (it also means 900k compression block size in pbzip2) don’t seem to be affected… and it also doesn’t seem to be affecting archives created with lbzip2 if -8 or lower is used.

So, I’m not sure what’s so different about lbzip2 archives compressed with that -9 option, that it seems to be confusing 7z.

I tested it with several large files of different types… I first noticed it when compressing an ISO with some data files that was about 730 MB large… I then tried some larger file and it happened again when I tried to see can it be uncompressed with 7z… but it didn’t happen with small files that were only a few MB big when they were compressed with lbzip2 when -9 option was used… I then tried it with some large HEVC video that was about 770 MB (it was just an experiment… I’m aware what incompressible data is… I just tried it with a video because it was a large file, to test it) and it happened again.

So… just get various large files and test it.

Let the developers of 7zip know, since lbzip2 seems to be kind of popular among Linux users.

Since both lbzip2 and 7z are open-source, maybe their developers could work together to resolve it, so lbzip2 wouldn’t create files that confuse 7z (including older versions of it) and so 7z could open old files from lbzip2.

[aonez]

Thanks for the report. Maybe Igor can fix this one: https://sourceforge.net/p/sevenzip/discussion/45797/thread/5ec87e8a

I'm closing this ticket in favor of the 7-zip one.

[aonez]

I'll indeed leave it open. I can use lbzip2 for the extraction, if this can't be fixed.

[MaxPower85]

You can use pbzip2... when I was comparing pbzip2 and lbzip2, they were both super fast even at their best compression levels and very optimized for multithreading, but pbzip2 seemed to have a slightly better compression on it's highest settings when I tested it... and it didn't have that bug.

I wrote in more detail about it while comparing them to 7z... I'll post it separately now, since it focuses on what would offer the best performance for compressing/uncompressing bzip2.

[aonez]

Right now I don't have in mind why I ultimately used lbzip2 as default instead of pbzip2. I'll need to check that again.

[aonez]

I'm going to set pbzip2 by default and lbzip2 as fallback.

[MaxPower85]

Why would a fallback from pbzip2 to lbzip2 be needed?

Both pbzip2 and lbzip2 claim full compatibility with bzip2 format, so pbzip2 should work well both for compressing to and uncompressing from bzip2 format.

If pbzip2 can't read some unusual bzip2 archive for whatever reason (if some archive is maybe not fully compatible with the bzip2 format), I'm not sure would lbzip2 have better chances of uncompressing it.

In case that some unusual archive has issues, then just use unar or 7z as fallback.

[aonez]

Yeah, fallback was not the correct word. Alternative 👍🏼 I’m not using any fallback.

[aonez]

I'll leave this ticket until lbzip2 or p7zip fix this issue, but going on with pbzip2 on version 1.1.0.