Binwalk 3.1.0 does not detect SquashFS in firmware files on Arch Linux

[VitoAlighieri]

Title: Binwalk 3.1.0 does not detect SquashFS in firmware files on Arch Linux

Description:

I encountered an issue with Binwalk 3.1.0 on Arch Linux where it fails to detect embedded SquashFS file systems in firmware files, even though it detects and extracts XZ-compressed layers correctly. On another machine running Ubuntu with Binwalk 2.3.3, the same firmware file is processed correctly, and SquashFS is identified and extracted successfully.

Steps to Reproduce:

1. Use a firmware file that contains XZ-compressed data and an embedded SquashFS file system.

2. Run the following command on Arch Linux with Binwalk 3.1.0:

   binwalk -e firmware.bin

3. Observe the output, which only shows multiple XZ-compressed sections but does not identify SquashFS.

4. Perform the same process on an Ubuntu machine with Binwalk 2.3.3, where SquashFS is detected and extracted properly.

Expected Behavior:
Binwalk should detect the embedded SquashFS file system after decompressing the XZ layers and extract it properly.

Actual Behavior:
Binwalk only detects and extracts XZ-compressed layers but does not identify SquashFS, stopping further extraction at the decompressed XZ layers.

Environment:

• Binwalk Version: 3.1.0
• Operating System: Arch Linux (fully updated)
• Dependencies Installed:
  • squashfs-tools
  • xz-utils
  • gzip
  • tar
  • bzip2
  • cpio
  • zstd

Logs and Output:
Here is an example output from the firmware file:

987988     0xF1354       XZ compressed data, total size: 101788 bytes
1089776    0x10A0F0      XZ compressed data, total size: 58484 bytes
1148260    0x118564      XZ compressed data, total size: 23512 bytes
...

The extraction log shows success in extracting XZ layers but does not recurse further to detect SquashFS.

Troubleshooting Attempts:

1. Verified dependencies (e.g., squashfs-tools and xz-utils) are installed and functional.

2. Used the --matryoshka flag to force recursive extraction:

   binwalk -e --matryoshka firmware.bin

   Result: Same issue persists; SquashFS is not detected.

3. Extracted XZ layers manually using dd and unxz, then inspected the extracted files with Binwalk, which still fails to detect SquashFS.

4. Compared results with Binwalk 2.3.3 on Ubuntu, where SquashFS is detected successfully.

5. Reinstalled Binwalk and rebuilt it from source, but the issue remains.

Potential Cause:
This issue could be due to changes in how Binwalk 3.1.0 processes signatures or handles recursive extraction. It might also be related to Arch Linux’s handling of dependencies or configurations for Binwalk.

Request:
Please investigate the issue and provide a fix or guidance on properly configuring Binwalk 3.1.0 to detect SquashFS after decompressing XZ layers. If possible, clarify whether this is a regression or a configuration problem specific to Arch Linux.

Additional Information:
If required, I can provide the problematic firmware file or additional logs to assist with debugging. Thank you for your help!

[devttys0]

I have tested Binwalk v3 on XZ-compressed SquashFS images successfully, so it seems that there is a bug in identifying the SquashFS image in that particular firmware file. If you can provide a copy of the problematic firmware I can take a look at it, it may be an easy fix. Thanks!

[VitoAlighieri]

I have Binwalk installed via Pacman directly on Arch Linux, and I am unsure if this is the optimal installation method. The same firmware has been tested on a different machine running Ubuntu with Binwalk 2.3.3, and it successfully detects and extracts SquashFS without any manual intervention.

The output that I get is this:

The file is this:
firmware.zip

[devttys0]

Ah, an interesting case. The very first XZ compression stream at offset 0x3930 has a valid XZ header, but the XZ data is malformed (possibly a missing XZ footer?). This was causing Binwalk to think that the XZ data extended past the start of the SquashFS file system, and hence Binwalk did not "see" the SquashFS header. The 7z utility is able to at least partially decompress the malformed XZ data though.

This has been fixed in the master branch, output now looks like:

[VitoAlighieri]

Wow, thank you so much. This is an absolutly amazing tool, and you are doing an incredible job, thank you so much! When will the Pacman package get updated?