Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mismatch compressed/decompressed NSP #101

Closed
proconsule opened this issue Sep 27, 2021 · 2 comments · Fixed by #131
Closed

Mismatch compressed/decompressed NSP #101

proconsule opened this issue Sep 27, 2021 · 2 comments · Fixed by #131
Assignees
Labels
enhancement New feature or request file format Issues with the NSZ file format
Milestone

Comments

@proconsule
Copy link

proconsule commented Sep 27, 2021

Hi nicoboss, i am working to fully support nsz in NSP Indexer.
I have right now a full decompression routine that devliver the same result as nsz command line.
But i noted that the original NSP and the decompressed NSP have a diff.
both works but i think there is a mismatch in offsets into the decompressed one

Original NSP
pfs0:/0100ea80032ea0000000000000000004.cert 000000000000-000000000700

Compressed/Decompressed NSP
pfs0:/0100ea80032ea0000000000000000004.cert 000000007df0-0000000084f0

Edit:

I modified the way i recreate the main PFS0 and now i have perfect match from Original to Compressed/Decompressed
same sha256sum!
i just started to put files from fileBodyOffset instead of starting from fileBodyOffset+ dataoffset (that in nsz is shifted)

@LittleWhite-tb
Copy link

LittleWhite-tb commented Nov 1, 2021

I was curious. It seems that the problem is only when decompressing the NSZ with nsz. When adding a file to the Pfs0Stream, it will use self.f.tell() as the offset. But the PFS0 file is already at a certain position (0x800), because the constructor is doing self.f.seek(self.headerSize).
It can be naively fixed in getHeader() by using self.headerSize as headerSize, instead of the 0x10 + len(self.files) * 0x18 + len(stringTable)

The good new is that the NSZ is correct, just the recreation of the NSP from NSZ is a bit wrong.

@nicoboss nicoboss added this to the v4.3 milestone Apr 11, 2023
@nicoboss nicoboss self-assigned this Apr 11, 2023
@nicoboss nicoboss added enhancement New feature or request file format Issues with the NSZ file format labels Apr 11, 2023
@nicoboss
Copy link
Owner

nicoboss commented Jul 7, 2023

This is fixed in the latest NSZ v4.3.0 release. The padding of the source NSP is now kept during compression/decompression making them always bit-identical (especially if --keep-delta is specified). The option --remove-padding was added to remove the padding of existing NSP/NSZ files to make them nxdumptool/no-intro compliant. This was done by applying the exact fix @LittleWhite-tb described.

container.getHeaderSize() if removePadding else container.getFirstFileOffset()

@proconsule and @LittleWhite-tb Thanks a lot for all your investigations regarding this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request file format Issues with the NSZ file format
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants