Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unzipping very slow for 285 kb file #38

Closed
cameck opened this issue May 2, 2017 · 13 comments · Fixed by #316
Closed

Unzipping very slow for 285 kb file #38

cameck opened this issue May 2, 2017 · 13 comments · Fixed by #316

Comments

@cameck
Copy link

cameck commented May 2, 2017

First of all thanks for adding the feature of password protected zips! It works great except it is really slow.
I know there's something going on behind the scenes as this should not take this long, 35 seconds!
I can't figure out at the present moment why this is taking so long, but maybe there's something I'm doing that is wrong?

Here is my code:

const unzip = (filename, password, sha256) => {
  const start = new Date();
  console.log(`key ${filename}`);
  unzipper.Open.file(filename)
    .then((data) => {
      console.log('got data');
      return new Promise((resolve, reject) => {
        data.files[0].stream(password)
          .pipe(fs.createWriteStream(data.files[0].path))
          .on('error', reject)
          .on('finish', () => {
            const end = new Date() - start;
            console.info('Execution time: %dms', end);
            resolve(verifyChecksum(data.files[0].path, sha256));
          });
      });
    });
};

And Standard Out:

(node:12627) DeprecationWarning: Using Buffer without `new` will soon stop working. Use `new Buffer()`, or preferably `Buffer.from()`, `Buffer.allocUnsafe()` or `Buffer.alloc()` instead.
key firefox.zip
got data
Execution time: 35347ms
verifyChecksum
84597a81c67a20d342d57d979b4a7887328f505f268acb9c3ef38224116fa283
84597a81c67a20d342d57d979b4a7887328f505f268acb9c3ef38224116fa283
Checksum for firefox.exe passed
@ZJONSSON
Copy link
Owner

ZJONSSON commented Jul 4, 2018

@cameck I don't think you are doing anything wrong here. The decrypt through javascript is inherently slow and there is little focus to make improvements. Feel free to investigate on your own and make a PR if you find any optimizations.

@ZJONSSON ZJONSSON closed this as completed Jul 4, 2018
@ZJONSSON
Copy link
Owner

ZJONSSON commented Jul 4, 2018

If you can share a slow zip-file please reopen with link and I can take a look.

@cameck
Copy link
Author

cameck commented Jul 4, 2018

@ZJONSSON Got it, no worries. Thanks for getting back to me.
Unfortunately I don't have any of the previous example files anymore.
I'll see if I can take a look and reproduce.

@dkostenevich
Copy link

dkostenevich commented Sep 26, 2020

Hi @ZJONSSON. Same issue for me, large files decryption very slow. (11mb - ~3min).
After some investigation I found that it depends to big-integer.

I found some solution for encryption (https://github.com/ksoichiro/node-archiver-zip-encryptable/blob/master/lib/zip-crypto.js). They use node-int64 instead of big-integer. I forked your repo and updated Decrypt.js.
After that performance was increased x20 for large files. (11mb - ~9sec).
I'm not sure that all stable, but for my case all works well. Your tests passed too.

I think you can play with it ;)
https://github.com/dkostenevich/node-unzipper/blob/master/lib/Decrypt.js

@molzahn-sio
Copy link

Hello, I'm having the same issue here. It's even worse as I need to extract on a slowish embedded device.

The change in #38 (comment) seems promising but I would really prefer to stay on the "official" repository. Any chance of this getting included here?

@vfa-trungnm
Copy link

@ZJONSSON i have same problem

@contactsimonwilson
Copy link

I also have the same problem. @dkostenevich's Decyrpt.js change decreased my extraction time from 180 seconds down to around 8 inside a GCP serverless function.

@MarcoSchmidt97
Copy link

I do also have the same problem.
@ZJONSSON Is there any chance that @dkostenevich's change in the Decyrpt.js file will get included?

@syedhannan
Copy link

Facing similar issues here for a 27mb file its taking around 30 minutes

@lechuhuuha
Copy link

@ZJONSSON should be a big boost if using decrypt from @dkostenevich

@ZJONSSON
Copy link
Owner

Agreed, can someone submit a PR?

@lechuhuuha
Copy link

lechuhuuha commented May 24, 2024

Agreed, can someone submit a PR?

I had submit a PR.

@ZJONSSON You could check its out.

Thanks

@ZJONSSON
Copy link
Owner

ZJONSSON commented Jun 8, 2024

closed with #316 (Thanks @lechuhuuha )
published as unzipper@0.12.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants