feat: adds js implementation of rabin chunker for windows and browser #30
Conversation
|
Also fixes a bug with the native chunker that meant it was truncating files. |
|
|
||
| const jsRabin = () => { | ||
| // see https://github.com/datproject/rabin/blob/c0378395dc0a125ab21ac176ec504f9995b34e62/src/rabin.cc | ||
| class Rabin { |
There was a problem hiding this comment.
Do you mean rabinjs? It has no repo on npm and only 5 downloads a week.
Not sure it's ready for production.
| maxChunkSize: KiB256 + (KiB256 / 2), | ||
| window: 16, | ||
| polynomial: '0x3DF305DFB2A805' | ||
| maxChunkSize: KiB256 + (KiB256 / 2) | ||
| } | ||
|
|
||
| const chunks = await all(chunker([file], opts)) |
There was a problem hiding this comment.
Should we add a test to check that the output of JS rabin impl is the same as native rabin impl?
There was a problem hiding this comment.
The rabin tests run in node and the browser, one of them tests the size of chunks emitted so I thought that was sufficient.
I was adding interop tests for more comprehensive testing but then @hugomrdias compiled the C version to WASM in rabin-wasm so may wait for that to be ready.
There was a problem hiding this comment.
Interop tests are at ipfs/interop#74 - though as ever the build is failing due to unrelated tests that depend on networking.
Ported from https://www.npmjs.com/package/rabin