Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zlib: add ArrayBuffer support #16042

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 56 additions & 12 deletions doc/api/zlib.md
Original file line number Diff line number Diff line change
Expand Up @@ -286,6 +286,9 @@ Compression strategy.
<!-- YAML
added: v0.11.1
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/16042
description: The `dictionary` option can be an ArrayBuffer.
- version: v8.0.0
pr-url: https://github.com/nodejs/node/pull/12001
description: The `dictionary` option can be an Uint8Array now.
Expand All @@ -308,8 +311,8 @@ ignored by the decompression classes.
* `level` {integer} (compression only)
* `memLevel` {integer} (compression only)
* `strategy` {integer} (compression only)
* `dictionary` {Buffer|TypedArray|DataView} (deflate/inflate only, empty dictionary by
default)
* `dictionary` {Buffer|TypedArray|DataView|ArrayBuffer} (deflate/inflate only,
empty dictionary by default)
* `info` {boolean} (If `true`, returns an object with `buffer` and `engine`)

See the description of `deflateInit2` and `inflateInit2` at
Expand Down Expand Up @@ -511,9 +514,10 @@ Creates and returns a new [Unzip][] object with the given [options][].

<!--type=misc-->

All of these take a [`Buffer`][], [`TypedArray`][], [`DataView`][], or string as
the first argument, an optional second argument to supply options to the `zlib`
classes and will call the supplied callback with `callback(error, result)`.
All of these take a [`Buffer`][], [`TypedArray`][], [`DataView`][],
[`ArrayBuffer`][] or string as the first argument, an optional second argument
to supply options to the `zlib` classes and will call the supplied callback
with `callback(error, result)`.

Every method has a `*Sync` counterpart, which accept the same arguments, but
without a callback.
Expand All @@ -522,6 +526,9 @@ without a callback.
<!-- YAML
added: v0.6.0
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/16042
description: The `buffer` parameter can be an ArrayBuffer.
- version: v8.0.0
pr-url: https://github.com/nodejs/node/pull/12223
description: The `buffer` parameter can be any TypedArray or DataView now.
Expand All @@ -533,6 +540,9 @@ changes:
<!-- YAML
added: v0.11.12
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/16042
description: The `buffer` parameter can be an ArrayBuffer.
- version: v8.0.0
pr-url: https://github.com/nodejs/node/pull/12223
description: The `buffer` parameter can be any TypedArray or DataView now.
Expand All @@ -541,7 +551,7 @@ changes:
description: The `buffer` parameter can be an Uint8Array now.
-->

- `buffer` {Buffer|TypedArray|DataView|string}
- `buffer` {Buffer|TypedArray|DataView|ArrayBuffer|string}

Compress a chunk of data with [Deflate][].

Expand All @@ -560,6 +570,9 @@ changes:
<!-- YAML
added: v0.11.12
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/16042
description: The `buffer` parameter can be an ArrayBuffer.
- version: v8.0.0
pr-url: https://github.com/nodejs/node/pull/12223
description: The `buffer` parameter can be any TypedArray or DataView now.
Expand All @@ -568,14 +581,17 @@ changes:
description: The `buffer` parameter can be an Uint8Array now.
-->

- `buffer` {Buffer|TypedArray|DataView|string}
- `buffer` {Buffer|TypedArray|DataView|ArrayBuffer|string}

Compress a chunk of data with [DeflateRaw][].

### zlib.gunzip(buffer[, options], callback)
<!-- YAML
added: v0.6.0
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/16042
description: The `buffer` parameter can be an ArrayBuffer.
- version: v8.0.0
pr-url: https://github.com/nodejs/node/pull/12223
description: The `buffer` parameter can be any TypedArray or DataView now.
Expand All @@ -587,6 +603,9 @@ changes:
<!-- YAML
added: v0.11.12
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/16042
description: The `buffer` parameter can be an ArrayBuffer.
- version: v8.0.0
pr-url: https://github.com/nodejs/node/pull/12223
description: The `buffer` parameter can be any TypedArray or DataView now.
Expand All @@ -595,14 +614,17 @@ changes:
description: The `buffer` parameter can be an Uint8Array now.
-->

- `buffer` {Buffer|TypedArray|DataView|string}
- `buffer` {Buffer|TypedArray|DataView|ArrayBuffer|string}

Decompress a chunk of data with [Gunzip][].

### zlib.gzip(buffer[, options], callback)
<!-- YAML
added: v0.6.0
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/16042
description: The `buffer` parameter can be an ArrayBuffer.
- version: v8.0.0
pr-url: https://github.com/nodejs/node/pull/12223
description: The `buffer` parameter can be any TypedArray or DataView now.
Expand All @@ -614,6 +636,9 @@ changes:
<!-- YAML
added: v0.11.12
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/16042
description: The `buffer` parameter can be an ArrayBuffer.
- version: v8.0.0
pr-url: https://github.com/nodejs/node/pull/12223
description: The `buffer` parameter can be any TypedArray or DataView now.
Expand All @@ -622,14 +647,17 @@ changes:
description: The `buffer` parameter can be an Uint8Array now.
-->

- `buffer` {Buffer|TypedArray|DataView|string}
- `buffer` {Buffer|TypedArray|DataView|ArrayBuffer|string}

Compress a chunk of data with [Gzip][].

### zlib.inflate(buffer[, options], callback)
<!-- YAML
added: v0.6.0
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/16042
description: The `buffer` parameter can be an ArrayBuffer.
- version: v8.0.0
pr-url: https://github.com/nodejs/node/pull/12223
description: The `buffer` parameter can be any TypedArray or DataView now.
Expand All @@ -641,6 +669,9 @@ changes:
<!-- YAML
added: v0.11.12
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/16042
description: The `buffer` parameter can be an ArrayBuffer.
- version: v8.0.0
pr-url: https://github.com/nodejs/node/pull/12223
description: The `buffer` parameter can be any TypedArray or DataView now.
Expand All @@ -649,14 +680,17 @@ changes:
description: The `buffer` parameter can be an Uint8Array now.
-->

- `buffer` {Buffer|TypedArray|DataView|string}
- `buffer` {Buffer|TypedArray|DataView|ArrayBuffer|string}

Decompress a chunk of data with [Inflate][].

### zlib.inflateRaw(buffer[, options], callback)
<!-- YAML
added: v0.6.0
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/16042
description: The `buffer` parameter can be an ArrayBuffer.
- version: v8.0.0
pr-url: https://github.com/nodejs/node/pull/12223
description: The `buffer` parameter can be any TypedArray or DataView now.
Expand All @@ -668,6 +702,9 @@ changes:
<!-- YAML
added: v0.11.12
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/16042
description: The `buffer` parameter can be an ArrayBuffer.
- version: v8.0.0
pr-url: https://github.com/nodejs/node/pull/12223
description: The `buffer` parameter can be any TypedArray or DataView now.
Expand All @@ -676,14 +713,17 @@ changes:
description: The `buffer` parameter can be an Uint8Array now.
-->

- `buffer` {Buffer|TypedArray|DataView|string}
- `buffer` {Buffer|TypedArray|DataView|ArrayBuffer|string}

Decompress a chunk of data with [InflateRaw][].

### zlib.unzip(buffer[, options], callback)
<!-- YAML
added: v0.6.0
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/16042
description: The `buffer` parameter can be an ArrayBuffer.
- version: v8.0.0
pr-url: https://github.com/nodejs/node/pull/12223
description: The `buffer` parameter can be any TypedArray or DataView now.
Expand All @@ -695,6 +735,9 @@ changes:
<!-- YAML
added: v0.11.12
changes:
- version: REPLACEME
pr-url: https://github.com/nodejs/node/pull/16042
description: The `buffer` parameter can be an ArrayBuffer.
- version: v8.0.0
pr-url: https://github.com/nodejs/node/pull/12223
description: The `buffer` parameter can be any TypedArray or DataView now.
Expand All @@ -703,12 +746,13 @@ changes:
description: The `buffer` parameter can be an Uint8Array now.
-->

- `buffer` {Buffer|TypedArray|DataView|string}
- `buffer` {Buffer|TypedArray|DataView|ArrayBuffer|string}

Decompress a chunk of data with [Unzip][].

[`.flush()`]: #zlib_zlib_flush_kind_callback
[`Accept-Encoding`]: https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.3
[`ArrayBuffer`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer
[`Buffer`]: buffer.html#buffer_class_buffer
[`Content-Encoding`]: https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.11
[`DataView`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/DataView
Expand Down
24 changes: 18 additions & 6 deletions lib/zlib.js
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
const errors = require('internal/errors');
const Transform = require('_stream_transform');
const { _extend } = require('util');
const { isAnyArrayBuffer } = process.binding('util');
const { isArrayBufferView } = require('internal/util/types');
const binding = process.binding('zlib');
const assert = require('assert').ok;
Expand Down Expand Up @@ -67,6 +68,8 @@ function zlibBuffer(engine, buffer, callback) {
if (isArrayBufferView(buffer) &&
Object.getPrototypeOf(buffer) !== Buffer.prototype) {
buffer = Buffer.from(buffer.buffer, buffer.byteOffset, buffer.byteLength);
} else if (isAnyArrayBuffer(buffer)) {
buffer = Buffer.from(buffer);
}
engine.buffers = null;
engine.nread = 0;
Expand Down Expand Up @@ -114,9 +117,14 @@ function zlibBufferSync(engine, buffer) {
if (typeof buffer === 'string') {
buffer = Buffer.from(buffer);
} else if (!isArrayBufferView(buffer)) {
throw new errors.TypeError('ERR_INVALID_ARG_TYPE',
'buffer',
['string', 'Buffer', 'TypedArray', 'DataView']);
if (isAnyArrayBuffer(buffer)) {
buffer = Buffer.from(buffer);
} else {
throw new errors.TypeError('ERR_INVALID_ARG_TYPE',
'buffer',
['string', 'Buffer', 'TypedArray', 'DataView',
'ArrayBuffer']);
}
}
buffer = processChunkSync(engine, buffer, engine._finishFlushFlag);
if (engine._info)
Expand Down Expand Up @@ -245,9 +253,13 @@ function Zlib(opts, mode) {

dictionary = opts.dictionary;
if (dictionary !== undefined && !isArrayBufferView(dictionary)) {
throw new errors.TypeError('ERR_INVALID_OPT_VALUE',
'dictionary',
dictionary);
if (isAnyArrayBuffer(dictionary)) {
dictionary = Buffer.from(dictionary);
} else {
throw new errors.TypeError('ERR_INVALID_OPT_VALUE',
'dictionary',
dictionary);
}
}

if (opts.encoding || opts.objectMode || opts.writableObjectMode) {
Expand Down
7 changes: 7 additions & 0 deletions test/common/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,13 @@ Checks if `pathname` exists

Returns an instance of all possible `ArrayBufferView`s of the provided Buffer.

### getBufferSources(buf)
* `buf` [&lt;Buffer>]
* return [&lt;BufferSource&#91;&#93;>]

Returns an instance of all possible `BufferSource`s of the provided Buffer,
consisting of all `ArrayBufferView` and an `ArrayBuffer`.

### getCallSite(func)
* `func` [&lt;Function>]
* return [&lt;String>]
Expand Down
4 changes: 4 additions & 0 deletions test/common/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -805,6 +805,10 @@ exports.getArrayBufferViews = function getArrayBufferViews(buf) {
return out;
};

exports.getBufferSources = function getBufferSources(buf) {
return [...exports.getArrayBufferViews(buf), new Uint8Array(buf).buffer];
};

// Crash the process on unhandled rejections.
exports.crashOnUnhandledRejection = function() {
process.on('unhandledRejection',
Expand Down
2 changes: 1 addition & 1 deletion test/parallel/test-zlib-convenience-methods.js
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ const optsInfo = {
for (const [type, expect] of [
['string', expectStr],
['Buffer', expectBuf],
...common.getArrayBufferViews(expectBuf).map((obj) =>
...common.getBufferSources(expectBuf).map((obj) =>
[obj[Symbol.toStringTag], obj]
)
]) {
Expand Down
2 changes: 1 addition & 1 deletion test/parallel/test-zlib-dictionary.js
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ function deflateRawResetDictionaryTest(spdyDict) {
});
}

for (const dict of [spdyDict, ...common.getArrayBufferViews(spdyDict)]) {
for (const dict of [spdyDict, ...common.getBufferSources(spdyDict)]) {
basicDictionaryTest(dict);
deflateResetDictionaryTest(dict);
rawDictionaryTest(dict);
Expand Down
2 changes: 1 addition & 1 deletion test/parallel/test-zlib-not-string-or-buffer.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ const zlib = require('zlib');
code: 'ERR_INVALID_ARG_TYPE',
type: TypeError,
message: 'The "buffer" argument must be one of type string, Buffer, ' +
'TypedArray, or DataView'
'TypedArray, DataView, or ArrayBuffer'
}
);
});