Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ExternalReader that delegates all reading to an external function #57

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -625,10 +625,17 @@ Default: `5`

Max amount of subsequent chunks allowed to read in which exifr searches for data segments and blocks. I.e. failsafe that prevents from reading the whole file if it does not contain all of the segments or blocks requested in `options`.

This limit is bypassed if multi-segment segments ocurs in the file and if [`options.multiSegment`](#optionsmultisegment) allows reading all of them.
This limit is bypassed if multi-segment segments occurs in the file and if [`options.multiSegment`](#optionsmultisegment) allows reading all of them.

*If the exif isn't found within N chunks (64\*5 = 320KB) it probably isn't in the file and it's not worth reading anymore.*

#### `options.externalReader`
Type: `function`
<br>
Default: `undefined`

Async function that receives three parameters - `input`, `offset`, and `length` – and returns an `ArrayBuffer` containing the requested chunk from the input source. If `offset` is missing, `undefined`, or `null`, the function should return the whole file.

### Output format

#### `options.mergeOutput`
Expand Down
1 change: 1 addition & 0 deletions index.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ interface Options extends FormatOptions {
firstChunkSize?: number,
chunkSize?: number,
chunkLimit?: number,
externalReader?: function,
}

interface IRotation {
Expand Down
1 change: 1 addition & 0 deletions src/bundles/lite.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ export default mini
// File Readers
import '../file-readers/UrlFetcher.mjs'
import '../file-readers/BlobReader.mjs'
import '../file-readers/ExternalReader.mjs'

// File Parser
import '../file-parsers/jpeg.mjs'
Expand Down
22 changes: 22 additions & 0 deletions src/file-readers/ExternalReader.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
import {fileReaders} from '../plugins.mjs'
import {ChunkedReader} from './ChunkedReader.mjs'

export class ExternalReader extends ChunkedReader {

async readWhole() {
this.chunked = false
let arrayBuffer = await this.options.externalReader(this.input)
this._swapArrayBuffer(arrayBuffer)
}

async _readChunk(offset, length) {
let abChunk = await this.options.externalReader(this.input, offset, length)
if (typeof abChunk === 'undefined') return undefined
let bytesRead = abChunk.byteLength
if (bytesRead !== length) this.size = offset + bytesRead
return this.set(abChunk, offset, true)
}

}

fileReaders.set('external', ExternalReader)
4 changes: 3 additions & 1 deletion src/reader.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ import {fileReaders} from './plugins.mjs'
const INVALID_INPUT = 'Invalid input argument'

export function read(arg, options) {
if (typeof arg === 'string')
if (options.externalReader && (typeof options.externalReader === 'function'))
return callReaderClass(arg, options, 'external')
else if (typeof arg === 'string')
return readString(arg, options)
else if (platform.browser && !platform.worker && arg instanceof HTMLImageElement)
return readString(arg.src, options)
Expand Down
14 changes: 14 additions & 0 deletions test/reader.spec.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,20 @@ describe('reader', () => {
assert.isObject(output, `output is undefined`)
})

it(`ExternalReader`, async() => {
let called;

const externalReader = async (input, offset, length) => {
called = {input, offset, length};
let buffer = await getFile(input);
return buffer.slice(offset, length);
}

var output = await exifr.parse('IMG_20180725_163423.jpg', {externalReader})
assert.isObject(output, `output is undefined`)
assert.isObject(called)
})

isNode && it(`Node: Buffer`, async () => {
var buffer = await fs.readFile(getPath('IMG_20180725_163423.jpg'))
var output = await exifr.parse(buffer)
Expand Down