-
Notifications
You must be signed in to change notification settings - Fork 95
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(csv): new api and examples section
- Loading branch information
Showing
10 changed files
with
113 additions
and
42 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
--- | ||
title: API | ||
description: CSV - stream, callback and sync APIs | ||
keywords: ['csv', 'parse', 'parser', 'api', 'callback', 'stream', 'sync', 'promise'] | ||
sort: 3 | ||
--- | ||
|
||
# CSV API | ||
|
||
There are multiple APIs and styles available, each with their own advantages and disadvantages. Under the hood, they are all based on the same implementation. | ||
|
||
* [Sync API](/parse/api/sync/) | ||
The sync API provides simplicity, readability and convenience. Like for the callback API, it is meant for small dataset which fit in memory. | ||
* [Stream API](/parse/api/stream/) | ||
The stream API might not be the most pleasant API to use but is scalable. | ||
* [Callback API](/parse/api/callback/) | ||
The callback API buffers all the emitted data from the stream API into a single object which is passed to a user provided function. Passing a function is easier than implementing the stream events function but it implies that the all dataset must fit into the available memory and it will only be available after the last record has been processed. This is usually not recommanded, use the Sync API instead. | ||
|
||
For additional usages and examples, you may refer to: | ||
|
||
* [the parser API pages](/parse/api/), | ||
* [the stringifier API pages](/stringify/api/), | ||
* [the "samples" folder](https://github.com/adaltas/node-csv/tree/master/packages/csv/samples) | ||
* [the "test" folder](https://github.com/adaltas/node-csv/tree/master/packages/csv/test). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
--- | ||
title: Callback | ||
description: CSV - learn how to leverage the Node.js stream pipe API with CSV | ||
keywords: ['csv', 'parse', 'parser', 'example', 'recipe', 'stream', 'async', 'pipe', 'read', 'write'] | ||
sort: 3.2 | ||
--- | ||
|
||
# Callback API | ||
|
||
Also available in the `csv` module is the callback API. The all dataset is available in the second callback argument. Thus it will not scale with large dataset. The [callback example](https://github.com/adaltas/node-csv/blob/master/packages/csv/samples/callback.js) initialize each CSV function sequentially, with the output of the previous one. Note, for the sake of clarity, the example doesn't deal with error management. It is enough spaghetti code. | ||
|
||
`embed:packages/csv/samples/callback.js` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
--- | ||
title: Stream | ||
description: CSV - learn how to leverage the Node.js stream pipe API with CSV | ||
keywords: ['csv', 'parse', 'parser', 'example', 'recipe', 'stream', 'async', 'pipe', 'read', 'write'] | ||
sort: 3.1 | ||
--- | ||
|
||
# Node.js stream API | ||
|
||
The Node.js stream API is scalable and offers the greatest control over the data flow. | ||
|
||
## Using the pipe API | ||
|
||
Pipes in Node.js is a native functionnality provided by the [stream API](https://nodejs.org/api/stream.html). It behave just like Unix pipes where the output of a process, here a stream reader, is redirected as the input of the following process, here a stream writer. | ||
|
||
The [pipe example](https://github.com/adaltas/node-csv/blob/master/packages/csv/samples/pipe.js) is quite readable while also scalable: | ||
|
||
`embed:packages/csv/samples/pipe.js` | ||
|
||
## Using the native stream functions | ||
|
||
The native stream functions provide flexibity but comes at the cost of being more verbose and harder to write. Data is consumed inside the `readable` event with the `stream.read` function. It is then written by calling the `stream.write` function. The [stream example](https://github.com/adaltas/node-csv/blob/master/packages/csv/samples/stream.js) illustrates how to initialize each packages and how to plug them. | ||
|
||
`embed:packages/csv/samples/stream.js` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
--- | ||
title: Sync | ||
description: CSV - learn how to leverage the Node.js stream pipe API with CSV | ||
keywords: ['csv', 'parse', 'parser', 'example', 'recipe', 'stream', 'async', 'pipe', 'read', 'write'] | ||
sort: 3.3 | ||
--- | ||
|
||
# Sync API | ||
|
||
The sync API behave like [pure functions](https://en.wikipedia.org/wiki/Pure_function). For a given input, it always produce the same output. | ||
|
||
Because of its simplicity, this is the recommended approach if you don't need scalability and if your dataset fit in memory. | ||
|
||
The module to import is `csv/sync`. The [sync example](https://github.com/adaltas/node-csv/blob/master/packages/csv/samples/sync.js) illustrate its usage. | ||
|
||
```embed:packages/csv/samples/sync.js``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
--- | ||
title: File system interaction | ||
description: Read and write files with pipes | ||
keywords: ['csv', 'example', 'recipe', 'file', 'fs', 'read', 'write'] | ||
--- | ||
|
||
# File system interaction | ||
|
||
The native Node.js File System module named `fs` is used to read and write the content of a file. | ||
|
||
This [file system recipe](https://github.com/adaltas/node-csv/blob/master/packages/csv/samples/example.fs.js) illustrates how to use the pipe function. | ||
|
||
`embed:packages/csv/samples/example.fs.js` |