Skip to content

Commit

Permalink
docs: document BATCH limitations (#1118)
Browse files Browse the repository at this point in the history
  • Loading branch information
aaronsteers authored Oct 26, 2022
1 parent 0122d47 commit 067d677
Showing 1 changed file with 7 additions and 0 deletions.
7 changes: 7 additions & 0 deletions docs/batch.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,3 +93,10 @@ class MySink(Sink):
def process_batch_files(self, encoding, storage, files):
# process the batch files
```

## Known Limitations

1. Currently the built-in `BATCH` implementation does not support incremental bookmarks or `STATE` tracking. This work is tracked in [Issue #976](https://github.com/meltano/sdk/issues/976).
2. The `BATCH` implementation is not currently compatible with [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html). This is certainly possible to implement in theory, but it would also introduce some performance penalties. This limitation is tracked in [Issue 1117#](https://github.com/meltano/sdk/issues/1117).

If you are interested in contributing to one or both of these features, please add a comment in the respective issue.

0 comments on commit 067d677

Please sign in to comment.