Thoughts on the current data pipeline design.

[hagbard]

Current design:
https://github.com/unicode-org/omnicu/blob/master/docs/data-pipeline.md

Brainstorming doc (please comment):
https://docs.google.com/document/d/1s_DE6zH27yGNv7rcfZEL8K3Hd0F3eIwMEUmbr7qs3lM/edit#

I think that either I am completely misunderstanding what the intended use case for the design is, or it's worth rethinking some of it. Please comment on the doc.

[sffc]

Thanks for this. I replied inline in the doc.

I think we're thinking the same thing, just not fully aligned on terminology. I also agree that it's probably better for data hunks to be a bit larger than the current proposal.

[sffc]

David: can you respond to the responses and additional questions I left in your doc, such that we can get on the same page about the data provider? I would like to iterate on the data provider design with your feedback.

[sffc]

A data provider POC is now checked into the repo, and we have a greater understanding of some of the key takeaways from David's doc, including larger chunk sizes (documented in #145) and error handling (#153). We have a separate issue open to discuss how you pass the data provider to the APIs (#150). I don't know what specifically remains as actionable on this issue, so I am closing it. @hagbard, if you have further suggestions, please open follow-up tickets.