Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add 5 min lookback for immutable streams to prevent missing records #45

Merged
merged 3 commits into from
Dec 6, 2019

Conversation

dmosorast
Copy link
Contributor

Description of change

We've observed a situation where events and balance_transactions can skip data when the record gets created exactly on (or very near) a date_window break. There appears to be some amount of time between when a record is stamped as created and when it's made available by the API. Likely only on the order of seconds.

For example

  1. Balance Transaction BT1 gets created at 1574260351.
  2. Tap, due to configuration and happenstance of timing, issues a request with parameters created[gte]=1574258564&created[lt]=1574260351&limit=100
  3. BT1 does not get returned with this result set, but the tap bookmarks the second after its created time, so it never gets re-requested

The Fix
This PR is to add a 5 min lookback window to the start_date of all requests for balance_transactions and events. 5 minutes was chosen to account for potential scenarios when Stripe is under load and this lag time becomes on the order of minutes.

  • The assumption is that 5 minutes shouldn't increase duplicates too much

Outstanding
This is likely also an issue for the "creation" of other streams' records, but since they are mutable, the record gets captured due to the events_updates pattern and causes them to be caught if missed.

Keeping this scoped to only the immutable streams to keep the surface of the change low.

Manual QA steps

  • Ran this for balance_transactions and events, and it was able to advance the bookmarks.

Risks

  • Row volume and time-to-sync could increase in situations where transaction and event volume is extremely high. I don't suspect this is cause for concern.

Rollback steps

  • revert this branch or adjust the lookback

@KAllan357 KAllan357 self-requested a review December 6, 2019 21:12
@dmosorast dmosorast merged commit 190face into master Dec 6, 2019
@dmosorast dmosorast deleted the fix/lookback_for_immutable_streams branch December 6, 2019 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants