You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Dashboard crawls depend on point-in-time versions of agency data.json files. If the jobs does not run on a specific day, there is no way to go back in time to get the data.json. This means that if a nightly job fails, it means data loss. If it fails several days in a row, it becomes a high priority issue.
Having a time sensitive operation is difficult for O&M.
How to reproduce
Disable the cron for day
Nightly cron does not run
Run the cron job the following day
Expected behavior
Crawl can create data for the missed day.
Actual behavior
Data is lost for the missed day.
The text was updated successfully, but these errors were encountered:
Not sure that we can ever fix this. We don't have a time machine and we don't control agency data.json files. Still, we can document this undesired property and identify mitigations that make it less painful for O&M.
E.g. if the daily snapshotting of data.json became commoditized, the nightly job could be parameterized with a date range, and just operate on the snapshots.
Dashboard crawls depend on point-in-time versions of agency data.json files. If the jobs does not run on a specific day, there is no way to go back in time to get the data.json. This means that if a nightly job fails, it means data loss. If it fails several days in a row, it becomes a high priority issue.
Having a time sensitive operation is difficult for O&M.
How to reproduce
Expected behavior
Crawl can create data for the missed day.
Actual behavior
Data is lost for the missed day.
The text was updated successfully, but these errors were encountered: