You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We modified the pruning logic in this PR to prune over multiple blocks instead of all at once at epoch time, saving us approx 30s in epoch time. This required a dumb solution, which leaves old records existing due to how we store the last key we iterated over and start again from there without retaining the last seen pool pairs. They eventually get pruned on the next epoch, but there will always be old records existing when we use this logic. Iterating from the same start key every time works, but causes us to reiterate over records many times.
Suggested Design
Restructure iterations and/or keys for TWAP in order to allow a per block pruning, while not skipping over old records.
Migrate old keys if needed.
Acceptance Criteria
TBD
The text was updated successfully, but these errors were encountered:
Right now we store every historical TWAP in two ways. Indexed by pool id | asset 0 | asset 1 | timestamp and known as HistoricalPoolIndex, and timestamp | pool id | asset 0 | asset 1 known as HistoricalTimeIndex.
We use the second storage for easing pruning, by just pruning by iterating over all entries from the first timestamp to delete to the last.
However, we can just store this data once on-chain under HistoricalPoolIndex, which is what we want for querying twap's by timestamp. (since we iterate to the nearest timestamp neighbour).
Then when we go to prune, we can just iterate over every pool ID, get the pool ID's assets, then iterate through their records in the pool id | asset 0 | asset 1| {timestamp range} to delete. This will cost one extra pool denom read, for every pool. This seems cheaper than doubling the writes (and deletes when we go to use this) for every TWAP which we do right now.
Background
We modified the pruning logic in this PR to prune over multiple blocks instead of all at once at epoch time, saving us approx 30s in epoch time. This required a dumb solution, which leaves old records existing due to how we store the last key we iterated over and start again from there without retaining the last seen pool pairs. They eventually get pruned on the next epoch, but there will always be old records existing when we use this logic. Iterating from the same start key every time works, but causes us to reiterate over records many times.
Suggested Design
Restructure iterations and/or keys for TWAP in order to allow a per block pruning, while not skipping over old records.
Migrate old keys if needed.
Acceptance Criteria
TBD
The text was updated successfully, but these errors were encountered: