Deprecate mappintracker and adopt stateless tracker #929
Labels
exp/intermediate
Prior experience is likely helpful
kind/enhancement
A net-new feature or improvement to an existing feature
P1
High: Likely tackled by core team if no one steps up
The
mappintracker
component has been used since the very early days.It keeps an in-memory representation of the pinset along with the current PinStatus for each CID. This means that the actual status as seen by Cluster for a CID (pinned) may differ of that actual status in IPFS (unpinned). This also means that the map needs to be filled-in during cluster boot.
To avoid these problems, the main component regularly runs a
StatusSync
. The first time it runs it will trigger a Track operation for each CID which results in a pin/ls operation on IPFS for each CID which is additionally heavy.The
stateless
pin tracker was developed to avoid all this. Rather than keeping all the information in memory, it would be fetched from IPFS when needed. If we need to check if a pin ls pinned, we dopin/ls
then.Additionally, most getting-out-of-sync issues were solved so that essentially does not happen and the
sync
operation is very redundant at this point (for both trackers).On the downside, the
stateless
pin tracker uses RPC calls to access the state, which is super innefficient and effectively loads the pinset onto memory as well by serializing and deserializing it.Let's fix all this.
Actionables
service
flags)state.ReadOnly
so operations likelocalStatus
can use that rather than calling RPC methods. The main problem here is that this is provided by theState
methods in consensus, and this method can only be called when consensus is ready. Thus it will need to be provided in a delayed fashion (SetState
method called appropiately from the Cluster main component (?))stateless
tracker fail. It is very important to understand how theoptracker
module works and how it is used to handle the pinning operation lifetime in scenarios with concurrency (very tricky). TheStateSync
operation incluster.go
will become a private method and should lose most of its logic (it needs to keep things like checking for pin expiration), probably it should have a different name.The text was updated successfully, but these errors were encountered: