Pintracker/local operation logic revamp

[hsanjuan]

A post to ipfs will hang and doesn't time out unless ipfs itself does.

We need:

• Timeout control for the POST operations to ipfs
• Bonus: configurable in the config
• Bonus: every ongoing operation is tracked with a context and cancellable at any moment
• Bonus: ipfsconn exposes ongoing operations and a method to cancel them
• Bonus: the pintracker (maptracker) can make use of those when it is pinning an item and receives an unpin request (cancel any pinning operation for that item and unpin)
• Bonus: pintracker removes any related pin operations from the pin queue

[hsanjuan]

I am going to re-use this issue to describe what we need to do in order to improve local operation handling (that is, a pin/unpin/sync/recover requests that try to match the ipfs state to the shared state).

This is logic taken care of by the mappintracker component which talks to the httpipfsconn component as needed.

There are a bunch of problems:

• When something needs to be pinned if goes into pinning state, but we have no way to know if it's just queuing because some other thing is pinning or it's actually being pinned (waiting for ipfs pin request to succeed)
• Once something is queued to pin there is no way to remove it from the queue
• Once something is queued to pin there is no way to know how far behind in the queue it sits
• sync() makes pin status show a timeout error when something has not been pinned in less than pinning_timeout since it was added to the queue, but it is not removed from the queue if it was waiting. Things should not timeout if it was just queued and the "stuck" pin is something else. Since pins are still queued, error states may well turn to pinned later if the queue is processed.
• Due to the above recover() resubmits items to the queue, even if they are still queued.
• RecoverAll() should not return REMOTE items. It should really only return pins which it has recovered or tried to.
• We have no way of canceling an ongoing pin from the pin tracker (see first comment in this issue)
• If an Unpin() request is received, and something is queued or waiting to be pinned, then it's for nothing.

The pin tracker works OK in an scenario with small amount of pin requests that can be processed in time, but not with large sets of pins injected to cluster. The pin tracker implementation is relatively old and naive and needs to step up to current needs.