[Snapshot Interop] Add Logic for Remote Store Cleanup in Lock Manager if there are no more locks.

[harishbhakuni]

Lock Manager Implementation that we have today only takes care of deletion of locks. Also, Remote Store GC only runs after a flush/refresh. So if a Remote Store Index is deleted but snapshot holds lock on it, there will be some stale segments that we will have in remote store which will never cleanup.

For this, long term solution would be to create a Poller which will periodically looks into cluster state to get the current live indices and then in remote store cleanup all the stale files for indices which are not present in cluster state anymore. Also, Remote Store GC which runs as part of refresh runs asynchronously, so if GC components fails to delete some stale segments, this poller will be responsible for cleanup of those stale files.

As a short term solution, for now we will be extending Lock Manager to cleanup stale segments as well if we are releasing the last lock from the segment store and index is not present in the cluster.

Update : The short term solution is already merged . Reopening this issue for the long term solution