Point in time snapshots

[gauravsarma1992]

The PR implements snapshotting a single store into the disk using the copy-on-write approach.

Check the README.md in the PR for more details on the design and implementation.

The PR doesn't implement the strategy of multi shard snapshots yet. It will be implemented
in a subsequent PR.

Test results

Snapshot time for 1 million keys:

• No Reads and no writes is 190ms.
• Reads and no writes is 230ms.
• Reads and with writes during the snapshots is 800ms.

The reads and writes mentioned above are during the snapshotting process on the actual store object.

Few improvements that can be done:

• Use protobuf instead of gob
• Use buffered writes instead of chunk writes
• Improve the locking mechanism of the SnapshotMap

[lucifercr07]

For multiple store instances there'd be snapshot run triggered at one point of time, in that case why'd we need multiple exitCh chan.
From my understanding flow is like below:

• Each Store has one snapshot
• Each snapshot has one flusher
• Each flusher sends exit signal after flush completed for all objects in that store

Also my assumption is each store will create it's own snapshotter and won't be shared among stores is that correct?
Please correct me if I'm wrong.

[gauravsarma1992]

Yes, the size of the exit channel can be 1 as well.
Yes, this is right. However, this PR will not cover the SnapshotManager that will spawn a snapshotter process for each of the stores.

Also my assumption is each store will create it's own snapshotter and won't be shared among stores is that correct?

[lucifercr07]

What are we using this variable for?

[gauravsarma1992]

I think this isn't required now, will remove it.

[lucifercr07]

nit: Shall we update this to?

func (pit *PointInTimeSnapshot) processStoreUpdates() error {
	select {
	case <-pit.ctx.Done():
	case <-pit.exitCh:
	}
	pit.Close()
	return nil
}