zoekt: Cleanup failed shards

[keegancsmith]

On sourcegraph.com we have the OOM killer coming in and killing an index while it is running. A side effect of this is most of the disk space on sourcegraph.com's index-search pod is taken up by half written files. They have a name like:

github.com%2Fzeit%2Fnext.js_v16.00000.zoekt976944826

More generally they match the glob *.zoekt[0-9]*.

A RFC detailing how to tackle this problem should be created. A suggestion I have is using an approach similiar to what we do on gitserver. We use a unique temporary directory per gitserver process. If any other directory matches that directories name on startup, we delete it. This may not be good enough.

[tsenart]

@keegancsmith, @kzh: Is this in the works and in time for 3.8? If not, please change the milestone accordingly.

[keegancsmith]

@kzh think you will get this in this week? Taking into account time for review/etc so we can land it by Monday.