The storage node needs to refine how to load data directories.

[ijsong]

Storage node reads all sub-directories of given volumes to load data for log stream replicas regardless of whether they are valid. Sometimes, the storage node reads a sub-directory that the storage node should have removed before fail-over. It doesn't look like a problem since the admin server will wipe garbage replicas. However, the garbage replica can be an obstacle if the admin server wants to rebalance a log stream to migrate a log stream to a storage node where the garbage replica exists.

So it is more comfortable for the storage node not to load unnecessary data directories. Currently, we can do that by setting --data-dirs and --volume-strict-check flags to the storage node, but it is very ambiguous and prone to error. Moreover, there is a limit for the length of arguments - see ARG_MAX; thus, we cannot list all data directories when there are too many log stream replicas.

I am proposing three small changes:

• Remove --data-dirs and --volume-strict-check flags.
• Remove unnecessary data directories before starting a storage node or during the bootstrap of a storage node.
• Add --ignore-malformed-data-directory to allow an incorrect data directory name. The default behavior of a storage node is not to start when there is a bad directory.