Fix the threading issues of the directory utility on nix platforms

[nkolev92]

Bug

Fixes: NuGet/Home#7977
Might be the root cause for some of the reports in NuGet/Home#7341
Regression: No

• Last working version: This was never correct
• How are we preventing it in future:

Fix

Details: Fix the directory utility threading issues.

During the investigation of NuGet/Home#7908, I found that this utility has threading issues. I have a test app that reliably repros the behavior: https://github.com/nkolev92/NuGet.Tools/tree/master/DirectoryUtilityRepro/TestApp

The Directory.CreateDirectory & Directory.Move methods do not guarantee thread safety. They merely guarantee filesystem consistency.

Through my tests I narrowed down the issue to Directory.Move not throwing every time when the dest path exists in the case where multiple threads are trying to create the same directory. Directory.Move never claims thread safety, but rather only filesystem consistency. Effectively on Linux we’d have multiple moves succeed, because the underlying “rename” syscall allows for the destdir to be overridden.

If I try to introduce the same concurrency, the failure is that a “process is using the file”, so not sure if the root cause is the same.

I considered fixing this with more granular locking but then I'd need to use a concurrent collection which adds its own complexity.
We can take the "small" performance hit here for the sake of correctness.

//cc @JeremyKuhne

Testing/Validation

Tests Added: No
Reason for not adding tests:
Validation: I ran the repro script overnight and did not reproduce the issue. https://github.com/nkolev92/NuGet.Tools/tree/master/DirectoryUtilityRepro/TestApp
Without the fixed behavior, I can reliably reproduce the issue in > 100 runs.