docker's data-root will lost on Azure Node restart

[ydye]

Organization Name: OpenPAI

Impact
Currently, PAI is using temp path, but if Azure restarts, temp data will lost. PAI can use other path as the temp path instead, while on the other side OS disk is too small to be used.

Short summary about the issue/question:

If you configure docker ```data-root``` to a tmp path and data on the tmp path is lost, OpenPAI will crushed.
What's more, kubelet won't restart due to the loss of docker's ```data-root```. 

https://forums.docker.com/t/unable-to-restart-the-container-when-the-data-of-data-root-is-lost/70581

Brief what process you are following:

How to reproduce it:

sudo systemctl stop docker
sudo rm -rf /path/to/docker/data/root
sudo systemctl start docker

OpenPAI Environment:

• OpenPAI version:
• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release):
• Kernel (e.g. uname -a):
• Hardware (e.g. core number, memory size, storage size, GPU type etc.):
• Others:

Anything else we need to know:

[fanyangCS]

@ydye, do you have time to fix this?

[ydye]

@fanyangCS If you accept the solution to start kubelet by systemd, instead of docker.

[fanyangCS]

@ydye, I agree with your solution. Please make sure user can upgrade smoothly from old to new solution.

[ydye]

@fanyangCS Obviously, it will be a broken upgrade. User will have to stop all service and restart all k8s node.

[fanyangCS]

That’s ok.

[scarlett2018]

@ydye - may you provide an estimation for this task? we are considering putting it in Aug release.

[ydye]

#3307

[ydye]

Code Merged.

[fanyangCS]

could you deploy it to the INT bed?