set FailureActions for docker, kubelet, kubeproxy on windows

[andyzhangx]

What this PR does / why we need it:
Original FailureActions setting of services docker, kubelet, kubeproxy are not set on Windows, so once service is down, it will never started, user needs to restart VM to make service running again, this PR set FailureActions as:
restart after 60000ms (1 minute) for the first, second, and third failures. Reset the counter after 86400 seconds (1 day)

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #3888

Special notes for your reviewer:
With this PR, after windows node set up:

PS C:\Users\azureuser> get-itemproperty hklm:\system\currentcontrolset\services\kubeproxy

Type                             : 16
Start                            : 2
ErrorControl                     : 1
ImagePath                        : c:\k\nssm.exe
DisplayName                      : Kubeproxy
ObjectName                       : LocalSystem
DelayedAutostart                 : 0
FailureActionsOnNonCrashFailures : 1
DependOnService                  : {Kubelet}
Description                      : Kubeproxy
FailureActions                   : {128, 81, 1, 0...}
PSPath                           : Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\system\currentcontrolset\serv
                                   ices\kubeproxy
PSParentPath                     : Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\system\currentcontrolset\serv
                                   ices
PSChildName                      : kubeproxy
PSDrive                          : HKLM
PSProvider                       : Microsoft.PowerShell.Core\Registry



PS C:\Users\azureuser> get-itemproperty hklm:\system\currentcontrolset\services\kubelet

Type                             : 16
Start                            : 2
ErrorControl                     : 1
ImagePath                        : c:\k\nssm.exe
DisplayName                      : Kubelet
ObjectName                       : LocalSystem
DelayedAutostart                 : 0
FailureActionsOnNonCrashFailures : 1
Description                      : Kubelet
FailureActions                   : {128, 81, 1, 0...}
PSPath                           : Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\system\currentcontrolset\serv
                                   ices\kubelet
PSParentPath                     : Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\system\currentcontrolset\serv
                                   ices
PSChildName                      : kubelet
PSDrive                          : HKLM
PSProvider                       : Microsoft.PowerShell.Core\Registry



PS C:\Users\azureuser> get-itemproperty hklm:\system\currentcontrolset\services\docker

Type           : 16
Start          : 2
ErrorControl   : 1
ImagePath      : "C:\Program Files\Docker\dockerd.exe" --run-service
ObjectName     : LocalSystem
FailureActions : {128, 81, 1, 0...}
PSPath         : Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\system\currentcontrolset\services\docker
PSParentPath   : Microsoft.PowerShell.Core\Registry::HKEY_LOCAL_MACHINE\system\currentcontrolset\services
PSChildName    : docker
PSDrive        : HKLM
PSProvider     : Microsoft.PowerShell.Core\Registry

If applicable:

• documentation
• unit tests
• tested backward compatibility (ie. deploy with previous version, upgrade with this branch)

Release note:

set FailureActions for docker, kubelet, kubeproxy on windows

@PatrickLang could you review it first, thanks.
cc @adelina-t @feiskyer

[codecov]

Codecov Report

Merging #3905 into master will decrease coverage by 0.1%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3905      +/-   ##
==========================================
- Coverage   57.52%   57.42%   -0.11%     
==========================================
  Files         108      109       +1     
  Lines       16744    16681      -63     
==========================================
- Hits         9632     9579      -53     
+ Misses       6341     6331      -10     
  Partials      771      771

[feiskyer]

reset the counter after 86400 seconds (1 day)

Could we set this value to something smaller, e.g. 10 min? If kubelet stopped because of something wrong in itself, it may be fixed by restarting kubelet itself (e.g. eviction or garbage collection). So I think it's better to try more times on failure.

By the way, kubelet restart policy is always on Linux.

[PatrickLang]

Yes, I agree with @feiskyer . Can you set reset= 900 (15 minutes) ?

If it's restarting repeatedly, that means it would:

• 1st restart after 1 minute
• 2nd restart after 1 minute
• 3rd restart after 1 minute
• wait 12 minutes
• restart again ...

If there's a resource leak causing the service to go down - having a 12 minute window would be good so that we can connect, and manually stop the service to prevent restart. Then you can start the service under a debugger instead.

[PatrickLang]

Can you update to reset= 900 ?

[andyzhangx]

@PatrickLang updated commit, PTAL, thanks.

[PatrickLang]

/ltgm

[PatrickLang]

/approve
/lgtm

[acs-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: andyzhangx, PatrickLang
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: jackfrancis

If they are not already assigned, you can assign the PR to them by writing /assign @jackfrancis in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[PatrickLang]

/assign @jackfrancis for approve

[acs-bot]

@PatrickLang: GitHub didn't allow me to assign the following users: for, approve.

Note that only Azure members and repo collaborators can be assigned.
For more information please see the contributor guide

In response to this:

/assign @jackfrancis for approve

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[andyzhangx]

ping @jackfrancis