Make rescheduler work with Kubernetes 1.6

[davidopp]

Taints are being moved from annotation to field of Node, and tolerations are being moved from annotation to field of PodSpec. See kubernetes/kubernetes#25320 and kubernetes/kubernetes#38957

There's no 1.6 milestone in this repo, but it is needed for 1.6.

[davidopp]

Also, write release notes for any special considerations when upgrading to 1.6.

[davidopp]

cc/ @fgrzadkowski

[kevin-wangzefeng]

/cc @kubernetes/huawei

[davidopp]

#38957 has merged, so HEAD now has the new behavior (taint/toleration annotations are ignored by scheduler, and now scheduler pays attention to taint/toleration fields).

Code freeze is getting close...

[piosz]

This requires #2381 to be fixed first, because Rescheduler reuse a lot of logic from CA and godeps are organized in the way that Rescheduler needs to use exactly the same Kubernetes version that CA. Since the version in CA hasn't been bumped yet it's not possible to bump it independently in Rescheduler.

cc @mwielgus

[davidopp]

cc/ @kubernetes/release-team

[k82cn]

@piosz , seems rescheduler still dependent on k8s upstream, why not client-go?

[davidopp]

I'm also interested more generally in how close this is to being done for 1.6.
#2381 (comment) is from 8 days ago. I realize we don't have an official code freeze for anything outside of kubernetes/kubernetes but I think this is getting very down to the wire.

[k82cn]

@piosz , I can contribute this if the work is not started :).

[k82cn]

/assign @k82cn

Also assign to myself, I'm going to build a PR based on client-go; if any comments, please let me know.

[piosz]

I'll do it by tomorrow. @k82cn if you want you can try to bump Kubernetes deps there but it's hard;)

[k82cn]

sure, let me try :).

[marun]

@piosz Any update?

[piosz]

I'll send a PR hopefully today.

[ethernetdan]

@piosz this issue is preventing the release of v1.6.0-rc.1. If not today, could you provide a timeframe for completion?

[piosz]

@ethernetdan huge apologies for the delay. I'll try do it today but I'm in CET timezone. In the worst case it will be done tomorrow.

[ethernetdan]

@piosz thanks!