From cd04a229d90707dc689c09a2925a6c34a9e540d2 Mon Sep 17 00:00:00 2001 From: caleb miles Date: Tue, 14 Mar 2017 13:13:49 -0700 Subject: [PATCH 01/30] Create empty release notes doc for commenting --- release-1.6/release-notes-draft.md | 1 + 1 file changed, 1 insertion(+) create mode 100644 release-1.6/release-notes-draft.md diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md new file mode 100644 index 00000000000..8b137891791 --- /dev/null +++ b/release-1.6/release-notes-draft.md @@ -0,0 +1 @@ + From 84dc55a21abf15cb2bbe75e433f6f7f3dd94e96f Mon Sep 17 00:00:00 2001 From: caleb miles Date: Wed, 15 Mar 2017 14:19:17 -0700 Subject: [PATCH 02/30] Adds skeleton for 1.6 release notes adds sections for each SIG --- release-1.6/release-notes-draft.md | 125 +++++++++++++++++++++++++++++ 1 file changed, 125 insertions(+) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index 8b137891791..0f0db9edbe2 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -1 +1,126 @@ +***if you can't edit this file please edit the [associated doc](https://docs.google.com/document/d/15iUswVe3gwJIR6JvFN7GSaYfd1fImblw261cZtInB-U/edit?usp=sharing) and comment on the [PR](https://github.com/kubernetes/features/pull/203) that changes have been made*** +## API Machinery +### New Features +### Notable Changes +### Breaking Changes + +## AWS +### New Features +### Notable Changes +### Breaking Changes + +## Apps +### New Features +### Notable Changes +### Breaking Changes + +## Auth +### New Features +### Notable Changes +### Breaking Changes + +## Autoscaling +### New Features +### Notable Changes +### Breaking Changes + +## Big Data +### New Features +### Notable Changes +### Breaking Changes + +## CLI +### New Features +### Notable Changes +### Breaking Changes + +## Cluster Lifecycle +### New Features +### Notable Changes +### Breaking Changes + +## Cluster Ops +### New Features +### Notable Changes +### Breaking Changes + +## Contributor Experience +### New Features +### Notable Changes +### Breaking Changes + +## Docs +### New Features +### Notable Changes +### Breaking Changes + +## Federation +### New Features +### Notable Changes +### Breaking Changes + +## Instrumentation +### New Features +### Notable Changes +### Breaking Changes + +## Network +### New Features +### Notable Changes +### Breaking Changes + +## Node +### New Features +### Notable Changes +### Breaking Changes + +## On Prem +### New Features +### Notable Changes +### Breaking Changes + +## OpenStack +### New Features +### Notable Changes +### Breaking Changes + +## PM +### New Features +### Notable Changes +### Breaking Changes + +## Scalability +### New Features +### Notable Changes +### Breaking Changes + +## Scheduling +### New Features +### Notable Changes +### Breaking Changes + +## Service Catalog +### New Features +### Notable Changes +### Breaking Changes + +## Storage +### New Features +### Notable Changes +### Breaking Changes + +## Testing +### New Features +### Notable Changes +### Breaking Changes + +## UI +### New Features +### Notable Changes +### Breaking Changes + +## Windows +### New Features +### Notable Changes +### Breaking Changes From 0e611ccade147d1d50a2e6f3b1960b9b48c317b9 Mon Sep 17 00:00:00 2001 From: caleb miles Date: Wed, 15 Mar 2017 14:45:44 -0700 Subject: [PATCH 03/30] update section for SIG PM no updates for 1.6 release --- release-1.6/release-notes-draft.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index 0f0db9edbe2..c8ae23f823f 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -86,9 +86,8 @@ ### Breaking Changes ## PM -### New Features -### Notable Changes -### Breaking Changes + +[ Nothing for this release] ## Scalability ### New Features From 81326831835374b20d3c955fad999861b2de395a Mon Sep 17 00:00:00 2001 From: caleb miles Date: Wed, 15 Mar 2017 16:16:01 -0700 Subject: [PATCH 04/30] Add automatically generated release notes to draft Humans can begin curation of these release notes --- release-1.6/release-notes-draft.md | 493 +++++++++++++++++++++++++++++ 1 file changed, 493 insertions(+) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index c8ae23f823f..b913a8d5a26 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -1,5 +1,7 @@ ***if you can't edit this file please edit the [associated doc](https://docs.google.com/document/d/15iUswVe3gwJIR6JvFN7GSaYfd1fImblw261cZtInB-U/edit?usp=sharing) and comment on the [PR](https://github.com/kubernetes/features/pull/203) that changes have been made*** +***automatically generated release notes [are at the bottom](#auto-generated-release-notes) please curate and move those notes into the correct SIG*** + ## API Machinery ### New Features ### Notable Changes @@ -123,3 +125,494 @@ ### New Features ### Notable Changes ### Breaking Changes + +# Auto Generated Release Notes +## (please move information into the correct SIG) + +**Release Note Preview - generated on Wed Mar 15 22:47:14 UTC 2017** + +# Branch master + +[Documentation](https://docs.k8s.io) & [Examples](https://releases.k8s.io/master/examples) + +## Major Themes + +* TBD + +## Other notable improvements + +* TBD + +## Known Issues + +* TBD + +## Provider-specific Notes + +* TBD + +## Changelog since v1.5.0 + +### Action Required + +* Deployment now fully respects ControllerRef to avoid fighting over Pods and ReplicaSets. At the time of upgrade, **you must not have Deployments with selectors that overlap**, or else [ownership of ReplicaSets may change](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md#upgrading). ([#42175](https://github.com/kubernetes/kubernetes/pull/42175), [@enisoc](https://github.com/enisoc)) +* StatefulSet now respects ControllerRef to avoid fighting over Pods. At the time of upgrade, **you must not have StatefulSets with selectors that overlap** with any other controllers (such as ReplicaSets), or else [ownership of Pods may change](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md#upgrading). ([#42080](https://github.com/kubernetes/kubernetes/pull/42080), [@enisoc](https://github.com/enisoc)) +* The --dns-provider argument of 'kubefed init' is now mandatory and does not default to `google-clouddns`. To initialize a Federation control plane with Google Cloud DNS, use the following invocation: 'kubefed init --dns-provider=google-clouddns' ([#42092](https://github.com/kubernetes/kubernetes/pull/42092), [@marun](https://github.com/marun)) +* Change taints/tolerations to api fields ([#38957](https://github.com/kubernetes/kubernetes/pull/38957), [@aveshagarwal](https://github.com/aveshagarwal)) +* Promote certificates.k8s.io to beta and enable it by default. Users using the alpha certificates API should delete v1alpha1 CSRs from the API before upgrading and recreate them as v1beta1 CSR after upgrading. ([#39772](https://github.com/kubernetes/kubernetes/pull/39772), [@mikedanese](https://github.com/mikedanese)) +* include bootstrap admin in super-user group, ensure tokens file is correct on upgrades ([#39537](https://github.com/kubernetes/kubernetes/pull/39537), [@liggitt](https://github.com/liggitt)) +* Switch default etcd version to 3.0.14. ([#36229](https://github.com/kubernetes/kubernetes/pull/36229), [@wojtek-t](https://github.com/wojtek-t)) + * Switch default storage backend flag in apiserver to `etcd3` mode. +* RBAC's special handling of the user "*" in RoleBinding and ClusterRoleBinding objects is deprecated and will be removed in v1beta1. To match all users, explicitly bind to the group "system:authenticated" and/or "system:unauthenticated". Existing v1alpha1 bindings to the user "*" will be automatically converted to the group "system:authenticated". ([#38981](https://github.com/kubernetes/kubernetes/pull/38981), [@liggitt](https://github.com/liggitt)) +* The 'endpoints.beta.kubernetes.io/hostnames-map' annotation is no longer supported. Users can use the 'Endpoints.subsets[].addresses[].hostname' field instead. ([#39284](https://github.com/kubernetes/kubernetes/pull/39284), [@bowei](https://github.com/bowei)) +* `federation/deploy/deploy.sh` was an interim solution introduced in Kubernetes v1.4 to simplify the federation control plane deployment experience. Now that we have `kubefed`, we are deprecating `deploy.sh` scripts. ([#38902](https://github.com/kubernetes/kubernetes/pull/38902), [@madhusudancs](https://github.com/madhusudancs)) +* Cluster federation servers have changed the location in etcd where federated services are stored, so existing federated services must be deleted and recreated. Before upgrading, export all federated services from the federation server and delete the services. After upgrading the cluster, recreate the federated services from the exported data. ([#37770](https://github.com/kubernetes/kubernetes/pull/37770), [@enj](https://github.com/enj)) +* etcd2: watching from 0 returns all initial states as ADDED events ([#38079](https://github.com/kubernetes/kubernetes/pull/38079), [@hongchaodeng](https://github.com/hongchaodeng)) + +### Other notable changes + +* Add -p to mkdirs in gci-mounter function of gce configure.sh script ([#43134](https://github.com/kubernetes/kubernetes/pull/43134), [@shyamjvs](https://github.com/shyamjvs)) +* Bumped rescheduler version to 0.3.0 ([#43106](https://github.com/kubernetes/kubernetes/pull/43106), [@piosz](https://github.com/piosz)) +* kubeadm: `kubeadm reset` won't drain and remove the current node anymore ([#42713](https://github.com/kubernetes/kubernetes/pull/42713), [@luxas](https://github.com/luxas)) +* hack/godep-restore.sh: use godep v79 which works ([#42965](https://github.com/kubernetes/kubernetes/pull/42965), [@sttts](https://github.com/sttts)) +* Patch CVE-2016-8859 in gcr.io/google-containers/cluster-proportional-autoscaler-amd64 ([#42933](https://github.com/kubernetes/kubernetes/pull/42933), [@timstclair](https://github.com/timstclair)) +* Disable devicemapper thin_ls due to excessive iops ([#42899](https://github.com/kubernetes/kubernetes/pull/42899), [@dashpole](https://github.com/dashpole)) +* Introduce new generator for apps/v1beta1 deployments ([#42362](https://github.com/kubernetes/kubernetes/pull/42362), [@soltysh](https://github.com/soltysh)) +* Use Prometheus instrumentation conventions ([#36704](https://github.com/kubernetes/kubernetes/pull/36704), [@fabxc](https://github.com/fabxc)) +* Add new DaemonSet status fields to kubectl printer and describer. ([#42843](https://github.com/kubernetes/kubernetes/pull/42843), [@janetkuo](https://github.com/janetkuo)) +* Dropped the support for docker 1.9.x and the belows. ([#42694](https://github.com/kubernetes/kubernetes/pull/42694), [@dchen1107](https://github.com/dchen1107)) +* DaemonSet now respects ControllerRef to avoid fighting over Pods. ([#42173](https://github.com/kubernetes/kubernetes/pull/42173), [@enisoc](https://github.com/enisoc)) +* restored normalization of custom `--etcd-prefix` when `--storage-backend` is set to etcd3 ([#42506](https://github.com/kubernetes/kubernetes/pull/42506), [@liggitt](https://github.com/liggitt)) +* kubelet created cgroups follow lowercase naming conventions ([#42497](https://github.com/kubernetes/kubernetes/pull/42497), [@derekwaynecarr](https://github.com/derekwaynecarr)) +* Support whitespace in command path for gcp auth plugin ([#41653](https://github.com/kubernetes/kubernetes/pull/41653), [@jlowdermilk](https://github.com/jlowdermilk)) +* Updates the dnsmasq cache/mux layer to be managed by dnsmasq-nanny. ([#41826](https://github.com/kubernetes/kubernetes/pull/41826), [@bowei](https://github.com/bowei)) + * dnsmasq-nanny manages dnsmasq based on values from the + * kube-system:kube-dns configmap: + * "stubDomains": { + * "acme.local": ["1.2.3.4"] + * }, + * is a map of domain to list of nameservers for the domain. This is used + * to inject private DNS domains into the kube-dns namespace. In the above + * example, any DNS requests for *.acme.local will be served by the + * nameserver 1.2.3.4. + * "upstreamNameservers": ["8.8.8.8", "8.8.4.4"] + * is a list of upstreamNameservers to use, overriding the configuration + * specified in /etc/resolv.conf. +* kubelet exports metrics for cgroup management ([#41988](https://github.com/kubernetes/kubernetes/pull/41988), [@sjenning](https://github.com/sjenning)) +* kubectl: respect deployment strategy parameters for rollout status ([#41809](https://github.com/kubernetes/kubernetes/pull/41809), [@kargakis](https://github.com/kargakis)) +* Remove cmd/kube-discovery from the tree since it's not necessary anymore ([#42070](https://github.com/kubernetes/kubernetes/pull/42070), [@luxas](https://github.com/luxas)) +* kubeadm: Hook up kubeadm against the BootstrapSigner ([#41417](https://github.com/kubernetes/kubernetes/pull/41417), [@luxas](https://github.com/luxas)) +* Federated Ingress over GCE no longer requires separate firewall rules to be created for each cluster to circumvent flapping firewall health checks. ([#41942](https://github.com/kubernetes/kubernetes/pull/41942), [@csbell](https://github.com/csbell)) +* ScaleIO Kubernetes Volume Plugin added enabling pods to seamlessly access and use data stored on ScaleIO volumes. ([#38924](https://github.com/kubernetes/kubernetes/pull/38924), [@vladimirvivien](https://github.com/vladimirvivien)) +* Pods are launched in a separate cgroup hierarchy than system services. ([#42350](https://github.com/kubernetes/kubernetes/pull/42350), [@vishh](https://github.com/vishh)) +* Experimental support to reserve a pod's memory request from being utilized by pods in lower QoS tiers. ([#41149](https://github.com/kubernetes/kubernetes/pull/41149), [@sjenning](https://github.com/sjenning)) +* Juju: Disable anonymous auth on kubelet ([#41919](https://github.com/kubernetes/kubernetes/pull/41919), [@Cynerva](https://github.com/Cynerva)) +* Remove support for debian masters in GCE kube-up. ([#41666](https://github.com/kubernetes/kubernetes/pull/41666), [@mikedanese](https://github.com/mikedanese)) +* Implement bulk polling of volumes ([#41306](https://github.com/kubernetes/kubernetes/pull/41306), [@gnufied](https://github.com/gnufied)) +* stop kubectl edit from updating the last-applied-configuration annotation when --save-config is unspecified or false. ([#41924](https://github.com/kubernetes/kubernetes/pull/41924), [@ymqytw](https://github.com/ymqytw)) +* kubeadm: Rename some flags for beta UI and fixup some logic ([#42064](https://github.com/kubernetes/kubernetes/pull/42064), [@luxas](https://github.com/luxas)) +* StorageClassName attribute has been added to PersistentVolume and PersistentVolumeClaim objects and should be used instead of annotation `volume.beta.kubernetes.io/storage-class`. The beta annotation is still working in this release, however it will be removed in a future release. ([#42128](https://github.com/kubernetes/kubernetes/pull/42128), [@jsafrane](https://github.com/jsafrane)) +* Remove Azure kube-up as the Azure community has focused efforts elsewhere. ([#41672](https://github.com/kubernetes/kubernetes/pull/41672), [@mikedanese](https://github.com/mikedanese)) +* Fluentd-gcp containers spawned by DaemonSet are now configured using ConfigMap ([#42126](https://github.com/kubernetes/kubernetes/pull/42126), [@crassirostris](https://github.com/crassirostris)) +* Modified kubemark startup scripts to restore master on reboot ([#41980](https://github.com/kubernetes/kubernetes/pull/41980), [@shyamjvs](https://github.com/shyamjvs)) +* Adds a new API resource `PodPreset` and admission controller to enable defining cross-cutting injection of Volumes and Environment into Pods. ([#41931](https://github.com/kubernetes/kubernetes/pull/41931), [@jessfraz](https://github.com/jessfraz)) +* AWS cloud provider: allow to run the master with a different AWS account or even on a different cloud provider than the nodes. ([#39996](https://github.com/kubernetes/kubernetes/pull/39996), [@scheeles](https://github.com/scheeles)) +* Update defaultbackend image to 1.3 ([#42212](https://github.com/kubernetes/kubernetes/pull/42212), [@timstclair](https://github.com/timstclair)) +* Allow the Horizontal Pod Autoscaler controller to talk to the metrics API and custom metrics API as standard APIs. ([#41824](https://github.com/kubernetes/kubernetes/pull/41824), [@DirectXMan12](https://github.com/DirectXMan12)) +* Implement support for mount options in PVs ([#41906](https://github.com/kubernetes/kubernetes/pull/41906), [@gnufied](https://github.com/gnufied)) +* Introduce apps/v1beta1.Deployments resource with modified defaults compared to extensions/v1beta1.Deployments. ([#39683](https://github.com/kubernetes/kubernetes/pull/39683), [@soltysh](https://github.com/soltysh)) +* Add DNS suffix search list support in Windows kube-proxy. ([#41618](https://github.com/kubernetes/kubernetes/pull/41618), [@JiangtianLi](https://github.com/JiangtianLi)) +* `--experimental-nvidia-gpus` flag is **replaced** by `Accelerators` alpha feature gate along with support for multiple Nvidia GPUs. ([#42116](https://github.com/kubernetes/kubernetes/pull/42116), [@vishh](https://github.com/vishh)) + * To use GPUs, pass `Accelerators=true` as part of `--feature-gates` flag. + * Works only with Docker runtime. +* Clean up the kube-proxy container image by removing unnecessary packages and files. ([#42090](https://github.com/kubernetes/kubernetes/pull/42090), [@timstclair](https://github.com/timstclair)) +* AWS: Support shared tag `kubernetes.io/cluster/` ([#41695](https://github.com/kubernetes/kubernetes/pull/41695), [@justinsb](https://github.com/justinsb)) +* Insecure access to the API Server at localhost:8080 will be turned off in v1.6 when using kubeadm ([#42066](https://github.com/kubernetes/kubernetes/pull/42066), [@luxas](https://github.com/luxas)) +* AWS: Do not consider master instance zones for dynamic volume creation ([#41702](https://github.com/kubernetes/kubernetes/pull/41702), [@justinsb](https://github.com/justinsb)) +* Added foreground garbage collection: the owner object will not be deleted until all its dependents are deleted by the garbage collector. Please checkout the [user doc](https://kubernetes.io/docs/concepts/abstractions/controllers/garbage-collection/) for details. ([#38676](https://github.com/kubernetes/kubernetes/pull/38676), [@caesarxuchao](https://github.com/caesarxuchao)) + * deleteOptions.orphanDependents is going to be deprecated in 1.7. Please use deleteOptions.propagationPolicy instead. +* force unlock rbd image if the image is not used ([#41597](https://github.com/kubernetes/kubernetes/pull/41597), [@rootfs](https://github.com/rootfs)) +* The kubernetes-master, kubernetes-worker and kubeapi-load-balancer charms have gained an nrpe-external-master relation, allowing the integration of their monitoring in an external Nagios server. ([#41923](https://github.com/kubernetes/kubernetes/pull/41923), [@Cynerva](https://github.com/Cynerva)) +* make kubectl describe pod show tolerationSeconds ([#42162](https://github.com/kubernetes/kubernetes/pull/42162), [@kevin-wangzefeng](https://github.com/kevin-wangzefeng)) +* Completed pods should not be hidden when requested by name via `kubectl get`. ([#42216](https://github.com/kubernetes/kubernetes/pull/42216), [@smarterclayton](https://github.com/smarterclayton)) +* [Federation][Kubefed] Flag cleanup ([#41335](https://github.com/kubernetes/kubernetes/pull/41335), [@irfanurrehman](https://github.com/irfanurrehman)) +* Add the support to the scheduler for spreading pods of StatefulSets. ([#41708](https://github.com/kubernetes/kubernetes/pull/41708), [@bsalamat](https://github.com/bsalamat)) +* Portworx Volume Plugin added enabling [Portworx](http://www.portworx.com) to be used as a storage provider for Kubernetes clusters. Portworx pools your servers capacity and turns your servers or cloud instances into converged, highly available compute and storage nodes. ([#39535](https://github.com/kubernetes/kubernetes/pull/39535), [@adityadani](https://github.com/adityadani)) +* Remove support for trusty in GCE kube-up. ([#41670](https://github.com/kubernetes/kubernetes/pull/41670), [@mikedanese](https://github.com/mikedanese)) +* Import a natural sorting library and use it in the sorting printer. ([#40746](https://github.com/kubernetes/kubernetes/pull/40746), [@matthyx](https://github.com/matthyx)) +* Parameter keys in a StorageClass `parameters` map may not use the `kubernetes.io` or `k8s.io` namespaces. ([#41837](https://github.com/kubernetes/kubernetes/pull/41837), [@liggitt](https://github.com/liggitt)) +* Make DaemonSet respect critical pods annotation when scheduling. ([#42028](https://github.com/kubernetes/kubernetes/pull/42028), [@janetkuo](https://github.com/janetkuo)) +* New Kubelet flag `--enforce-node-allocatable` with a default value of `pods` is added which will make kubelet create a top level cgroup for all pods to enforce Node Allocatable. Optionally, `system-reserved` & `kube-reserved` values can also be specified separated by comma to enforce node allocatable on cgroups specified via `--system-reserved-cgroup` & `--kube-reserved-cgroup` respectively. Note the default value of the latter flags are "". ([#41234](https://github.com/kubernetes/kubernetes/pull/41234), [@vishh](https://github.com/vishh)) + * This feature requires a **Node Drain** prior to upgrade failing which pods will be restarted if possible or terminated if they have a `RestartNever` policy. +* Deployment of AWS Kubernetes clusters using the in-tree bash deployment (i.e. cluster/kube-up.sh or get-kube.sh) is obsolete. v1.5.x will be the last release to support cluster/kube-up.sh with AWS. For a list of viable alternatives, see: http://kubernetes.io/docs/getting-started-guides/aws/ ([#42196](https://github.com/kubernetes/kubernetes/pull/42196), [@zmerlynn](https://github.com/zmerlynn)) +* kubectl logs allows getting logs directly from deployment, job and statefulset ([#40927](https://github.com/kubernetes/kubernetes/pull/40927), [@soltysh](https://github.com/soltysh)) +* make kubectl taint command respect effect NoExecute ([#42120](https://github.com/kubernetes/kubernetes/pull/42120), [@kevin-wangzefeng](https://github.com/kevin-wangzefeng)) +* Flex volume plugin is updated to support attach/detach interfaces. It broke backward compatibility. Please update your drivers and implement the new callouts. ([#41804](https://github.com/kubernetes/kubernetes/pull/41804), [@chakri-nelluri](https://github.com/chakri-nelluri)) +* Implement the update feature for DaemonSet. ([#41116](https://github.com/kubernetes/kubernetes/pull/41116), [@lukaszo](https://github.com/lukaszo)) +* [Federation] Create configmap for the cluster kube-dns when cluster joins and remove when it unjoins ([#39338](https://github.com/kubernetes/kubernetes/pull/39338), [@irfanurrehman](https://github.com/irfanurrehman)) +* New GKE certificates controller. ([#41160](https://github.com/kubernetes/kubernetes/pull/41160), [@pipejakob](https://github.com/pipejakob)) +* Juju: Fix shebangs in charm actions to use python3 ([#42058](https://github.com/kubernetes/kubernetes/pull/42058), [@Cynerva](https://github.com/Cynerva)) +* Support kubectl apply set-last-applied command to update the applied-applied-configuration annotation ([#41694](https://github.com/kubernetes/kubernetes/pull/41694), [@shiywang](https://github.com/shiywang)) +* On GCI by default logrotate is disabled for application containers in favor of rotation mechanism provided by docker logging driver. ([#40634](https://github.com/kubernetes/kubernetes/pull/40634), [@crassirostris](https://github.com/crassirostris)) +* Cleanup fluentd-gcp image: rebase on debian-base, switch to upstream packages, remove fluent-ui & rails ([#41998](https://github.com/kubernetes/kubernetes/pull/41998), [@timstclair](https://github.com/timstclair)) +* Updating apiserver to return http status code 202 for a delete request when the resource is not immediately deleted because of user requesting cascading deletion using DeleteOptions.OrphanDependents=false. ([#41165](https://github.com/kubernetes/kubernetes/pull/41165), [@nikhiljindal](https://github.com/nikhiljindal)) +* [Federation][kubefed] Support configuring dns-provider ([#40528](https://github.com/kubernetes/kubernetes/pull/40528), [@shashidharatd](https://github.com/shashidharatd)) +* Added support to minimize sending verbose node information to scheduler extender by sending only node names and expecting extenders to cache the rest of the node information ([#41119](https://github.com/kubernetes/kubernetes/pull/41119), [@sarat-k](https://github.com/sarat-k)) +* Guaranteed admission for Critical Pods ([#40952](https://github.com/kubernetes/kubernetes/pull/40952), [@dashpole](https://github.com/dashpole)) +* Switch to the `node-role.kubernetes.io/master` label for marking and tainting the master node in kubeadm ([#41835](https://github.com/kubernetes/kubernetes/pull/41835), [@luxas](https://github.com/luxas)) +* Allow drain --force to remove pods whose managing resource is deleted. ([#41864](https://github.com/kubernetes/kubernetes/pull/41864), [@marun](https://github.com/marun)) +* add kubectl can-i to see if you can perform an action ([#41077](https://github.com/kubernetes/kubernetes/pull/41077), [@deads2k](https://github.com/deads2k)) +* enable DefaultTolerationSeconds admission controller by default ([#41815](https://github.com/kubernetes/kubernetes/pull/41815), [@kevin-wangzefeng](https://github.com/kevin-wangzefeng)) +* Make DaemonSets survive taint-based evictions when nodes turn unreachable/notReady. ([#41896](https://github.com/kubernetes/kubernetes/pull/41896), [@kevin-wangzefeng](https://github.com/kevin-wangzefeng)) +* Add configurable limits to CronJob resource to specify how many successful and failed jobs are preserved. ([#40932](https://github.com/kubernetes/kubernetes/pull/40932), [@peay](https://github.com/peay)) +* Deprecate outofdisk-transition-frequency and low-diskspace-threshold-mb flags ([#41941](https://github.com/kubernetes/kubernetes/pull/41941), [@dashpole](https://github.com/dashpole)) +* Add OWNERS for sample-apiserver in staging ([#42094](https://github.com/kubernetes/kubernetes/pull/42094), [@sttts](https://github.com/sttts)) +* Update gcr.io/google-containers/rescheduler to v0.2.2, which uses busybox as a base image instead of ubuntu. ([#41911](https://github.com/kubernetes/kubernetes/pull/41911), [@ixdy](https://github.com/ixdy)) +* Add storage.k8s.io/v1 API ([#40088](https://github.com/kubernetes/kubernetes/pull/40088), [@jsafrane](https://github.com/jsafrane)) +* Juju - K8s master charm now properly keeps distributed master files in sync for an HA control plane. ([#41351](https://github.com/kubernetes/kubernetes/pull/41351), [@chuckbutler](https://github.com/chuckbutler)) +* Fix zsh completion: unknown file attribute error ([#38104](https://github.com/kubernetes/kubernetes/pull/38104), [@elipapa](https://github.com/elipapa)) +* kubelet config should ignore file start with dots ([#39196](https://github.com/kubernetes/kubernetes/pull/39196), [@resouer](https://github.com/resouer)) +* Add an alpha feature that makes NodeController set Taints instead of deleting Pods from not Ready Nodes. ([#41133](https://github.com/kubernetes/kubernetes/pull/41133), [@gmarek](https://github.com/gmarek)) +* Base etcd-empty-dir-cleanup on busybox, run as nobody, and update to etcdctl 3.0.14 ([#41674](https://github.com/kubernetes/kubernetes/pull/41674), [@ixdy](https://github.com/ixdy)) +* Fix zone placement heuristics so that multiple mounts in a StatefulSet pod are created in the same zone ([#40910](https://github.com/kubernetes/kubernetes/pull/40910), [@justinsb](https://github.com/justinsb)) +* Flag --use-kubernetes-version for kubeadm init renamed to --kubernetes-version ([#41820](https://github.com/kubernetes/kubernetes/pull/41820), [@kad](https://github.com/kad)) +* `kube-dns` now runs using a separate `system:serviceaccount:kube-system:kube-dns` service account which is automatically bound to the correct RBAC permissions. ([#38816](https://github.com/kubernetes/kubernetes/pull/38816), [@deads2k](https://github.com/deads2k)) +* [Kubemark] Fixed hollow-npd container command to log to file ([#41858](https://github.com/kubernetes/kubernetes/pull/41858), [@shyamjvs](https://github.com/shyamjvs)) +* kubeadm: Remove the --cloud-provider flag for beta init UX ([#41710](https://github.com/kubernetes/kubernetes/pull/41710), [@luxas](https://github.com/luxas)) +* The CertificateSigningRequest API added the `extra` field to persist all information about the requesting user. This mirrors the fields in the SubjectAccessReview API used to check authorization. ([#41755](https://github.com/kubernetes/kubernetes/pull/41755), [@liggitt](https://github.com/liggitt)) +* Upgrade golang versions to 1.7.5 ([#41771](https://github.com/kubernetes/kubernetes/pull/41771), [@cblecker](https://github.com/cblecker)) +* Added a new secret type "bootstrap.kubernetes.io/token" for dynamically creating TLS bootstrapping bearer tokens. ([#41281](https://github.com/kubernetes/kubernetes/pull/41281), [@ericchiang](https://github.com/ericchiang)) +* Remove unnecessary metrics (http/process/go) from being exposed by etcd-version-monitor ([#41807](https://github.com/kubernetes/kubernetes/pull/41807), [@shyamjvs](https://github.com/shyamjvs)) +* Added `kubectl create clusterrole` command. ([#41538](https://github.com/kubernetes/kubernetes/pull/41538), [@xingzhou](https://github.com/xingzhou)) +* Support new kubectl apply view-last-applied command for viewing the last configuration file applied ([#41146](https://github.com/kubernetes/kubernetes/pull/41146), [@shiywang](https://github.com/shiywang)) +* Bump GCI to gci-stable-56-9000-84-2 ([#41819](https://github.com/kubernetes/kubernetes/pull/41819), [@dchen1107](https://github.com/dchen1107)) +* list-resources: don't fail if the grep fails to match any resources ([#41933](https://github.com/kubernetes/kubernetes/pull/41933), [@ixdy](https://github.com/ixdy)) +* client-go no longer imports GCP OAuth2 and OpenID Connect packages by default. ([#41532](https://github.com/kubernetes/kubernetes/pull/41532), [@ericchiang](https://github.com/ericchiang)) +* Each pod has its own associated cgroup by default. ([#41349](https://github.com/kubernetes/kubernetes/pull/41349), [@derekwaynecarr](https://github.com/derekwaynecarr)) +* Whitelist kubemark in node_ssh_supported_providers for log dump ([#41800](https://github.com/kubernetes/kubernetes/pull/41800), [@shyamjvs](https://github.com/shyamjvs)) +* Support KUBE_MAX_PD_VOLS on Azure ([#41398](https://github.com/kubernetes/kubernetes/pull/41398), [@codablock](https://github.com/codablock)) +* Projected volume plugin ([#37237](https://github.com/kubernetes/kubernetes/pull/37237), [@jpeeler](https://github.com/jpeeler)) +* `--output-version` is ignored for all commands except `kubectl convert`. This is consistent with the generic nature of `kubectl` CRUD commands and the previous removal of `--api-version`. Specific versions can be specified in the resource field: `resource.version.group`, `jobs.v1.batch`. ([#41576](https://github.com/kubernetes/kubernetes/pull/41576), [@deads2k](https://github.com/deads2k)) +* Added bool type support for jsonpath. ([#39063](https://github.com/kubernetes/kubernetes/pull/39063), [@xingzhou](https://github.com/xingzhou)) +* Nodes can now report two additional address types in their status: InternalDNS and ExternalDNS. The apiserver can use `--kubelet-preferred-address-types` to give priority to the type of address it uses to reach nodes. ([#34259](https://github.com/kubernetes/kubernetes/pull/34259), [@liggitt](https://github.com/liggitt)) +* Clients now use the `?watch=true` parameter to make watch API calls, instead of the `/watch/` path prefix ([#41722](https://github.com/kubernetes/kubernetes/pull/41722), [@liggitt](https://github.com/liggitt)) +* ResourceQuota ability to support default limited resources ([#36765](https://github.com/kubernetes/kubernetes/pull/36765), [@derekwaynecarr](https://github.com/derekwaynecarr)) +* Fix kubemark default e2e test suite's name ([#41751](https://github.com/kubernetes/kubernetes/pull/41751), [@shyamjvs](https://github.com/shyamjvs)) +* federation aws: add logging of route53 calls ([#39964](https://github.com/kubernetes/kubernetes/pull/39964), [@justinsb](https://github.com/justinsb)) +* Fix ConfigMap for Windows Containers. ([#39373](https://github.com/kubernetes/kubernetes/pull/39373), [@jbhurat](https://github.com/jbhurat)) +* add defaultTolerationSeconds admission controller ([#41414](https://github.com/kubernetes/kubernetes/pull/41414), [@kevin-wangzefeng](https://github.com/kevin-wangzefeng)) +* Node Problem Detector is beta now. New features added: journald support, standalone mode and arbitrary system log monitoring. ([#40206](https://github.com/kubernetes/kubernetes/pull/40206), [@Random-Liu](https://github.com/Random-Liu)) +* services of type loadbalancer consume both loadbalancer and nodeport quota. ([#39364](https://github.com/kubernetes/kubernetes/pull/39364), [@zhouhaibing089](https://github.com/zhouhaibing089)) +* Fix the output of health-mointor.sh ([#41525](https://github.com/kubernetes/kubernetes/pull/41525), [@yujuhong](https://github.com/yujuhong)) +* kubectl describe no longer prints the last-applied-configuration annotation for secrets. ([#34664](https://github.com/kubernetes/kubernetes/pull/34664), [@ymqytw](https://github.com/ymqytw)) +* Report node not ready on failed PLEG health check ([#41569](https://github.com/kubernetes/kubernetes/pull/41569), [@yujuhong](https://github.com/yujuhong)) +* Delay Deletion of a Pod until volumes are cleaned up ([#41456](https://github.com/kubernetes/kubernetes/pull/41456), [@dashpole](https://github.com/dashpole)) +* Alpha version of dynamic volume provisioning is removed in this release. Annotation ([#40000](https://github.com/kubernetes/kubernetes/pull/40000), [@jsafrane](https://github.com/jsafrane)) + * "volume.alpha.kubernetes.io/storage-class" does not have any special meaning. A default storage class + * and DefaultStorageClass admission plugin can be used to preserve similar behavior of Kubernetes cluster, + * see https://kubernetes.io/docs/user-guide/persistent-volumes/#class-1 for details. +* An `automountServiceAccountToken *bool` field was added to ServiceAccount and PodSpec objects. If set to `false` on a pod spec, no service account token is automounted in the pod. If set to `false` on a service account, no service account token is automounted for that service account unless explicitly overridden in the pod spec. ([#37953](https://github.com/kubernetes/kubernetes/pull/37953), [@liggitt](https://github.com/liggitt)) +* Bump addon-manager version to v6.4-alpha.1 in kubemark ([#41506](https://github.com/kubernetes/kubernetes/pull/41506), [@shyamjvs](https://github.com/shyamjvs)) +* Do not daemonize `salt-minion` for the openstack-heat provider. ([#40722](https://github.com/kubernetes/kubernetes/pull/40722), [@micmro](https://github.com/micmro)) +* Move private key parsing from serviceaccount/jwt.go to client-go/util/cert ([#40907](https://github.com/kubernetes/kubernetes/pull/40907), [@cblecker](https://github.com/cblecker)) +* Added configurable etcd initial-cluster-state to kube-up script. ([#41332](https://github.com/kubernetes/kubernetes/pull/41332), [@jszczepkowski](https://github.com/jszczepkowski)) +* Fix AWS device allocator to only use valid device names ([#41455](https://github.com/kubernetes/kubernetes/pull/41455), [@gnufied](https://github.com/gnufied)) +* [Federation][Kubefed] Bug fix relating kubeconfig path in kubefed init ([#41410](https://github.com/kubernetes/kubernetes/pull/41410), [@irfanurrehman](https://github.com/irfanurrehman)) +* On GCE, the apiserver audit log (`/var/log/kube-apiserver-audit.log`) will be sent through fluentd if enabled. It will go to the same place as `kube-apiserver.log`, but tagged as its own stream. ([#41360](https://github.com/kubernetes/kubernetes/pull/41360), [@enisoc](https://github.com/enisoc)) +* Bump GCE ContainerVM to container-vm-v20170214 to address CVE-2016-9962. ([#41449](https://github.com/kubernetes/kubernetes/pull/41449), [@zmerlynn](https://github.com/zmerlynn)) +* Fixed issues [#39202](https://github.com/kubernetes/kubernetes/pull/39202), [#41041](https://github.com/kubernetes/kubernetes/pull/41041) and [#40941](https://github.com/kubernetes/kubernetes/pull/40941) that caused the iSCSI connections to be prematurely closed when deleting a pod with an iSCSI persistent volume attached and that prevented the use of newly created LUNs on targets with preestablished connections. ([#41196](https://github.com/kubernetes/kubernetes/pull/41196), [@CristianPop](https://github.com/CristianPop)) +* The kube-apiserver [basic audit log](https://kubernetes.io/docs/admin/audit/) can be enabled in GCE by exporting the environment variable `ENABLE_APISERVER_BASIC_AUDIT=true` before running `cluster/kube-up.sh`. This will log to `/var/log/kube-apiserver-audit.log` and use the same `logrotate` settings as `/var/log/kube-apiserver.log`. ([#41211](https://github.com/kubernetes/kubernetes/pull/41211), [@enisoc](https://github.com/enisoc)) +* On kube-up.sh clusters on GCE, kube-scheduler now contacts the API on the secured port. ([#41285](https://github.com/kubernetes/kubernetes/pull/41285), [@liggitt](https://github.com/liggitt)) +* Default RBAC ClusterRole and ClusterRoleBinding objects are automatically updated at server start to add missing permissions and subjects (extra permissions and subjects are left in place). To prevent autoupdating a particular role or rolebinding, annotate it with `rbac.authorization.kubernetes.io/autoupdate=false`. ([#41155](https://github.com/kubernetes/kubernetes/pull/41155), [@liggitt](https://github.com/liggitt)) +* Make EnableCRI default to true ([#41378](https://github.com/kubernetes/kubernetes/pull/41378), [@yujuhong](https://github.com/yujuhong)) +* `kubectl edit` now edits objects exactly as they were retrieved from the API. This allows using `kubectl edit` with third-party resources and extension API servers. Because client-side conversion is no longer done, the `--output-version` option is deprecated for `kubectl edit`. To edit using a particular API version, fully-qualify the resource, version, and group used to fetch the object (for example, `job.v1.batch/myjob`) ([#41304](https://github.com/kubernetes/kubernetes/pull/41304), [@liggitt](https://github.com/liggitt)) +* We change the default attach_detach_controller sync period to 1 minute to reduce the query frequency through cloud provider to check whether volumes are attached or not. ([#41363](https://github.com/kubernetes/kubernetes/pull/41363), [@jingxu97](https://github.com/jingxu97)) +* RBAC `v1beta1` RoleBinding/ClusterRoleBinding subjects changed `apiVersion` to `apiGroup` to fully-qualify a subject. ServiceAccount subjects default to an apiGroup of `""`, User and Group subjects default to an apiGroup of `"rbac.authorization.k8s.io"`. ([#41184](https://github.com/kubernetes/kubernetes/pull/41184), [@liggitt](https://github.com/liggitt)) +* Add support for finalizers in federated configmaps (deletes configmaps from underlying clusters). ([#40464](https://github.com/kubernetes/kubernetes/pull/40464), [@csbell](https://github.com/csbell)) +* Make DaemonSet controller respect node taints and pod tolerations. ([#41172](https://github.com/kubernetes/kubernetes/pull/41172), [@janetkuo](https://github.com/janetkuo)) +* Added kubectl create role command ([#39852](https://github.com/kubernetes/kubernetes/pull/39852), [@xingzhou](https://github.com/xingzhou)) +* If `experimentalCriticalPodAnnotation` feature gate is set to true, fluentd pods will not be evicted by the kubelet. ([#41035](https://github.com/kubernetes/kubernetes/pull/41035), [@vishh](https://github.com/vishh)) +* Align the hyperkube image to support running binaries at /usr/local/bin/ like the other server images ([#41017](https://github.com/kubernetes/kubernetes/pull/41017), [@luxas](https://github.com/luxas)) +* Native support for token based bootstrap flow. This includes signing a well known ConfigMap in the `kube-public` namespace and cleaning out expired tokens. ([#36101](https://github.com/kubernetes/kubernetes/pull/36101), [@jbeda](https://github.com/jbeda)) +* Reverts to looking up the current VM in vSphere using the machine's UUID, either obtained via sysfs or via the `vm-uuid` parameter in the cloud configuration file. ([#40892](https://github.com/kubernetes/kubernetes/pull/40892), [@robdaemon](https://github.com/robdaemon)) +* This PR adds a manager to NodeController that is responsible for removing Pods from Nodes tainted with NoExecute Taints. This feature is beta (as the rest of taints) and enabled by default. It's gated by controller-manager enable-taint-manager flag. ([#40355](https://github.com/kubernetes/kubernetes/pull/40355), [@gmarek](https://github.com/gmarek)) +* The authentication.k8s.io API group was promoted to v1 ([#41058](https://github.com/kubernetes/kubernetes/pull/41058), [@liggitt](https://github.com/liggitt)) +* Fixes issue [#38418](https://github.com/kubernetes/kubernetes/pull/38418) which, under circumstance, could cause StatefulSet to deadlock. ([#40838](https://github.com/kubernetes/kubernetes/pull/40838), [@kow3ns](https://github.com/kow3ns)) + * Mediates issue [#36859](https://github.com/kubernetes/kubernetes/pull/36859). StatefulSet only acts on Pods whose identity matches the StatefulSet, providing a partial mediation for overlapping controllers. +* Introduces an new alpha version of the Horizontal Pod Autoscaler including expanded support for specifying metrics. ([#36033](https://github.com/kubernetes/kubernetes/pull/36033), [@DirectXMan12](https://github.com/DirectXMan12)) +* Set all node conditions to Unknown when node is unreachable ([#36592](https://github.com/kubernetes/kubernetes/pull/36592), [@andrewsykim](https://github.com/andrewsykim)) +* [Federation] Add override flags options to kubefed init ([#40917](https://github.com/kubernetes/kubernetes/pull/40917), [@irfanurrehman](https://github.com/irfanurrehman)) +* Fix for detach volume when node is not present/ powered off ([#40118](https://github.com/kubernetes/kubernetes/pull/40118), [@BaluDontu](https://github.com/BaluDontu)) +* Bump up GLBC version from 0.9.0-beta to 0.9.1 ([#41037](https://github.com/kubernetes/kubernetes/pull/41037), [@bprashanth](https://github.com/bprashanth)) +* The deprecated flags --config, --auth-path, --resource-container, and --system-container were removed. ([#40048](https://github.com/kubernetes/kubernetes/pull/40048), [@mtaufen](https://github.com/mtaufen)) +* Add `kubectl attach` support for multiple types ([#40365](https://github.com/kubernetes/kubernetes/pull/40365), [@shiywang](https://github.com/shiywang)) +* [Kubelet] Delay deletion of pod from the API server until volumes are deleted ([#41095](https://github.com/kubernetes/kubernetes/pull/41095), [@dashpole](https://github.com/dashpole)) +* remove the create-external-load-balancer flag in cmd/expose.go ([#38183](https://github.com/kubernetes/kubernetes/pull/38183), [@tianshapjq](https://github.com/tianshapjq)) +* The authorization.k8s.io API group was promoted to v1 ([#40709](https://github.com/kubernetes/kubernetes/pull/40709), [@liggitt](https://github.com/liggitt)) +* Bump GCI to gci-beta-56-9000-80-0 ([#41027](https://github.com/kubernetes/kubernetes/pull/41027), [@dchen1107](https://github.com/dchen1107)) +* [Federation][kubefed] Add option to expose federation apiserver on nodeport service ([#40516](https://github.com/kubernetes/kubernetes/pull/40516), [@shashidharatd](https://github.com/shashidharatd)) +* Rename --experiemental-cgroups-per-qos to --cgroups-per-qos ([#39972](https://github.com/kubernetes/kubernetes/pull/39972), [@derekwaynecarr](https://github.com/derekwaynecarr)) +* PV E2E: provide each spec with a fresh nfs host ([#40879](https://github.com/kubernetes/kubernetes/pull/40879), [@copejon](https://github.com/copejon)) +* Remove the temporary fix for pre-1.0 mirror pods ([#40877](https://github.com/kubernetes/kubernetes/pull/40877), [@yujuhong](https://github.com/yujuhong)) +* fix --save-config in create subcommand ([#40289](https://github.com/kubernetes/kubernetes/pull/40289), [@xilabao](https://github.com/xilabao)) +* We should mention the caveats of in-place upgrade in release note. ([#40727](https://github.com/kubernetes/kubernetes/pull/40727), [@Random-Liu](https://github.com/Random-Liu)) +* The SubjectAccessReview API passes subresource and resource name information to the authorizer to answer authorization queries. ([#40935](https://github.com/kubernetes/kubernetes/pull/40935), [@liggitt](https://github.com/liggitt)) +* When feature gate "ExperimentalCriticalPodAnnotation" is set, Kubelet will avoid evicting pods in "kube-system" namespace that contains a special annotation - `scheduler.alpha.kubernetes.io/critical-pod` ([#40655](https://github.com/kubernetes/kubernetes/pull/40655), [@vishh](https://github.com/vishh)) + * This feature should be used in conjunction with the rescheduler to guarantee availability for critical system pods - https://kubernetes.io/docs/admin/rescheduler/ +* make tolerations respect wildcard key ([#39914](https://github.com/kubernetes/kubernetes/pull/39914), [@kevin-wangzefeng](https://github.com/kevin-wangzefeng)) +* [Federation][kubefed] Add option to disable persistence storage for etcd ([#40862](https://github.com/kubernetes/kubernetes/pull/40862), [@shashidharatd](https://github.com/shashidharatd)) +* Init containers have graduated to GA and now appear as a field. The beta annotation value will still be respected and overrides the field value. ([#38382](https://github.com/kubernetes/kubernetes/pull/38382), [@hodovska](https://github.com/hodovska)) +* apply falls back to generic 3-way JSON merge patch if no go struct is registered for the target GVK ([#40666](https://github.com/kubernetes/kubernetes/pull/40666), [@ymqytw](https://github.com/ymqytw)) +* HorizontalPodAutoscaler is no longer supported in extensions/v1beta1 version. Use autoscaling/v1 instead. ([#35782](https://github.com/kubernetes/kubernetes/pull/35782), [@piosz](https://github.com/piosz)) +* Port forwarding can forward over websockets or SPDY. ([#33684](https://github.com/kubernetes/kubernetes/pull/33684), [@fraenkel](https://github.com/fraenkel)) +* Improve kubectl describe node output by adding closing paren ([#39217](https://github.com/kubernetes/kubernetes/pull/39217), [@luksa](https://github.com/luksa)) +* Bump GCE ContainerVM to container-vm-v20170201 to address CVE-2016-9962. ([#40828](https://github.com/kubernetes/kubernetes/pull/40828), [@zmerlynn](https://github.com/zmerlynn)) +* Use full package path for definition name in OpenAPI spec ([#40124](https://github.com/kubernetes/kubernetes/pull/40124), [@mbohlool](https://github.com/mbohlool)) +* kubectl apply now supports explicitly clearing values not present in the config by setting them to null ([#40630](https://github.com/kubernetes/kubernetes/pull/40630), [@liggitt](https://github.com/liggitt)) +* Fixed an issue where 'kubectl get --sort-by=' would return an error when the specified field were not present in at least one of the returned objects, even that being a valid field in the object model. ([#40541](https://github.com/kubernetes/kubernetes/pull/40541), [@fabianofranz](https://github.com/fabianofranz)) +* Add initial french translations for kubectl ([#40645](https://github.com/kubernetes/kubernetes/pull/40645), [@brendandburns](https://github.com/brendandburns)) +* OpenStack-Heat will now look for an image named "CentOS-7-x86_64-GenericCloud-1604". To restore the previous behavior set OPENSTACK_IMAGE_NAME="CentOS7" ([#40368](https://github.com/kubernetes/kubernetes/pull/40368), [@sc68cal](https://github.com/sc68cal)) +* Preventing nil pointer reference in client_config ([#40508](https://github.com/kubernetes/kubernetes/pull/40508), [@vjsamuel](https://github.com/vjsamuel)) +* The bash AWS deployment via kube-up.sh has been deprecated. See http://kubernetes.io/docs/getting-started-guides/aws/ for alternatives. ([#38772](https://github.com/kubernetes/kubernetes/pull/38772), [@zmerlynn](https://github.com/zmerlynn)) +* Fix failing load balancers in Azure ([#40405](https://github.com/kubernetes/kubernetes/pull/40405), [@codablock](https://github.com/codablock)) +* kubefed init creates a service account for federation controller manager in the federation-system namespace and binds that service account to the federation-system:federation-controller-manager role that has read and list access on secrets in the federation-system namespace. ([#40392](https://github.com/kubernetes/kubernetes/pull/40392), [@madhusudancs](https://github.com/madhusudancs)) +* Fixed an SELinux issue in kubeadm on Docker 1.12+ by moving etcd SELinux options from container to pod. ([#40682](https://github.com/kubernetes/kubernetes/pull/40682), [@dgoodwin](https://github.com/dgoodwin)) +* Juju kubernetes-master charm: improve status messages ([#40691](https://github.com/kubernetes/kubernetes/pull/40691), [@Cynerva](https://github.com/Cynerva)) +* genericapiserver: cut off more dependencies – episode 3 ([#40426](https://github.com/kubernetes/kubernetes/pull/40426), [@sttts](https://github.com/sttts)) +* Adding vmdk file extension for vmDiskPath in vsphere DeleteVolume ([#40538](https://github.com/kubernetes/kubernetes/pull/40538), [@divyenpatel](https://github.com/divyenpatel)) +* Remove outdated net.experimental.kubernetes.io/proxy-mode and net.beta.kubernetes.io/proxy-mode annotations from kube-proxy. ([#40585](https://github.com/kubernetes/kubernetes/pull/40585), [@cblecker](https://github.com/cblecker)) +* Improve the ARM builds and make hyperkube on ARM working again by upgrading the Go version for ARM to go1.8beta2 ([#38926](https://github.com/kubernetes/kubernetes/pull/38926), [@luxas](https://github.com/luxas)) +* Prevent hotloops on error conditions, which could fill up the disk faster than log rotation can free space. ([#40497](https://github.com/kubernetes/kubernetes/pull/40497), [@lavalamp](https://github.com/lavalamp)) +* DaemonSet controller actively kills failed pods (to recreate them) ([#40330](https://github.com/kubernetes/kubernetes/pull/40330), [@janetkuo](https://github.com/janetkuo)) +* forgiveness alpha version api definition ([#39469](https://github.com/kubernetes/kubernetes/pull/39469), [@kevin-wangzefeng](https://github.com/kevin-wangzefeng)) +* Bump up glbc version to 0.9.0-beta.1 ([#40565](https://github.com/kubernetes/kubernetes/pull/40565), [@bprashanth](https://github.com/bprashanth)) +* Improve formatting of EventSource in kubectl get and kubectl describe ([#40073](https://github.com/kubernetes/kubernetes/pull/40073), [@matthyx](https://github.com/matthyx)) +* CRI: use more gogoprotobuf plugins ([#40397](https://github.com/kubernetes/kubernetes/pull/40397), [@yujuhong](https://github.com/yujuhong)) +* Adds `shortNames` to the `APIResource` from discovery which is a list of recommended shortNames for clients like `kubectl`. ([#40312](https://github.com/kubernetes/kubernetes/pull/40312), [@p0lyn0mial](https://github.com/p0lyn0mial)) +* Use existing ABAC policy file when upgrading GCE cluster ([#40172](https://github.com/kubernetes/kubernetes/pull/40172), [@liggitt](https://github.com/liggitt)) +* Added support for creating HA clusters for centos using kube-up.sh. ([#39462](https://github.com/kubernetes/kubernetes/pull/39462), [@Shawyeok](https://github.com/Shawyeok)) +* azure: fix Azure Container Registry integration ([#40142](https://github.com/kubernetes/kubernetes/pull/40142), [@colemickens](https://github.com/colemickens)) +* - Splits Juju Charm layers into master/worker roles ([#40324](https://github.com/kubernetes/kubernetes/pull/40324), [@chuckbutler](https://github.com/chuckbutler)) + * - Adds support for 1.5.x series of Kubernetes + * - Introduces a tactic for keeping templates in sync with upstream eliminating template drift + * - Adds CNI support to the Juju Charms + * - Adds durable storage support to the Juju Charms + * - Introduces an e2e Charm layer for repeatable testing efforts and validation of clusters +* Move b.gcr.io/k8s_authenticated_test to gcr.io/k8s-authenticated-test ([#40335](https://github.com/kubernetes/kubernetes/pull/40335), [@zmerlynn](https://github.com/zmerlynn)) +* genericapiserver: more dependency cutoffs ([#40216](https://github.com/kubernetes/kubernetes/pull/40216), [@sttts](https://github.com/sttts)) +* AWS: trust region if found from AWS metadata ([#38880](https://github.com/kubernetes/kubernetes/pull/38880), [@justinsb](https://github.com/justinsb)) +* Volumes and environment variables populated from ConfigMap and Secret objects can now tolerate the named source object or specific keys being missing, by adding `optional: true` to the volume or environment variable source specifications. ([#39981](https://github.com/kubernetes/kubernetes/pull/39981), [@fraenkel](https://github.com/fraenkel)) +* kubectl create now accepts the label selector flag for filtering objects to create ([#40057](https://github.com/kubernetes/kubernetes/pull/40057), [@MrHohn](https://github.com/MrHohn)) +* A new field `terminationMessagePolicy` has been added to containers that allows a user to request `FallbackToLogsOnError`, which will read from the container's logs to populate the termination message if the user does not write to the termination message log file. The termination message file is now properly readable for end users and has a maximum size (4k bytes) to prevent abuse. Each pod may have up to 12k bytes of termination messages before the contents of each will be truncated. ([#39341](https://github.com/kubernetes/kubernetes/pull/39341), [@smarterclayton](https://github.com/smarterclayton)) +* Add a special purpose tool for editing individual fields in a ConfigMap with kubectl ([#38445](https://github.com/kubernetes/kubernetes/pull/38445), [@brendandburns](https://github.com/brendandburns)) +* [Federation] Expose autoscaling apis through federation api server ([#38976](https://github.com/kubernetes/kubernetes/pull/38976), [@irfanurrehman](https://github.com/irfanurrehman)) +* Powershell script to start kubelet and kube-proxy ([#36250](https://github.com/kubernetes/kubernetes/pull/36250), [@jbhurat](https://github.com/jbhurat)) +* Reduce time needed to attach Azure disks ([#40066](https://github.com/kubernetes/kubernetes/pull/40066), [@codablock](https://github.com/codablock)) +* fixing Cassandra shutdown example to avoid data corruption ([#39199](https://github.com/kubernetes/kubernetes/pull/39199), [@deimosfr](https://github.com/deimosfr)) +* kubeadm: add optional self-hosted deployment for apiserver, controller-manager and scheduler. ([#40075](https://github.com/kubernetes/kubernetes/pull/40075), [@pires](https://github.com/pires)) +* kubelet tears down pod volumes on pod termination rather than pod deletion ([#37228](https://github.com/kubernetes/kubernetes/pull/37228), [@sjenning](https://github.com/sjenning)) +* The default client certificate generated by kube-up now contains the superuser `system:masters` group ([#39966](https://github.com/kubernetes/kubernetes/pull/39966), [@liggitt](https://github.com/liggitt)) +* CRI: upgrade protobuf to v3 ([#39158](https://github.com/kubernetes/kubernetes/pull/39158), [@feiskyer](https://github.com/feiskyer)) +* Add SIGCHLD handler to pause container ([#36853](https://github.com/kubernetes/kubernetes/pull/36853), [@verb](https://github.com/verb)) +* Populate environment variables from a secrets. ([#39446](https://github.com/kubernetes/kubernetes/pull/39446), [@fraenkel](https://github.com/fraenkel)) +* fluentd config for GKE clusters updated: detect exceptions in container log streams and forward them as one log entry. ([#39656](https://github.com/kubernetes/kubernetes/pull/39656), [@thomasschickinger](https://github.com/thomasschickinger)) +* Made multi-scheduler graduated to Beta and then v1. ([#38871](https://github.com/kubernetes/kubernetes/pull/38871), [@k82cn](https://github.com/k82cn)) +* Add authorization mode to kubeadm ([#39846](https://github.com/kubernetes/kubernetes/pull/39846), [@andrewrynhard](https://github.com/andrewrynhard)) +* Update dependencies: aws-sdk-go to 1.6.10; also cadvisor ([#40095](https://github.com/kubernetes/kubernetes/pull/40095), [@dashpole](https://github.com/dashpole)) +* Fixes a bug in the OpenStack-Heat kubernetes provider, in the handling of differences between the Identity v2 and Identity v3 APIs ([#40105](https://github.com/kubernetes/kubernetes/pull/40105), [@sc68cal](https://github.com/sc68cal)) +* Update GCE ContainerVM deployment to container-vm-v20170117 to pick up CVE fixes in base image. ([#40094](https://github.com/kubernetes/kubernetes/pull/40094), [@zmerlynn](https://github.com/zmerlynn)) +* AWS: Remove duplicate calls to DescribeInstance during volume operations ([#39842](https://github.com/kubernetes/kubernetes/pull/39842), [@gnufied](https://github.com/gnufied)) +* The `attributeRestrictions` field has been removed from the PolicyRule type in the rbac.authorization.k8s.io/v1alpha1 API. The field was not used by the RBAC authorizer. ([#39625](https://github.com/kubernetes/kubernetes/pull/39625), [@deads2k](https://github.com/deads2k)) +* Enable lazy inode table and journal initialization for ext3 and ext4 ([#38865](https://github.com/kubernetes/kubernetes/pull/38865), [@codablock](https://github.com/codablock)) +* azure disk: restrict name length for Azure specifications ([#40030](https://github.com/kubernetes/kubernetes/pull/40030), [@colemickens](https://github.com/colemickens)) +* Follow redirects for streaming requests (exec/attach/port-forward) in the apiserver by default (alpha -> beta). ([#40039](https://github.com/kubernetes/kubernetes/pull/40039), [@timstclair](https://github.com/timstclair)) +* Use kube-dns:1.11.0 ([#39925](https://github.com/kubernetes/kubernetes/pull/39925), [@sadlil](https://github.com/sadlil)) +* Anonymous authentication is now automatically disabled if the API server is started with the AlwaysAllow authorizer. ([#38706](https://github.com/kubernetes/kubernetes/pull/38706), [@deads2k](https://github.com/deads2k)) +* genericapiserver: cut off kube pkg/version dependency ([#39943](https://github.com/kubernetes/kubernetes/pull/39943), [@sttts](https://github.com/sttts)) +* genericapiserver: cut off pkg/serviceaccount dependency ([#39945](https://github.com/kubernetes/kubernetes/pull/39945), [@sttts](https://github.com/sttts)) +* Move pkg/api/rest into genericapiserver ([#39948](https://github.com/kubernetes/kubernetes/pull/39948), [@sttts](https://github.com/sttts)) +* genericapiserver: cut off pkg/apis/extensions and pkg/storage dependencies ([#39946](https://github.com/kubernetes/kubernetes/pull/39946), [@sttts](https://github.com/sttts)) +* genericapiserver: cut off certificates api dependency ([#39947](https://github.com/kubernetes/kubernetes/pull/39947), [@sttts](https://github.com/sttts)) +* Admission control support for versioned configuration files ([#39109](https://github.com/kubernetes/kubernetes/pull/39109), [@derekwaynecarr](https://github.com/derekwaynecarr)) +* Fix issue around merging lists of primitives when using PATCH or kubectl apply. ([#38665](https://github.com/kubernetes/kubernetes/pull/38665), [@ymqytw](https://github.com/ymqytw)) +* Fixes API compatibility issue with empty lists incorrectly returning a null `items` field instead of an empty array. ([#39834](https://github.com/kubernetes/kubernetes/pull/39834), [@liggitt](https://github.com/liggitt)) +* kubelet: remove the pleg health check from healthz ([#37865](https://github.com/kubernetes/kubernetes/pull/37865), [@yujuhong](https://github.com/yujuhong)) +* [scheduling] Moved pod affinity and anti-affinity from annotations to api fields [#25319](https://github.com/kubernetes/kubernetes/pull/25319) ([#39478](https://github.com/kubernetes/kubernetes/pull/39478), [@rrati](https://github.com/rrati)) +* PodSecurityPolicy resource is now enabled by default in the extensions API group. ([#39743](https://github.com/kubernetes/kubernetes/pull/39743), [@pweil-](https://github.com/pweil-)) +* add --controllers to controller manager ([#39740](https://github.com/kubernetes/kubernetes/pull/39740), [@deads2k](https://github.com/deads2k)) +* proxy/iptables: don't sync proxy rules if services map didn't change ([#38996](https://github.com/kubernetes/kubernetes/pull/38996), [@dcbw](https://github.com/dcbw)) +* NONE ([#39768](https://github.com/kubernetes/kubernetes/pull/39768), [@rkouj](https://github.com/rkouj)) +* Update amd64 kube-proxy base image to debian-iptables-amd64:v5 ([#39725](https://github.com/kubernetes/kubernetes/pull/39725), [@ixdy](https://github.com/ixdy)) +* Update dashboard version to v1.5.1 ([#39662](https://github.com/kubernetes/kubernetes/pull/39662), [@rf232](https://github.com/rf232)) +* Fix kubectl get -f -o so it prints all items in the file ([#39038](https://github.com/kubernetes/kubernetes/pull/39038), [@ncdc](https://github.com/ncdc)) +* Scheduler treats StatefulSet pods as belonging to a single equivalence class. ([#39718](https://github.com/kubernetes/kubernetes/pull/39718), [@foxish](https://github.com/foxish)) +* --basic-auth-file supports optionally specifying groups in the fourth column of the file ([#39651](https://github.com/kubernetes/kubernetes/pull/39651), [@liggitt](https://github.com/liggitt)) +* To create or update an RBAC RoleBinding or ClusterRoleBinding object, a user must: ([#39383](https://github.com/kubernetes/kubernetes/pull/39383), [@liggitt](https://github.com/liggitt)) + * 1. Be authorized to make the create or update API request + * 2. Be allowed to bind the referenced role, either by already having all of the permissions contained in the referenced role, or by having the "bind" permission on the referenced role. +* Fixes an HPA-related panic due to division-by-zero. ([#39694](https://github.com/kubernetes/kubernetes/pull/39694), [@DirectXMan12](https://github.com/DirectXMan12)) +* federation: Adding support for DeleteOptions.OrphanDependents for federated services. Setting it to false while deleting a federated service also deletes the corresponding services from all registered clusters. ([#36390](https://github.com/kubernetes/kubernetes/pull/36390), [@nikhiljindal](https://github.com/nikhiljindal)) +* Update kube-proxy image to be based off of Debian 8.6 base image. ([#39695](https://github.com/kubernetes/kubernetes/pull/39695), [@ixdy](https://github.com/ixdy)) +* Update FitError as a message component into the PodConditionUpdater. ([#39491](https://github.com/kubernetes/kubernetes/pull/39491), [@jayunit100](https://github.com/jayunit100)) +* rename kubernetes-discovery to kube-aggregator ([#39619](https://github.com/kubernetes/kubernetes/pull/39619), [@deads2k](https://github.com/deads2k)) +* Allow missing keys in templates by default ([#39486](https://github.com/kubernetes/kubernetes/pull/39486), [@ncdc](https://github.com/ncdc)) +* Caching added to the OIDC client auth plugin to fix races and reduce the time kubectl commands using this plugin take by several seconds. ([#38167](https://github.com/kubernetes/kubernetes/pull/38167), [@ericchiang](https://github.com/ericchiang)) +* AWS: recognize eu-west-2 region ([#38746](https://github.com/kubernetes/kubernetes/pull/38746), [@justinsb](https://github.com/justinsb)) +* Provide kubernetes-controller-manager flags to control volume attach/detach reconciler sync. The duration of the syncs can be controlled, and the syncs can be shut off as well. ([#39551](https://github.com/kubernetes/kubernetes/pull/39551), [@chrislovecnm](https://github.com/chrislovecnm)) +* Generate OpenAPI definition for inlined types ([#39466](https://github.com/kubernetes/kubernetes/pull/39466), [@mbohlool](https://github.com/mbohlool)) +* ShortcutExpander has been extended in a way that it will examine a ha… ([#38835](https://github.com/kubernetes/kubernetes/pull/38835), [@p0lyn0mial](https://github.com/p0lyn0mial)) +* fixes nil dereference when doing a volume type check on persistent volumes ([#39493](https://github.com/kubernetes/kubernetes/pull/39493), [@sjenning](https://github.com/sjenning)) +* Fix issue with PodDisruptionBudgets in which `minAvailable` specified as a percentage did not work with StatefulSet Pods. ([#39454](https://github.com/kubernetes/kubernetes/pull/39454), [@foxish](https://github.com/foxish)) +* fix issue with kubectl proxy so that it will proxy an empty path - e.g. http://localhost:8001 ([#39226](https://github.com/kubernetes/kubernetes/pull/39226), [@luksa](https://github.com/luksa)) +* Check if pathExists before performing Unmount ([#39311](https://github.com/kubernetes/kubernetes/pull/39311), [@rkouj](https://github.com/rkouj)) +* Adding kubectl tests for federation ([#38844](https://github.com/kubernetes/kubernetes/pull/38844), [@nikhiljindal](https://github.com/nikhiljindal)) +* Fix comment and optimize code ([#38084](https://github.com/kubernetes/kubernetes/pull/38084), [@tanshanshan](https://github.com/tanshanshan)) +* When using OIDC authentication and specifying --oidc-username-claim=email, an `"email_verified":true` claim must be returned from the identity provider. ([#36087](https://github.com/kubernetes/kubernetes/pull/36087), [@ericchiang](https://github.com/ericchiang)) +* Allow pods to define multiple environment variables from a whole ConfigMap ([#36245](https://github.com/kubernetes/kubernetes/pull/36245), [@fraenkel](https://github.com/fraenkel)) +* Refactor the certificate and kubeconfig code in the kubeadm binary into two phases ([#39280](https://github.com/kubernetes/kubernetes/pull/39280), [@luxas](https://github.com/luxas)) +* Added support for printing in all supported `--output` formats to `kubectl create ...` and `kubectl apply ...` ([#38112](https://github.com/kubernetes/kubernetes/pull/38112), [@juanvallejo](https://github.com/juanvallejo)) +* genericapiserver: extract CA cert from server cert and SNI cert chains ([#39022](https://github.com/kubernetes/kubernetes/pull/39022), [@sttts](https://github.com/sttts)) +* Endpoints, that tolerate unready Pods, are now listing Pods in state Terminating as well ([#37093](https://github.com/kubernetes/kubernetes/pull/37093), [@simonswine](https://github.com/simonswine)) +* DaemonSet ObservedGeneration ([#39157](https://github.com/kubernetes/kubernetes/pull/39157), [@lukaszo](https://github.com/lukaszo)) +* The `--reconcile-cidr` kubelet flag was removed since it had been deprecated since v1.5 ([#39322](https://github.com/kubernetes/kubernetes/pull/39322), [@luxas](https://github.com/luxas)) +* Add ready replicas in Deployments ([#37959](https://github.com/kubernetes/kubernetes/pull/37959), [@kargakis](https://github.com/kargakis)) +* Remove all MAINTAINER statements in Dockerfiles in the codebase as they are deprecated by docker ([#38927](https://github.com/kubernetes/kubernetes/pull/38927), [@luxas](https://github.com/luxas)) +* Remove the deprecated vsphere kube-up. ([#39140](https://github.com/kubernetes/kubernetes/pull/39140), [@kerneltime](https://github.com/kerneltime)) +* Kubectl top now also accepts short forms for "node" and "pod" ("no", "po") ([#39218](https://github.com/kubernetes/kubernetes/pull/39218), [@luksa](https://github.com/luksa)) +* Remove 'exec' network plugin - use CNI instead ([#39254](https://github.com/kubernetes/kubernetes/pull/39254), [@freehan](https://github.com/freehan)) +* Add three more columns to `kubectl get deploy -o wide` output. ([#39240](https://github.com/kubernetes/kubernetes/pull/39240), [@xingzhou](https://github.com/xingzhou)) +* Add path exist check in getPodVolumePathListFromDisk ([#38909](https://github.com/kubernetes/kubernetes/pull/38909), [@jingxu97](https://github.com/jingxu97)) +* Begin paths for internationalization in kubectl ([#36802](https://github.com/kubernetes/kubernetes/pull/36802), [@brendandburns](https://github.com/brendandburns)) +* Fixes an issue where commas were not accepted in --from-literal flags when creating secrets. Passing multiple values separated by a comma in a single --from-literal flag is no longer supported. Please use multiple --from-literal flags to provide multiple values. ([#35191](https://github.com/kubernetes/kubernetes/pull/35191), [@SamiHiltunen](https://github.com/SamiHiltunen)) +* Support loading UTF16 files if a byte-order-mark is present ([#39008](https://github.com/kubernetes/kubernetes/pull/39008), [@brendandburns](https://github.com/brendandburns)) +* Fix fsGroup to vSphere ([#38655](https://github.com/kubernetes/kubernetes/pull/38655), [@abrarshivani](https://github.com/abrarshivani)) +* ReplicaSet has onwer ref of the Deployment that created it ([#35676](https://github.com/kubernetes/kubernetes/pull/35676), [@krmayankk](https://github.com/krmayankk)) +* Don't evict static pods ([#39059](https://github.com/kubernetes/kubernetes/pull/39059), [@bprashanth](https://github.com/bprashanth)) +* Fixed a bug where the --server, --token, and --certificate-authority flags were not overriding the related in-cluster configs when provided in a `kubectl` call inside a cluster. ([#39006](https://github.com/kubernetes/kubernetes/pull/39006), [@fabianofranz](https://github.com/fabianofranz)) +* Make fluentd pods critical ([#39146](https://github.com/kubernetes/kubernetes/pull/39146), [@crassirostris](https://github.com/crassirostris)) +* assign -998 as the oom_score_adj for critical pods (e.g. kube-proxy) ([#39114](https://github.com/kubernetes/kubernetes/pull/39114), [@dchen1107](https://github.com/dchen1107)) +* delete continue in monitorNodeStatus ([#38798](https://github.com/kubernetes/kubernetes/pull/38798), [@NickrenREN](https://github.com/NickrenREN)) +* add create rolebinding ([#38991](https://github.com/kubernetes/kubernetes/pull/38991), [@deads2k](https://github.com/deads2k)) +* Add new command "kubectl set selector" ([#38966](https://github.com/kubernetes/kubernetes/pull/38966), [@kargakis](https://github.com/kargakis)) +* Federation: Add `batch/jobs` API objects to federation-apiserver ([#35943](https://github.com/kubernetes/kubernetes/pull/35943), [@jianhuiz](https://github.com/jianhuiz)) +* ABAC policies using "user":"*" or "group":"*" to match all users or groups will only match authenticated requests. To match unauthenticated requests, ABAC policies must explicitly specify "group":"system:unauthenticated" ([#38968](https://github.com/kubernetes/kubernetes/pull/38968), [@liggitt](https://github.com/liggitt)) +* To add local registry to libvirt_coreos ([#36751](https://github.com/kubernetes/kubernetes/pull/36751), [@sdminonne](https://github.com/sdminonne)) +* Add a KUBERNETES_NODE_* section to build kubelet/kube-proxy for windows ([#38919](https://github.com/kubernetes/kubernetes/pull/38919), [@brendandburns](https://github.com/brendandburns)) +* Added kubeadm commands to manage bootstrap tokens and the duration they are valid for. ([#35805](https://github.com/kubernetes/kubernetes/pull/35805), [@dgoodwin](https://github.com/dgoodwin)) +* AWS: Recognize ca-central-1 region ([#38410](https://github.com/kubernetes/kubernetes/pull/38410), [@justinsb](https://github.com/justinsb)) +* Unmount operation should not fail if volume is already unmounted ([#38547](https://github.com/kubernetes/kubernetes/pull/38547), [@rkouj](https://github.com/rkouj)) +* Changed default scsi controller type in vSphere Cloud Provider ([#38426](https://github.com/kubernetes/kubernetes/pull/38426), [@abrarshivani](https://github.com/abrarshivani)) +* Move non-generic apiserver code out of the generic packages ([#38191](https://github.com/kubernetes/kubernetes/pull/38191), [@sttts](https://github.com/sttts)) +* Remove extensions/v1beta1 Jobs resource, and job/v1beta1 generator. ([#38614](https://github.com/kubernetes/kubernetes/pull/38614), [@soltysh](https://github.com/soltysh)) +* Admit critical pods in the kubelet ([#38836](https://github.com/kubernetes/kubernetes/pull/38836), [@bprashanth](https://github.com/bprashanth)) +* Use daemonset in docker registry add on ([#35582](https://github.com/kubernetes/kubernetes/pull/35582), [@surajssd](https://github.com/surajssd)) +* Node affinity has moved from annotations to api fields in the pod spec. Node affinity that is defined in the annotations will be ignored. ([#37299](https://github.com/kubernetes/kubernetes/pull/37299), [@rrati](https://github.com/rrati)) +* Since `kubernetes.tar.gz` no longer includes client or server binaries, `cluster/kube-{up,down,push}.sh` now automatically download released binaries if they are missing. ([#38730](https://github.com/kubernetes/kubernetes/pull/38730), [@ixdy](https://github.com/ixdy)) +* genericapiserver: turn APIContainer.SecretRoutes into a real ServeMux ([#38826](https://github.com/kubernetes/kubernetes/pull/38826), [@sttts](https://github.com/sttts)) +* Migrated fluentd addon to daemon set ([#32088](https://github.com/kubernetes/kubernetes/pull/32088), [@piosz](https://github.com/piosz)) +* AWS: Add sequential allocator for device names. ([#38818](https://github.com/kubernetes/kubernetes/pull/38818), [@jsafrane](https://github.com/jsafrane)) +* Add 'X-Content-Type-Options: nosniff" to some error messages ([#37190](https://github.com/kubernetes/kubernetes/pull/37190), [@brendandburns](https://github.com/brendandburns)) +* Display pod node selectors with kubectl describe. ([#36396](https://github.com/kubernetes/kubernetes/pull/36396), [@aveshagarwal](https://github.com/aveshagarwal)) +* The main repository does not keep multiple releases of clientsets anymore. Please find previous releases at https://github.com/kubernetes/client-go ([#38154](https://github.com/kubernetes/kubernetes/pull/38154), [@caesarxuchao](https://github.com/caesarxuchao)) +* Remove a release-note on multiple OpenAPI specs ([#38732](https://github.com/kubernetes/kubernetes/pull/38732), [@mbohlool](https://github.com/mbohlool)) +* genericapiserver: unify swagger and openapi in config ([#38690](https://github.com/kubernetes/kubernetes/pull/38690), [@sttts](https://github.com/sttts)) +* fix connection upgrades through kuberentes-discovery ([#38724](https://github.com/kubernetes/kubernetes/pull/38724), [@deads2k](https://github.com/deads2k)) +* Fixes bug in resolving client-requested API versions ([#38533](https://github.com/kubernetes/kubernetes/pull/38533), [@DirectXMan12](https://github.com/DirectXMan12)) +* apiserver(s): Replace glog.Fatals with fmt.Errorfs ([#38175](https://github.com/kubernetes/kubernetes/pull/38175), [@sttts](https://github.com/sttts)) +* Remove Azure Subnet RouteTable check ([#38334](https://github.com/kubernetes/kubernetes/pull/38334), [@mogthesprog](https://github.com/mogthesprog)) +* Ensure the GCI metadata files do not have newline at the end ([#38727](https://github.com/kubernetes/kubernetes/pull/38727), [@Amey-D](https://github.com/Amey-D)) +* Significantly speed-up make ([#38700](https://github.com/kubernetes/kubernetes/pull/38700), [@sttts](https://github.com/sttts)) +* add QoS pod status field ([#37968](https://github.com/kubernetes/kubernetes/pull/37968), [@sjenning](https://github.com/sjenning)) +* Fixed validation of multizone cluster for GCE ([#38695](https://github.com/kubernetes/kubernetes/pull/38695), [@jszczepkowski](https://github.com/jszczepkowski)) +* Fixed detection of master during creation of multizone nodes cluster by kube-up. ([#38617](https://github.com/kubernetes/kubernetes/pull/38617), [@jszczepkowski](https://github.com/jszczepkowski)) +* Update CHANGELOG.md to warn about anon auth flag ([#38675](https://github.com/kubernetes/kubernetes/pull/38675), [@erictune](https://github.com/erictune)) +* Fixes NotAuthenticated errors that appear in the kubelet and kube-controller-manager due to never logging in to vSphere ([#36169](https://github.com/kubernetes/kubernetes/pull/36169), [@robdaemon](https://github.com/robdaemon)) +* Fix an issue where AWS tear-down leaks an DHCP Option Set. ([#38645](https://github.com/kubernetes/kubernetes/pull/38645), [@zmerlynn](https://github.com/zmerlynn)) +* Issue a warning when using `kubectl apply` on a resource lacking the `LastAppliedConfig` annotation ([#36672](https://github.com/kubernetes/kubernetes/pull/36672), [@ymqytw](https://github.com/ymqytw)) +* Re-add /healthz/ping handler in genericapiserver ([#38603](https://github.com/kubernetes/kubernetes/pull/38603), [@sttts](https://github.com/sttts)) +* Fail kubelet if runtime is unresponsive for 30 seconds ([#38527](https://github.com/kubernetes/kubernetes/pull/38527), [@derekwaynecarr](https://github.com/derekwaynecarr)) +* Add support for Azure Container Registry, update Azure dependencies ([#37783](https://github.com/kubernetes/kubernetes/pull/37783), [@brendandburns](https://github.com/brendandburns)) +* Fix panic in vSphere cloud provider ([#38423](https://github.com/kubernetes/kubernetes/pull/38423), [@BaluDontu](https://github.com/BaluDontu)) +* fix broken cluster/centos and enhance the style ([#34002](https://github.com/kubernetes/kubernetes/pull/34002), [@xiaoping378](https://github.com/xiaoping378)) +* Ability to quota storage by storage class ([#34554](https://github.com/kubernetes/kubernetes/pull/34554), [@derekwaynecarr](https://github.com/derekwaynecarr)) +* [Part 2] Adding s390x cross-compilation support for gcr.io images in this repo ([#36050](https://github.com/kubernetes/kubernetes/pull/36050), [@gajju26](https://github.com/gajju26)) +* kubectl run --rm no longer prints "pod xxx deleted" ([#38429](https://github.com/kubernetes/kubernetes/pull/38429), [@duglin](https://github.com/duglin)) +* Bump GCE debian image to container-vm-v20161208 ([release notes](https://cloud.google.com/compute/docs/containers/container_vms#changelog)) ([#38432](https://github.com/kubernetes/kubernetes/pull/38432), [@timstclair](https://github.com/timstclair)) +* Kubelet: Add image cache. ([#38375](https://github.com/kubernetes/kubernetes/pull/38375), [@Random-Liu](https://github.com/Random-Liu)) +* Allow a selector when retrieving logs ([#32752](https://github.com/kubernetes/kubernetes/pull/32752), [@fraenkel](https://github.com/fraenkel)) +* Fix unmountDevice issue caused by shared mount in GCI ([#38411](https://github.com/kubernetes/kubernetes/pull/38411), [@jingxu97](https://github.com/jingxu97)) +* [Federation] Implement dry run support in kubefed init ([#36447](https://github.com/kubernetes/kubernetes/pull/36447), [@irfanurrehman](https://github.com/irfanurrehman)) +* Fix space issue in volumePath with vSphere Cloud Provider ([#38338](https://github.com/kubernetes/kubernetes/pull/38338), [@BaluDontu](https://github.com/BaluDontu)) +* [Federation] Make federation etcd PVC size configurable ([#36310](https://github.com/kubernetes/kubernetes/pull/36310), [@irfanurrehman](https://github.com/irfanurrehman)) +* Allow no ports when exposing headless service ([#32811](https://github.com/kubernetes/kubernetes/pull/32811), [@fraenkel](https://github.com/fraenkel)) +* Wait for the port to be ready before starting ([#38260](https://github.com/kubernetes/kubernetes/pull/38260), [@fraenkel](https://github.com/fraenkel)) +* Add Version to the resource printer for 'get nodes' ([#37943](https://github.com/kubernetes/kubernetes/pull/37943), [@ailusazh](https://github.com/ailusazh)) +* contribute deis/registry-proxy as a replacement for kube-registry-proxy ([#35797](https://github.com/kubernetes/kubernetes/pull/35797), [@bacongobbler](https://github.com/bacongobbler)) +* [Part 1] Add support for cross-compiling s390x binaries ([#37092](https://github.com/kubernetes/kubernetes/pull/37092), [@gajju26](https://github.com/gajju26)) +* kernel memcg notification enabled via experimental flag ([#38258](https://github.com/kubernetes/kubernetes/pull/38258), [@derekwaynecarr](https://github.com/derekwaynecarr)) +* fix permissions when using fsGroup ([#37009](https://github.com/kubernetes/kubernetes/pull/37009), [@sjenning](https://github.com/sjenning)) +* Pipe get options to storage ([#37693](https://github.com/kubernetes/kubernetes/pull/37693), [@wojtek-t](https://github.com/wojtek-t)) +* add a configuration for kubelet to register as a node with taints ([#31647](https://github.com/kubernetes/kubernetes/pull/31647), [@mikedanese](https://github.com/mikedanese)) +* Remove genericapiserver.Options.MasterServiceNamespace ([#38186](https://github.com/kubernetes/kubernetes/pull/38186), [@sttts](https://github.com/sttts)) +* The --long-running-request-regexp flag to kube-apiserver is deprecated and will be removed in a future release. Long-running requests are now detected based on specific verbs (watch, proxy) or subresources (proxy, portforward, log, exec, attach). ([#38119](https://github.com/kubernetes/kubernetes/pull/38119), [@liggitt](https://github.com/liggitt)) +* Better compat with very old iptables (e.g. CentOS 6) ([#37594](https://github.com/kubernetes/kubernetes/pull/37594), [@thockin](https://github.com/thockin)) +* Fix GCI mounter issue ([#38124](https://github.com/kubernetes/kubernetes/pull/38124), [@jingxu97](https://github.com/jingxu97)) +* Kubelet will no longer set hairpin mode on every interface on the machine when an error occurs in setting up hairpin for a specific interface. ([#36990](https://github.com/kubernetes/kubernetes/pull/36990), [@bboreham](https://github.com/bboreham)) +* fix mesos unit tests ([#38196](https://github.com/kubernetes/kubernetes/pull/38196), [@deads2k](https://github.com/deads2k)) +* Add `--controllers` flag to federation controller manager for enable/disable federation ingress controller ([#36643](https://github.com/kubernetes/kubernetes/pull/36643), [@kzwang](https://github.com/kzwang)) +* Allow backendpools in Azure Load Balancers which are not owned by cloud provider ([#36882](https://github.com/kubernetes/kubernetes/pull/36882), [@codablock](https://github.com/codablock)) +* remove rbac super user ([#38121](https://github.com/kubernetes/kubernetes/pull/38121), [@deads2k](https://github.com/deads2k)) +* Fix resync goroutine leak in ListAndWatch ([#35672](https://github.com/kubernetes/kubernetes/pull/35672), [@tatsuhiro-t](https://github.com/tatsuhiro-t)) +* API server have two separate limits for read-only and mutating inflight requests. ([#36064](https://github.com/kubernetes/kubernetes/pull/36064), [@gmarek](https://github.com/gmarek)) +* check the value of min and max in kubectl ([#37789](https://github.com/kubernetes/kubernetes/pull/37789), [@yarntime](https://github.com/yarntime)) +* The glusterfs dynamic volume provisioner will now choose a unique GID for new persistent volumes from a range that can be configured in the storage class with the "gidMin" and "gidMax" parameters. The default range is 2000 - 2147483647 (max int32). ([#37886](https://github.com/kubernetes/kubernetes/pull/37886), [@obnoxxx](https://github.com/obnoxxx)) +* Add kubectl create poddisruptionbudget command ([#36646](https://github.com/kubernetes/kubernetes/pull/36646), [@kargakis](https://github.com/kargakis)) +* Set kernel.softlockup_panic =1 based on the flag. ([#38001](https://github.com/kubernetes/kubernetes/pull/38001), [@dchen1107](https://github.com/dchen1107)) +* portfordwardtester: avoid data loss during send+close+exit ([#37103](https://github.com/kubernetes/kubernetes/pull/37103), [@sttts](https://github.com/sttts)) +* Enable containerized mounter only for nfs and glusterfs types ([#37990](https://github.com/kubernetes/kubernetes/pull/37990), [@jingxu97](https://github.com/jingxu97)) +* Add flag to enable contention profiling in scheduler. ([#37357](https://github.com/kubernetes/kubernetes/pull/37357), [@gmarek](https://github.com/gmarek)) +* Add kubernetes-anywhere as a new e2e deployment option ([#37019](https://github.com/kubernetes/kubernetes/pull/37019), [@pipejakob](https://github.com/pipejakob)) +* add create clusterrolebinding command ([#37098](https://github.com/kubernetes/kubernetes/pull/37098), [@deads2k](https://github.com/deads2k)) +* kubectl create service externalname ([#34789](https://github.com/kubernetes/kubernetes/pull/34789), [@adohe](https://github.com/adohe)) +* Fix logic error in graceful deletion ([#37721](https://github.com/kubernetes/kubernetes/pull/37721), [@derekwaynecarr](https://github.com/derekwaynecarr)) +* Exit with error if is not the final parameter. ([#37723](https://github.com/kubernetes/kubernetes/pull/37723), [@mtaufen](https://github.com/mtaufen)) +* Fix Service Update on LoadBalancerSourceRanges Field ([#37720](https://github.com/kubernetes/kubernetes/pull/37720), [@freehan](https://github.com/freehan)) +* Bug fix. Incoming UDP packets not reach newly deployed services ([#32561](https://github.com/kubernetes/kubernetes/pull/32561), [@zreigz](https://github.com/zreigz)) +* GCI: Remove /var/lib/docker/network ([#37593](https://github.com/kubernetes/kubernetes/pull/37593), [@yujuhong](https://github.com/yujuhong)) +* configure local-up-cluster.sh to handle auth proxies ([#36838](https://github.com/kubernetes/kubernetes/pull/36838), [@deads2k](https://github.com/deads2k)) +* Add `clusterid`, an optional parameter to storageclass. ([#36437](https://github.com/kubernetes/kubernetes/pull/36437), [@humblec](https://github.com/humblec)) +* local-up-cluster: avoid sudo for control plane ([#37443](https://github.com/kubernetes/kubernetes/pull/37443), [@sttts](https://github.com/sttts)) +* Add error message when trying to use clusterrole with namespace in kubectl ([#36424](https://github.com/kubernetes/kubernetes/pull/36424), [@xilabao](https://github.com/xilabao)) +* `kube-up.sh`/`kube-down.sh` no longer force update gcloud for provider=gce|gke. ([#36292](https://github.com/kubernetes/kubernetes/pull/36292), [@jlowdermilk](https://github.com/jlowdermilk)) +* Fix issue when attempting to unmount a wrong vSphere volume ([#37413](https://github.com/kubernetes/kubernetes/pull/37413), [@BaluDontu](https://github.com/BaluDontu)) +* Fix the equality checks for numeric values in cluster/gce/util.sh. ([#37638](https://github.com/kubernetes/kubernetes/pull/37638), [@roberthbailey](https://github.com/roberthbailey)) +* kubelet: don't reject pods without adding them to the pod manager ([#37661](https://github.com/kubernetes/kubernetes/pull/37661), [@yujuhong](https://github.com/yujuhong)) +* Modify GCI mounter to enable NFSv3 ([#37582](https://github.com/kubernetes/kubernetes/pull/37582), [@jingxu97](https://github.com/jingxu97)) +* Fix photon controller plugin to construct with correct PdID ([#37167](https://github.com/kubernetes/kubernetes/pull/37167), [@luomiao](https://github.com/luomiao)) +* Collect logs for dead kubelets too ([#37671](https://github.com/kubernetes/kubernetes/pull/37671), [@mtaufen](https://github.com/mtaufen)) +* Set Dashboard UI version to v1.5.0 ([#37684](https://github.com/kubernetes/kubernetes/pull/37684), [@rf232](https://github.com/rf232)) +* When deleting an object with `--grace-period=0`, the client will begin a graceful deletion and wait until the resource is fully deleted. To force deletion, use the `--force` flag. ([#37263](https://github.com/kubernetes/kubernetes/pull/37263), [@smarterclayton](https://github.com/smarterclayton)) +* federation service controller: stop deleting services from underlying clusters when federated service is deleted. ([#37353](https://github.com/kubernetes/kubernetes/pull/37353), [@nikhiljindal](https://github.com/nikhiljindal)) +* Fix nil pointer dereference in test framework ([#37583](https://github.com/kubernetes/kubernetes/pull/37583), [@mtaufen](https://github.com/mtaufen)) +* Kubernetes now automatically installs a StorageClass object when deployed on ([#31617](https://github.com/kubernetes/kubernetes/pull/31617), [@jsafrane](https://github.com/jsafrane)) + * AWS, Google Compute Engine, Google Container Engine, and OpenStack using + * the default kube-up.sh scripts. This StorageClass is marked as default so that + * a PersistentVolumeClaim without a StorageClass annotation now results in + * automatic provisioning of storage (GCE PersistentDisk on Google Cloud, AWS + * EBS on AWS, and Cinder on OpenStack). In previous versions of Kubernetes + * a PersistentVolumeClaim without a StorageClass annotation on these cloud + * platforms would be satisfied by manually-created PersistentVolume objects. + * Administrators can choose to disable this behavior by deleting the automatically + * installed StorageClass API object. Alternatively, administrators may choose to + * keep the automatically installed StorageClass and only disable the defaulting + * behavior by removing the "is-default-class" annotation from the StorageClass + * API object. +* Fix TestServiceAlloc flakes ([#37487](https://github.com/kubernetes/kubernetes/pull/37487), [@wojtek-t](https://github.com/wojtek-t)) +* Use gsed on the Mac ([#37562](https://github.com/kubernetes/kubernetes/pull/37562), [@roberthbailey](https://github.com/roberthbailey)) +* Try self-repair scheduler cache or panic ([#37379](https://github.com/kubernetes/kubernetes/pull/37379), [@wojtek-t](https://github.com/wojtek-t)) +* Fluentd/Elastisearch add-on: correctly parse and index kubernetes labels ([#36857](https://github.com/kubernetes/kubernetes/pull/36857), [@Shrugs](https://github.com/Shrugs)) +* Mention overflows when mistakenly call function FromInt ([#36487](https://github.com/kubernetes/kubernetes/pull/36487), [@xialonglee](https://github.com/xialonglee)) +* Update doc for kubectl apply ([#37397](https://github.com/kubernetes/kubernetes/pull/37397), [@ymqytw](https://github.com/ymqytw)) +* Removes shorthand flag -w from kubectl apply ([#37345](https://github.com/kubernetes/kubernetes/pull/37345), [@MrHohn](https://github.com/MrHohn)) +* Curating Owners: pkg/api ([#36525](https://github.com/kubernetes/kubernetes/pull/36525), [@apelisse](https://github.com/apelisse)) +* Reduce verbosity of volume reconciler when attaching volumes ([#36900](https://github.com/kubernetes/kubernetes/pull/36900), [@codablock](https://github.com/codablock)) From 4afb98d6686f364014bb240d82d82c229384ceaa Mon Sep 17 00:00:00 2001 From: caleb miles Date: Wed, 15 Mar 2017 16:29:47 -0700 Subject: [PATCH 05/30] Add feature release notes for 1.6 Draws from [feature tracking spreadsheet](https://docs.google.com/spreadsheets/d/1nspIeRVNjAQHRslHQD1-6gPv99OcYZLMezrBe3Pfhhg/edit#gid=0) as of 2017-03-15 4:30PM PST --- release-1.6/release-notes-draft.md | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index b913a8d5a26..027e7995da5 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -5,6 +5,7 @@ ## API Machinery ### New Features ### Notable Changes +- The internal storage layer for kubernetes cluster state has been updated to use etcd v3 by default for new clusters. Old clusters will have to plan for a data migration window. ### Breaking Changes ## AWS @@ -15,16 +16,21 @@ ## Apps ### New Features ### Notable Changes +- Implement the update feature for DaemonSet. +- Deployments that cannot make progress in rolling out the newest version will now indicate via the API they are blocked ### Breaking Changes ## Auth ### New Features ### Notable Changes +- RBAC API is promoted to v1beta1 (rbac.authorization.k8s.io/v1beta1), and defines default roles for control plane, node, and controller components. ### Breaking Changes ## Autoscaling ### New Features ### Notable Changes +- The Horizontal Pod Autoscaler now supports drawing metrics throug the API server aggregator. +- The Horizontal Pod Autoscaler now supports scaling on multiple, custom metrics. ### Breaking Changes ## Big Data @@ -40,6 +46,10 @@ ## Cluster Lifecycle ### New Features ### Notable Changes +- New Bootstrap Token authentication and management method. Works well with kubeadm. +- Adds a new cloud-controller-manager binary that may be used for testing the new out-of-core cloudprovider flow +- Introduces an API for the Kubelet to request TLS certificates from the API server. +- kubeadm is enhanced and improved with a baseline feature set and command line flags that are now marked as beta. ### Breaking Changes ## Cluster Ops @@ -60,6 +70,7 @@ ## Federation ### New Features ### Notable Changes +- `kubefed` has graduated to beta: supports hosting federation on on-prem clusters, automatically configures `kube-dns` in joining clusters and allows passing arguments to federation components. ### Breaking Changes ## Instrumentation @@ -75,6 +86,8 @@ ## Node ### New Features ### Notable Changes +- The Docker-CRI implementation is Beta and is enabled by default in kubelet. You can disable it by --enable-cri=false. +- kubelet launches pods in a new cgroup hierarchy to better enforce quality of service. Operators must drain all pods from their nodes prior to upgrade of the kubelet. They must ensure the configured cgroup driver matches their associated container runtime cgroup driver. ### Breaking Changes ## On Prem @@ -99,9 +112,15 @@ ## Scheduling ### New Features ### Notable Changes +- Rules for spreading and packing pods relative to one another, within arbitrary topologies (node, zone, etc.) +- Per-pod configurable duration to stay bound to a node that becomes unreachable, not ready, or has other problems +- Rules for restricting which node(s) a pod can schedule onto +- User can run multiple schedulers in parallel resposible for different sets of pods +- Rules for "repelling" pods from nodes by default (support use cases like dedicated nodes, and reserve nodes with special features for pods that need those features) ### Breaking Changes ## Service Catalog +- Adds a new API resource `PodPreset` and admission controller to enable defining cross-cutting injection of Volumes and Environment into Pods. ### New Features ### Notable Changes ### Breaking Changes @@ -109,6 +128,16 @@ ## Storage ### New Features ### Notable Changes +- A new volume driver capable of projecting secrets, configmaps, and downward API items into the same directory. +- Flex volume plugin is updated to support attach/detach interfaces. It broke backward compatibility. Please update your drivers and implement the new callouts. +- Added support for mount options in persistent volumes +- StorageClass API is promoted to v1 (storage.k8s.io/v1) +- Default storage classes are deployed during installation on Azure, AWS, GCE, OpenStack and vSphere +- Populate environment variables from a configmap or secret. +- Support for user-written/run dynamic PV provisioners. See github.com/kubernetes-incubator/external-storage for a golang library and examples +- ScaleIO Kubernetes Volume Plugin added enabling pods to seamlessly access and use data stored on ScaleIO volumes. +- Portworx Volume Plugin added capability to use [Portworx](http://www.portworx.com) as a storage provider for Kubernetes clusters. Portworx pools your servers capacity and turns your servers or cloud instances into converged, highly available compute and storage nodes. +- Add support to use NFSv3, NFSv4, and GlusterFS on GCI image cluster ### Breaking Changes ## Testing From 1f4daf9541495a60c6a228fa8bbd0214ef828de1 Mon Sep 17 00:00:00 2001 From: Maciej Szulik Date: Thu, 16 Mar 2017 15:54:59 +0100 Subject: [PATCH 06/30] Sort some sig-apps and sig-cli changes --- release-1.6/release-notes-draft.md | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index 027e7995da5..2dbd97c0e35 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -15,10 +15,20 @@ ## Apps ### New Features +- Introduce apps/v1beta1.Deployments resource with modified defaults compared to extensions/v1beta1.Deployments. ([#39683](https://github.com/kubernetes/kubernetes/pull/39683), [@soltysh](https://github.com/soltysh)) +- Introduce new generator for apps/v1beta1 deployments ([#42362](https://github.com/kubernetes/kubernetes/pull/42362), [@soltysh](https://github.com/soltysh)) ### Notable Changes - Implement the update feature for DaemonSet. - Deployments that cannot make progress in rolling out the newest version will now indicate via the API they are blocked +- kubectl: respect deployment strategy parameters for rollout status ([#41809](https://github.com/kubernetes/kubernetes/pull/41809), [@kargakis](https://github.com/kargakis)) +- kubectl logs allows getting logs directly from deployment, job and statefulset ([#40927](https://github.com/kubernetes/kubernetes/pull/40927), [@soltysh](https://github.com/soltysh)) +- Deployment now fully respects ControllerRef to avoid fighting over Pods and ReplicaSets. At the time of upgrade, **you must not have Deployments with selectors that overlap**, or else [ownership of ReplicaSets may change](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md#upgrading). ([#42175](https://github.com/kubernetes/kubernetes/pull/42175), [@enisoc](https://github.com/enisoc)) +- StatefulSet now respects ControllerRef to avoid fighting over Pods. At the time of upgrade, **you must not have StatefulSets with selectors that overlap** with any other controllers (such as ReplicaSets), or else [ownership of Pods may change](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md#upgrading). ([#42080](https://github.com/kubernetes/kubernetes/pull/42080), [@enisoc](https://github.com/enisoc)) +- Add ready replicas in Deployments ([#37959](https://github.com/kubernetes/kubernetes/pull/37959), [@kargakis](https://github.com/kargakis)) +- Add configurable limits to CronJob resource to specify how many successful and failed jobs are preserved. ([#40932](https://github.com/kubernetes/kubernetes/pull/40932), [@peay](https://github.com/peay)) +- ReplicaSet has onwer ref of the Deployment that created it ([#35676](https://github.com/kubernetes/kubernetes/pull/35676), [@krmayankk](https://github.com/krmayankk)) ### Breaking Changes +- Remove extensions/v1beta1 Jobs resource, and job/v1beta1 generator. ([#38614](https://github.com/kubernetes/kubernetes/pull/38614), [@soltysh](https://github.com/soltysh)) ## Auth ### New Features @@ -40,7 +50,10 @@ ## CLI ### New Features +- Add new command "kubectl set selector" ([#38966](https://github.com/kubernetes/kubernetes/pull/38966), [@kargakis](https://github.com/kargakis)) +- Add kubectl create poddisruptionbudget command ([#36646](https://github.com/kubernetes/kubernetes/pull/36646), [@kargakis](https://github.com/kargakis)) ### Notable Changes +- `kubectl edit` now edits objects exactly as they were retrieved from the API. This allows using `kubectl edit` with third-party resources and extension API servers. Because client-side conversion is no longer done, the `--output-version` option is deprecated for `kubectl edit`. To edit using a particular API version, fully-qualify the resource, version, and group used to fetch the object (for example, `job.v1.batch/myjob`) ([#41304](https://github.com/kubernetes/kubernetes/pull/41304), [@liggitt](https://github.com/liggitt)) ### Breaking Changes ## Cluster Lifecycle @@ -155,7 +168,7 @@ ### Notable Changes ### Breaking Changes -# Auto Generated Release Notes +# Auto Generated Release Notes ## (please move information into the correct SIG) **Release Note Preview - generated on Wed Mar 15 22:47:14 UTC 2017** @@ -184,8 +197,6 @@ ### Action Required -* Deployment now fully respects ControllerRef to avoid fighting over Pods and ReplicaSets. At the time of upgrade, **you must not have Deployments with selectors that overlap**, or else [ownership of ReplicaSets may change](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md#upgrading). ([#42175](https://github.com/kubernetes/kubernetes/pull/42175), [@enisoc](https://github.com/enisoc)) -* StatefulSet now respects ControllerRef to avoid fighting over Pods. At the time of upgrade, **you must not have StatefulSets with selectors that overlap** with any other controllers (such as ReplicaSets), or else [ownership of Pods may change](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/controller-ref.md#upgrading). ([#42080](https://github.com/kubernetes/kubernetes/pull/42080), [@enisoc](https://github.com/enisoc)) * The --dns-provider argument of 'kubefed init' is now mandatory and does not default to `google-clouddns`. To initialize a Federation control plane with Google Cloud DNS, use the following invocation: 'kubefed init --dns-provider=google-clouddns' ([#42092](https://github.com/kubernetes/kubernetes/pull/42092), [@marun](https://github.com/marun)) * Change taints/tolerations to api fields ([#38957](https://github.com/kubernetes/kubernetes/pull/38957), [@aveshagarwal](https://github.com/aveshagarwal)) * Promote certificates.k8s.io to beta and enable it by default. Users using the alpha certificates API should delete v1alpha1 CSRs from the API before upgrading and recreate them as v1beta1 CSR after upgrading. ([#39772](https://github.com/kubernetes/kubernetes/pull/39772), [@mikedanese](https://github.com/mikedanese)) @@ -206,7 +217,6 @@ * hack/godep-restore.sh: use godep v79 which works ([#42965](https://github.com/kubernetes/kubernetes/pull/42965), [@sttts](https://github.com/sttts)) * Patch CVE-2016-8859 in gcr.io/google-containers/cluster-proportional-autoscaler-amd64 ([#42933](https://github.com/kubernetes/kubernetes/pull/42933), [@timstclair](https://github.com/timstclair)) * Disable devicemapper thin_ls due to excessive iops ([#42899](https://github.com/kubernetes/kubernetes/pull/42899), [@dashpole](https://github.com/dashpole)) -* Introduce new generator for apps/v1beta1 deployments ([#42362](https://github.com/kubernetes/kubernetes/pull/42362), [@soltysh](https://github.com/soltysh)) * Use Prometheus instrumentation conventions ([#36704](https://github.com/kubernetes/kubernetes/pull/36704), [@fabxc](https://github.com/fabxc)) * Add new DaemonSet status fields to kubectl printer and describer. ([#42843](https://github.com/kubernetes/kubernetes/pull/42843), [@janetkuo](https://github.com/janetkuo)) * Dropped the support for docker 1.9.x and the belows. ([#42694](https://github.com/kubernetes/kubernetes/pull/42694), [@dchen1107](https://github.com/dchen1107)) @@ -228,7 +238,6 @@ * is a list of upstreamNameservers to use, overriding the configuration * specified in /etc/resolv.conf. * kubelet exports metrics for cgroup management ([#41988](https://github.com/kubernetes/kubernetes/pull/41988), [@sjenning](https://github.com/sjenning)) -* kubectl: respect deployment strategy parameters for rollout status ([#41809](https://github.com/kubernetes/kubernetes/pull/41809), [@kargakis](https://github.com/kargakis)) * Remove cmd/kube-discovery from the tree since it's not necessary anymore ([#42070](https://github.com/kubernetes/kubernetes/pull/42070), [@luxas](https://github.com/luxas)) * kubeadm: Hook up kubeadm against the BootstrapSigner ([#41417](https://github.com/kubernetes/kubernetes/pull/41417), [@luxas](https://github.com/luxas)) * Federated Ingress over GCE no longer requires separate firewall rules to be created for each cluster to circumvent flapping firewall health checks. ([#41942](https://github.com/kubernetes/kubernetes/pull/41942), [@csbell](https://github.com/csbell)) @@ -249,7 +258,6 @@ * Update defaultbackend image to 1.3 ([#42212](https://github.com/kubernetes/kubernetes/pull/42212), [@timstclair](https://github.com/timstclair)) * Allow the Horizontal Pod Autoscaler controller to talk to the metrics API and custom metrics API as standard APIs. ([#41824](https://github.com/kubernetes/kubernetes/pull/41824), [@DirectXMan12](https://github.com/DirectXMan12)) * Implement support for mount options in PVs ([#41906](https://github.com/kubernetes/kubernetes/pull/41906), [@gnufied](https://github.com/gnufied)) -* Introduce apps/v1beta1.Deployments resource with modified defaults compared to extensions/v1beta1.Deployments. ([#39683](https://github.com/kubernetes/kubernetes/pull/39683), [@soltysh](https://github.com/soltysh)) * Add DNS suffix search list support in Windows kube-proxy. ([#41618](https://github.com/kubernetes/kubernetes/pull/41618), [@JiangtianLi](https://github.com/JiangtianLi)) * `--experimental-nvidia-gpus` flag is **replaced** by `Accelerators` alpha feature gate along with support for multiple Nvidia GPUs. ([#42116](https://github.com/kubernetes/kubernetes/pull/42116), [@vishh](https://github.com/vishh)) * To use GPUs, pass `Accelerators=true` as part of `--feature-gates` flag. @@ -274,7 +282,6 @@ * New Kubelet flag `--enforce-node-allocatable` with a default value of `pods` is added which will make kubelet create a top level cgroup for all pods to enforce Node Allocatable. Optionally, `system-reserved` & `kube-reserved` values can also be specified separated by comma to enforce node allocatable on cgroups specified via `--system-reserved-cgroup` & `--kube-reserved-cgroup` respectively. Note the default value of the latter flags are "". ([#41234](https://github.com/kubernetes/kubernetes/pull/41234), [@vishh](https://github.com/vishh)) * This feature requires a **Node Drain** prior to upgrade failing which pods will be restarted if possible or terminated if they have a `RestartNever` policy. * Deployment of AWS Kubernetes clusters using the in-tree bash deployment (i.e. cluster/kube-up.sh or get-kube.sh) is obsolete. v1.5.x will be the last release to support cluster/kube-up.sh with AWS. For a list of viable alternatives, see: http://kubernetes.io/docs/getting-started-guides/aws/ ([#42196](https://github.com/kubernetes/kubernetes/pull/42196), [@zmerlynn](https://github.com/zmerlynn)) -* kubectl logs allows getting logs directly from deployment, job and statefulset ([#40927](https://github.com/kubernetes/kubernetes/pull/40927), [@soltysh](https://github.com/soltysh)) * make kubectl taint command respect effect NoExecute ([#42120](https://github.com/kubernetes/kubernetes/pull/42120), [@kevin-wangzefeng](https://github.com/kevin-wangzefeng)) * Flex volume plugin is updated to support attach/detach interfaces. It broke backward compatibility. Please update your drivers and implement the new callouts. ([#41804](https://github.com/kubernetes/kubernetes/pull/41804), [@chakri-nelluri](https://github.com/chakri-nelluri)) * Implement the update feature for DaemonSet. ([#41116](https://github.com/kubernetes/kubernetes/pull/41116), [@lukaszo](https://github.com/lukaszo)) @@ -293,7 +300,6 @@ * add kubectl can-i to see if you can perform an action ([#41077](https://github.com/kubernetes/kubernetes/pull/41077), [@deads2k](https://github.com/deads2k)) * enable DefaultTolerationSeconds admission controller by default ([#41815](https://github.com/kubernetes/kubernetes/pull/41815), [@kevin-wangzefeng](https://github.com/kevin-wangzefeng)) * Make DaemonSets survive taint-based evictions when nodes turn unreachable/notReady. ([#41896](https://github.com/kubernetes/kubernetes/pull/41896), [@kevin-wangzefeng](https://github.com/kevin-wangzefeng)) -* Add configurable limits to CronJob resource to specify how many successful and failed jobs are preserved. ([#40932](https://github.com/kubernetes/kubernetes/pull/40932), [@peay](https://github.com/peay)) * Deprecate outofdisk-transition-frequency and low-diskspace-threshold-mb flags ([#41941](https://github.com/kubernetes/kubernetes/pull/41941), [@dashpole](https://github.com/dashpole)) * Add OWNERS for sample-apiserver in staging ([#42094](https://github.com/kubernetes/kubernetes/pull/42094), [@sttts](https://github.com/sttts)) * Update gcr.io/google-containers/rescheduler to v0.2.2, which uses busybox as a base image instead of ubuntu. ([#41911](https://github.com/kubernetes/kubernetes/pull/41911), [@ixdy](https://github.com/ixdy)) @@ -354,7 +360,6 @@ * On kube-up.sh clusters on GCE, kube-scheduler now contacts the API on the secured port. ([#41285](https://github.com/kubernetes/kubernetes/pull/41285), [@liggitt](https://github.com/liggitt)) * Default RBAC ClusterRole and ClusterRoleBinding objects are automatically updated at server start to add missing permissions and subjects (extra permissions and subjects are left in place). To prevent autoupdating a particular role or rolebinding, annotate it with `rbac.authorization.kubernetes.io/autoupdate=false`. ([#41155](https://github.com/kubernetes/kubernetes/pull/41155), [@liggitt](https://github.com/liggitt)) * Make EnableCRI default to true ([#41378](https://github.com/kubernetes/kubernetes/pull/41378), [@yujuhong](https://github.com/yujuhong)) -* `kubectl edit` now edits objects exactly as they were retrieved from the API. This allows using `kubectl edit` with third-party resources and extension API servers. Because client-side conversion is no longer done, the `--output-version` option is deprecated for `kubectl edit`. To edit using a particular API version, fully-qualify the resource, version, and group used to fetch the object (for example, `job.v1.batch/myjob`) ([#41304](https://github.com/kubernetes/kubernetes/pull/41304), [@liggitt](https://github.com/liggitt)) * We change the default attach_detach_controller sync period to 1 minute to reduce the query frequency through cloud provider to check whether volumes are attached or not. ([#41363](https://github.com/kubernetes/kubernetes/pull/41363), [@jingxu97](https://github.com/jingxu97)) * RBAC `v1beta1` RoleBinding/ClusterRoleBinding subjects changed `apiVersion` to `apiGroup` to fully-qualify a subject. ServiceAccount subjects default to an apiGroup of `""`, User and Group subjects default to an apiGroup of `"rbac.authorization.k8s.io"`. ([#41184](https://github.com/kubernetes/kubernetes/pull/41184), [@liggitt](https://github.com/liggitt)) * Add support for finalizers in federated configmaps (deletes configmaps from underlying clusters). ([#40464](https://github.com/kubernetes/kubernetes/pull/40464), [@csbell](https://github.com/csbell)) @@ -504,7 +509,6 @@ * Endpoints, that tolerate unready Pods, are now listing Pods in state Terminating as well ([#37093](https://github.com/kubernetes/kubernetes/pull/37093), [@simonswine](https://github.com/simonswine)) * DaemonSet ObservedGeneration ([#39157](https://github.com/kubernetes/kubernetes/pull/39157), [@lukaszo](https://github.com/lukaszo)) * The `--reconcile-cidr` kubelet flag was removed since it had been deprecated since v1.5 ([#39322](https://github.com/kubernetes/kubernetes/pull/39322), [@luxas](https://github.com/luxas)) -* Add ready replicas in Deployments ([#37959](https://github.com/kubernetes/kubernetes/pull/37959), [@kargakis](https://github.com/kargakis)) * Remove all MAINTAINER statements in Dockerfiles in the codebase as they are deprecated by docker ([#38927](https://github.com/kubernetes/kubernetes/pull/38927), [@luxas](https://github.com/luxas)) * Remove the deprecated vsphere kube-up. ([#39140](https://github.com/kubernetes/kubernetes/pull/39140), [@kerneltime](https://github.com/kerneltime)) * Kubectl top now also accepts short forms for "node" and "pod" ("no", "po") ([#39218](https://github.com/kubernetes/kubernetes/pull/39218), [@luksa](https://github.com/luksa)) @@ -515,14 +519,12 @@ * Fixes an issue where commas were not accepted in --from-literal flags when creating secrets. Passing multiple values separated by a comma in a single --from-literal flag is no longer supported. Please use multiple --from-literal flags to provide multiple values. ([#35191](https://github.com/kubernetes/kubernetes/pull/35191), [@SamiHiltunen](https://github.com/SamiHiltunen)) * Support loading UTF16 files if a byte-order-mark is present ([#39008](https://github.com/kubernetes/kubernetes/pull/39008), [@brendandburns](https://github.com/brendandburns)) * Fix fsGroup to vSphere ([#38655](https://github.com/kubernetes/kubernetes/pull/38655), [@abrarshivani](https://github.com/abrarshivani)) -* ReplicaSet has onwer ref of the Deployment that created it ([#35676](https://github.com/kubernetes/kubernetes/pull/35676), [@krmayankk](https://github.com/krmayankk)) * Don't evict static pods ([#39059](https://github.com/kubernetes/kubernetes/pull/39059), [@bprashanth](https://github.com/bprashanth)) * Fixed a bug where the --server, --token, and --certificate-authority flags were not overriding the related in-cluster configs when provided in a `kubectl` call inside a cluster. ([#39006](https://github.com/kubernetes/kubernetes/pull/39006), [@fabianofranz](https://github.com/fabianofranz)) * Make fluentd pods critical ([#39146](https://github.com/kubernetes/kubernetes/pull/39146), [@crassirostris](https://github.com/crassirostris)) * assign -998 as the oom_score_adj for critical pods (e.g. kube-proxy) ([#39114](https://github.com/kubernetes/kubernetes/pull/39114), [@dchen1107](https://github.com/dchen1107)) * delete continue in monitorNodeStatus ([#38798](https://github.com/kubernetes/kubernetes/pull/38798), [@NickrenREN](https://github.com/NickrenREN)) * add create rolebinding ([#38991](https://github.com/kubernetes/kubernetes/pull/38991), [@deads2k](https://github.com/deads2k)) -* Add new command "kubectl set selector" ([#38966](https://github.com/kubernetes/kubernetes/pull/38966), [@kargakis](https://github.com/kargakis)) * Federation: Add `batch/jobs` API objects to federation-apiserver ([#35943](https://github.com/kubernetes/kubernetes/pull/35943), [@jianhuiz](https://github.com/jianhuiz)) * ABAC policies using "user":"*" or "group":"*" to match all users or groups will only match authenticated requests. To match unauthenticated requests, ABAC policies must explicitly specify "group":"system:unauthenticated" ([#38968](https://github.com/kubernetes/kubernetes/pull/38968), [@liggitt](https://github.com/liggitt)) * To add local registry to libvirt_coreos ([#36751](https://github.com/kubernetes/kubernetes/pull/36751), [@sdminonne](https://github.com/sdminonne)) @@ -532,7 +534,6 @@ * Unmount operation should not fail if volume is already unmounted ([#38547](https://github.com/kubernetes/kubernetes/pull/38547), [@rkouj](https://github.com/rkouj)) * Changed default scsi controller type in vSphere Cloud Provider ([#38426](https://github.com/kubernetes/kubernetes/pull/38426), [@abrarshivani](https://github.com/abrarshivani)) * Move non-generic apiserver code out of the generic packages ([#38191](https://github.com/kubernetes/kubernetes/pull/38191), [@sttts](https://github.com/sttts)) -* Remove extensions/v1beta1 Jobs resource, and job/v1beta1 generator. ([#38614](https://github.com/kubernetes/kubernetes/pull/38614), [@soltysh](https://github.com/soltysh)) * Admit critical pods in the kubelet ([#38836](https://github.com/kubernetes/kubernetes/pull/38836), [@bprashanth](https://github.com/bprashanth)) * Use daemonset in docker registry add on ([#35582](https://github.com/kubernetes/kubernetes/pull/35582), [@surajssd](https://github.com/surajssd)) * Node affinity has moved from annotations to api fields in the pod spec. Node affinity that is defined in the annotations will be ignored. ([#37299](https://github.com/kubernetes/kubernetes/pull/37299), [@rrati](https://github.com/rrati)) @@ -595,7 +596,6 @@ * API server have two separate limits for read-only and mutating inflight requests. ([#36064](https://github.com/kubernetes/kubernetes/pull/36064), [@gmarek](https://github.com/gmarek)) * check the value of min and max in kubectl ([#37789](https://github.com/kubernetes/kubernetes/pull/37789), [@yarntime](https://github.com/yarntime)) * The glusterfs dynamic volume provisioner will now choose a unique GID for new persistent volumes from a range that can be configured in the storage class with the "gidMin" and "gidMax" parameters. The default range is 2000 - 2147483647 (max int32). ([#37886](https://github.com/kubernetes/kubernetes/pull/37886), [@obnoxxx](https://github.com/obnoxxx)) -* Add kubectl create poddisruptionbudget command ([#36646](https://github.com/kubernetes/kubernetes/pull/36646), [@kargakis](https://github.com/kargakis)) * Set kernel.softlockup_panic =1 based on the flag. ([#38001](https://github.com/kubernetes/kubernetes/pull/38001), [@dchen1107](https://github.com/dchen1107)) * portfordwardtester: avoid data loss during send+close+exit ([#37103](https://github.com/kubernetes/kubernetes/pull/37103), [@sttts](https://github.com/sttts)) * Enable containerized mounter only for nfs and glusterfs types ([#37990](https://github.com/kubernetes/kubernetes/pull/37990), [@jingxu97](https://github.com/jingxu97)) From a5df0f05adaee0405cfaa6b5a6122afc6d6923c9 Mon Sep 17 00:00:00 2001 From: David Oppenheimer Date: Thu, 16 Mar 2017 16:30:41 -0700 Subject: [PATCH 07/30] Add release notes for scheduler features moving from alpha to beta, and new alpha feature. --- release-1.6/release-notes-draft.md | 40 ++++++++++++++++++++++++++---- 1 file changed, 35 insertions(+), 5 deletions(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index 2dbd97c0e35..deb7e777959 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -124,14 +124,44 @@ ## Scheduling ### New Features + +- **(Alpha feature)** Represent node problems "not ready" and "unreachable" using `NoExecute` taints. In combination with `tolerationSeconds` described below, this allows per-pod specification of how long to remain bound to a node that becomes unreachable or not ready, rather than using the default of 5 minutes. You can enable this alpha feature by including `TaintBasedEvictions=true` in `--feature-gates`, such as `--feature-gates=FooBar=true,TaintBasedEvictions=true`. Documentation is [here](https://kubernetes.io/docs/user-guide/node-selection/). + ### Notable Changes -- Rules for spreading and packing pods relative to one another, within arbitrary topologies (node, zone, etc.) -- Per-pod configurable duration to stay bound to a node that becomes unreachable, not ready, or has other problems -- Rules for restricting which node(s) a pod can schedule onto -- User can run multiple schedulers in parallel resposible for different sets of pods -- Rules for "repelling" pods from nodes by default (support use cases like dedicated nodes, and reserve nodes with special features for pods that need those features) + +- The [multiple schedulers](https://kubernetes.io/docs/admin/multiple-schedulers/) feature is now in Beta. This feature allows you to run multiple schedulers in parallel, each responsible for different sets of pods. When using multiple schedulers, the scheduler name is now specified in a new-in-1.6 `schedulerName` field of the PodSpec rather than using the `scheduler.alpha.kubernetes.io/name` annotation on the Pod. When you upgrade to 1.6, the Kubernetes default scheduler will start using the `schedulerName` field of the PodSpec and will ignore the annotation. + +- [Node affinity/anti-affinity](https://kubernetes.io/docs/user-guide/node-selection/) and [pod affinity/anti-affinity](https://kubernetes.io/docs/user-guide/node-selection/) are now in Beta. Node affinity/anti-affinity allow you to specify rules for restricting which node(s) a pod can schedule onto, based on the labels on the node. Pod affinity/anti-affinity allow you to specify rules for spreading and packing pods relative to one another, across arbitrary topologies (node, zone, etc.) These affinity rules are now be specified in a new-in-1.6 `affinity` field of the PodSpec. Kubernetes 1.6 continues to support the alpha `scheduler.alpha.kubernetes.io/affinity` annotation on the Pod if you explicitly enable the alpha feature "AffinityInAnnotations", but it will be removed in a future release. When you upgrade to 1.6, if you have not enabled the alpha feature, then the scheduler will use the `affinity` field in PodSpec and will ignore the annotation. If you have enabled the alpha feature, then the scheduler will use the `affinity` field in PodSpec if it is present, and otherwise will use the annotation. + +- [Taints and tolerations](https://kubernetes.io/docs/user-guide/node-selection/) are now in Beta. This feature allows you to specify rules for "repelling" pods from nodes by default, which enables use cases like dedicated nodes and reserving nodes with special features for pods that need those features. We've also added a `NoExecute` taint type that evicts already-running pods, and an associated `tolerationSeconds` field to tolerations to delay the eviction for a specified amount of time. As before, taints are created using `kubectl taint` (but internally they are now represented as a field `taints` in the NodeSpec rather than using the `scheduler.alpha.kubernetes.io/taints` annotation on Node). Tolerations are now specified in a new-in-1.6 `tolerations` field of the PodSpec rather than using the `scheduler.alpha.kubernetes.io/tolerations` annotation on the Pod. When you upgrade to 1.6, the scheduler will start using the fields and will ignore the annotations. + ### Breaking Changes +The features described above are now specified using fields rather than annotations. As a result, you must take the following actions if you are already using any of the features (in their alpha form): + +- **Multiple schedulers** + - Modify your PodSpecs that currently use the `scheduler.alpha.kubernetes.io/name` annotation on Pod, to instead use the `schedulerName` field in the PodSpec. + - Modify any custom scheduler(s) you have written so that they read the `schedulerName` field on Pod instead of the `scheduler.alpha.kubernetes.io/name` annotation. + - Note that you can only start using the `schedulerName` field **after** you upgrade to 1.6; it is not recognized in 1.5. + +- **Node affinity/anti-affinity and pod affinity/anti-affinity** + - You can continue to use the alpha version of this feature, in which you specify affinity requests using Pod annotations, in 1.6 by including `AffinityInAnnotations=true` in `--feature-gates`, such as `--feature-gates=FooBar=true,AffinityInAnnotations=true`. Otherwise, you must modify your PodSpecs that currently use the `scheduler.alpha.kubernetes.io/affinity` annotation on Pod, to instead use the `affinity` field in the PodSpec. Support for the annotation will be removed in a future release, so we encourage you to switch to using the field as soon as possible. + - Note that you can only start using the `affinity` field **after** you upgrade to 1.6; it is not recognized in 1.5. + - The `--failure-domains` scheduler command line-argument is not supported in the beta vesion of the feature. + +- **Taints** + - You will need to use `kubectl taint` to re-create all of your taints after kubectl and the master are upgraded to 1.6. Between the time the master is upgraded to 1.6 and when you do this, your existing taints will have no effect. + - You can find out what taints you have in place on a node while you are still running Kubernetes 1.5 by doing `kubectl describe node `; the `Taints` section will show the taints you have in place. To see the taints that were created under 1.5 when you are running 1.6, do `kubectl get node -o yaml` and look for the "Annotation" section with the annotation key `scheduler.alpha.kubernetes.io/taints` + - You can remove the "old" taints (stored internally as annotations) at any time after the upgrade by doing `kubectl annotate nodes scheduler.alpha.kubernetes.io/taints-` +(note the minus at the end, which means "delete the taint with this key") + - Note that because any taints you might have created with Kubernetes 1.5 can only affect the scheduling of new pods (the `NoExecute` taint effect is introduced in 1.6), neither the master upgrade nor your running `kubectl taint` to re-create the taints will affect pods that are already running. + +- **Tolerations** + - After your master is upgraded to 1.6, you will need to update your PodSpecs to set the `tolerations` field of the PodSpec and remove the `scheduler.alpha.kubernetes.io/tolerations` annotation on the Pod. (It's not strictly necessary to remove the annotation, as it will have no effect once you upgrade to 1.6.) Between the time the master is upgraded to 1.6 and when you do this, tolerations attached to Pods that are created will have no effect. Pods that are already running will not be affected by the upgrade. + - You can find the PodSpec tolerations that were created as annotations (if any) in a controller definition by doing `kubectl get -o yaml` and looking for the "Annotation" section with the annotation key `scheduler.alpha.kubernetes.io/tolerations` (This will work whether you are running Kubernetes 1.5 or 1.6). + - To update a controller's PodSpec to use the field instead of the annotation, use one of the kubectl commands that do update ("kubectl replace" or "kubectl apply" or "kubectl patch") or just delete the controller entirely and re-create it with a new pod template. Note that you will be able to do these things only after the upgrade. + + ## Service Catalog - Adds a new API resource `PodPreset` and admission controller to enable defining cross-cutting injection of Volumes and Environment into Pods. ### New Features From c660ce43a6e3a3ad9dd67c6863374e1a9f9aacfb Mon Sep 17 00:00:00 2001 From: Michael Michael Date: Thu, 16 Mar 2017 22:34:31 -0500 Subject: [PATCH 08/30] sig-windows update --- release-1.6/release-notes-draft.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index deb7e777959..7f681bb1493 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -194,9 +194,7 @@ The features described above are now specified using fields rather than annotati ### Breaking Changes ## Windows -### New Features -### Notable Changes -### Breaking Changes +[Nothing for this release] # Auto Generated Release Notes ## (please move information into the correct SIG) From 8785e3e183c424170125c0445242ff5f4dd002e8 Mon Sep 17 00:00:00 2001 From: David Oppenheimer Date: Fri, 17 Mar 2017 12:03:13 -0700 Subject: [PATCH 09/30] Add manual release note for #43271 --- release-1.6/release-notes-draft.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index deb7e777959..e62209f06d4 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -145,7 +145,8 @@ The features described above are now specified using fields rather than annotati - Note that you can only start using the `schedulerName` field **after** you upgrade to 1.6; it is not recognized in 1.5. - **Node affinity/anti-affinity and pod affinity/anti-affinity** - - You can continue to use the alpha version of this feature, in which you specify affinity requests using Pod annotations, in 1.6 by including `AffinityInAnnotations=true` in `--feature-gates`, such as `--feature-gates=FooBar=true,AffinityInAnnotations=true`. Otherwise, you must modify your PodSpecs that currently use the `scheduler.alpha.kubernetes.io/affinity` annotation on Pod, to instead use the `affinity` field in the PodSpec. Support for the annotation will be removed in a future release, so we encourage you to switch to using the field as soon as possible. + - You can continue to use the alpha version of this feature (with one caveat -- see below), in which you specify affinity requests using Pod annotations, in 1.6 by including `AffinityInAnnotations=true` in `--feature-gates`, such as `--feature-gates=FooBar=true,AffinityInAnnotations=true`. Otherwise, you must modify your PodSpecs that currently use the `scheduler.alpha.kubernetes.io/affinity` annotation on Pod, to instead use the `affinity` field in the PodSpec. Support for the annotation will be removed in a future release, so we encourage you to switch to using the field as soon as possible. + - Caveat: The alpha version no longer supports, and the beta version does not support, the "empty `podAffinityTerm.namespaces` list means all namespaces" behavior. In both alpha and beta it now means "same namespace as the pod specifying this affinity rule." - Note that you can only start using the `affinity` field **after** you upgrade to 1.6; it is not recognized in 1.5. - The `--failure-domains` scheduler command line-argument is not supported in the beta vesion of the feature. From edff3d4c0ec19d329b83cedfe262004fec6c91fb Mon Sep 17 00:00:00 2001 From: Luke Marsden Date: Fri, 17 Mar 2017 19:31:34 +0000 Subject: [PATCH 10/30] First draft of kubeadm 1.6 release notes. --- release-1.6/release-notes-draft.md | 29 +++++++++++++++++++++++++---- 1 file changed, 25 insertions(+), 4 deletions(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index deb7e777959..7930356bed3 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -58,12 +58,33 @@ ## Cluster Lifecycle ### New Features +- kubeadm is a simple tool for creating Kubernetes clusters on existing + machines. kubeadm's command line interface is now in beta. This means that + using it is considered safe. + Please [try it out](https://kubernetes.io/docs/getting-started-guides/kubeadm/) and + [give us feedback](https://kubernetes.io/docs/getting-started-guides/kubeadm/#feedback)! + ### Notable Changes -- New Bootstrap Token authentication and management method. Works well with kubeadm. -- Adds a new cloud-controller-manager binary that may be used for testing the new out-of-core cloudprovider flow -- Introduces an API for the Kubelet to request TLS certificates from the API server. -- kubeadm is enhanced and improved with a baseline feature set and command line flags that are now marked as beta. +- kubeadm 1.6 sets up a more secure cluster by default. It uses [RBAC](https://kubernetes.io/) + to grant limited privileges to workloads running on the cluster. By default, + service accounts and users are restricted from accessing the API. They can be + granted permissions + [using the RBAC command-line utilities](https://kubernetes.io/docs/admin/authorization/rbac/#command-line-utilities). +- kubeadm now uses a new [bootstrap mechanism](https://kubernetes.io/docs/admin/bootstrap-tokens/) + which is baked into the Kubernetes API server, rather than a separate + service. The API for TLS bootstrapping is now baked in to the Kubernetes API + server also. +- kubeadm now supports managing the bootstrap tokens active in your cluster + using the new [`kubeadm token`](https://kubernetes.io/docs/admin/bootstrap-tokens/) subcommands. +- Many other bug-fixes and improvements have been contributed that make using + `kubeadm` a more pleasant and reliable experience. + ### Breaking Changes +- kubeadm creates RBAC-enabled clusters. This means that some add-ons which + worked previously won't work without having their YAML configuration updated. + For example, to use Weave Net with a Kubernetes 1.6 cluster created by + kubeadm 1.6, you must use the + [latest YAML](https://www.weave.works/documentation/net-latest-kube-addon/). ## Cluster Ops ### New Features From 777a5c465cfba392efd3ea647a3d09334f367656 Mon Sep 17 00:00:00 2001 From: Yu-Ju Hong Date: Fri, 17 Mar 2017 15:32:56 -0700 Subject: [PATCH 11/30] Update release-notes-draft.md --- release-1.6/release-notes-draft.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index 1407475daad..6d525dcf022 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -99,9 +99,11 @@ ## Node ### New Features ### Notable Changes -- The Docker-CRI implementation is Beta and is enabled by default in kubelet. You can disable it by --enable-cri=false. +- The Docker-CRI implementation is Beta and is enabled by default in kubelet. You can disable it by --enable-cri=false. See [notes on the new implementation]( https://github.com/kubernetes/community/blob/master/contributors/devel/container-runtime-interface.md#kubernetes-v16-release-docker-cri-integration-beta) for more details. - kubelet launches pods in a new cgroup hierarchy to better enforce quality of service. Operators must drain all pods from their nodes prior to upgrade of the kubelet. They must ensure the configured cgroup driver matches their associated container runtime cgroup driver. ### Breaking Changes +- The Docker-CRI implementation is not compatible with containers created by older Kubelets. It is recommended to drain your node before upgrade. If you choose to perform an in-place upgrade, the Kubelet will automatically restart all Kubernetes-managed containers on the node. +- The Docker-CRI implementation is not compatible with CNI plugins that do not conform to the [error handling behavior in the spec](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration-list-error-handling). The plugins are being updated to resolve this issue ([#43014](https://github.com/kubernetes/kubernetes/issues/43014)). ## On Prem ### New Features From 5c866efbc2c95588cecaafc792a58b6cbe844b1c Mon Sep 17 00:00:00 2001 From: Yu-Ju Hong Date: Fri, 17 Mar 2017 15:48:22 -0700 Subject: [PATCH 12/30] Update release-notes-draft.md --- release-1.6/release-notes-draft.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index 6d525dcf022..61f58def3f8 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -102,8 +102,6 @@ - The Docker-CRI implementation is Beta and is enabled by default in kubelet. You can disable it by --enable-cri=false. See [notes on the new implementation]( https://github.com/kubernetes/community/blob/master/contributors/devel/container-runtime-interface.md#kubernetes-v16-release-docker-cri-integration-beta) for more details. - kubelet launches pods in a new cgroup hierarchy to better enforce quality of service. Operators must drain all pods from their nodes prior to upgrade of the kubelet. They must ensure the configured cgroup driver matches their associated container runtime cgroup driver. ### Breaking Changes -- The Docker-CRI implementation is not compatible with containers created by older Kubelets. It is recommended to drain your node before upgrade. If you choose to perform an in-place upgrade, the Kubelet will automatically restart all Kubernetes-managed containers on the node. -- The Docker-CRI implementation is not compatible with CNI plugins that do not conform to the [error handling behavior in the spec](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration-list-error-handling). The plugins are being updated to resolve this issue ([#43014](https://github.com/kubernetes/kubernetes/issues/43014)). ## On Prem ### New Features @@ -239,6 +237,8 @@ The features described above are now specified using fields rather than annotati * `federation/deploy/deploy.sh` was an interim solution introduced in Kubernetes v1.4 to simplify the federation control plane deployment experience. Now that we have `kubefed`, we are deprecating `deploy.sh` scripts. ([#38902](https://github.com/kubernetes/kubernetes/pull/38902), [@madhusudancs](https://github.com/madhusudancs)) * Cluster federation servers have changed the location in etcd where federated services are stored, so existing federated services must be deleted and recreated. Before upgrading, export all federated services from the federation server and delete the services. After upgrading the cluster, recreate the federated services from the exported data. ([#37770](https://github.com/kubernetes/kubernetes/pull/37770), [@enj](https://github.com/enj)) * etcd2: watching from 0 returns all initial states as ADDED events ([#38079](https://github.com/kubernetes/kubernetes/pull/38079), [@hongchaodeng](https://github.com/hongchaodeng)) +* Kubelet: The Docker-CRI implementation is not compatible with containers created by older Kubelets. It is recommended to drain your node before upgrade. If you choose to perform an in-place upgrade, the Kubelet will automatically restart all Kubernetes-managed containers on the node. +- Kubelet: The Docker-CRI implementation is not compatible with CNI plugins that do not conform to the [error handling behavior in the spec](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration-list-error-handling). The plugins are being updated to resolve this issue ([#43014](https://github.com/kubernetes/kubernetes/issues/43014)). You can disable CRI explicity (`--enable-cri=false`) as a temporary workaround. ### Other notable changes From ff7d5a8ee67a9184db6f7f0c62cdd80c7d120084 Mon Sep 17 00:00:00 2001 From: Yu-Ju Hong Date: Fri, 17 Mar 2017 15:48:57 -0700 Subject: [PATCH 13/30] Update release-notes-draft.md --- release-1.6/release-notes-draft.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index 61f58def3f8..e0fca94fc6f 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -238,7 +238,7 @@ The features described above are now specified using fields rather than annotati * Cluster federation servers have changed the location in etcd where federated services are stored, so existing federated services must be deleted and recreated. Before upgrading, export all federated services from the federation server and delete the services. After upgrading the cluster, recreate the federated services from the exported data. ([#37770](https://github.com/kubernetes/kubernetes/pull/37770), [@enj](https://github.com/enj)) * etcd2: watching from 0 returns all initial states as ADDED events ([#38079](https://github.com/kubernetes/kubernetes/pull/38079), [@hongchaodeng](https://github.com/hongchaodeng)) * Kubelet: The Docker-CRI implementation is not compatible with containers created by older Kubelets. It is recommended to drain your node before upgrade. If you choose to perform an in-place upgrade, the Kubelet will automatically restart all Kubernetes-managed containers on the node. -- Kubelet: The Docker-CRI implementation is not compatible with CNI plugins that do not conform to the [error handling behavior in the spec](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration-list-error-handling). The plugins are being updated to resolve this issue ([#43014](https://github.com/kubernetes/kubernetes/issues/43014)). You can disable CRI explicity (`--enable-cri=false`) as a temporary workaround. +* Kubelet: The Docker-CRI implementation is not compatible with CNI plugins that do not conform to the [error handling behavior in the spec](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration-list-error-handling). The plugins are being updated to resolve this issue ([#43014](https://github.com/kubernetes/kubernetes/issues/43014)). You can disable CRI explicity (`--enable-cri=false`) as a temporary workaround. ### Other notable changes From e9ccfa873222b1ea2bf85f96630080c27a7e31c9 Mon Sep 17 00:00:00 2001 From: Luke Marsden Date: Mon, 20 Mar 2017 16:02:13 +0000 Subject: [PATCH 14/30] Address review feedback --- release-1.6/release-notes-draft.md | 50 +++++++++++++++++++++++------- 1 file changed, 38 insertions(+), 12 deletions(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index 87b1b7a39f5..cedbc1fbdef 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -59,32 +59,58 @@ ## Cluster Lifecycle ### New Features - kubeadm is a simple tool for creating Kubernetes clusters on existing - machines. kubeadm's command line interface is now in beta. This means that - using it is considered safe. + machines. kubeadm's command line interface is now in beta. Using it is + considered safe, although note that upgrades and HA are not yet supported. Please [try it out](https://kubernetes.io/docs/getting-started-guides/kubeadm/) and [give us feedback](https://kubernetes.io/docs/getting-started-guides/kubeadm/#feedback)! +- Debian packages are additionally built for ppc64le and s390x architectures. +- kubeadm now deploys etcd3, and no longer supports etcd2. ### Notable Changes -- kubeadm 1.6 sets up a more secure cluster by default. It uses [RBAC](https://kubernetes.io/) - to grant limited privileges to workloads running on the cluster. By default, - service accounts and users are restricted from accessing the API. They can be - granted permissions +- kubeadm 1.6 now sets up a more secure cluster by default. It uses + [RBAC](https://kubernetes.io/) to grant limited privileges to workloads + running on the cluster. By default, service accounts and users are + restricted from accessing the API. They can be granted permissions [using the RBAC command-line utilities](https://kubernetes.io/docs/admin/authorization/rbac/#command-line-utilities). -- kubeadm now uses a new [bootstrap mechanism](https://kubernetes.io/docs/admin/bootstrap-tokens/) +- Added experimental self-hosting support (only available via + [config file](https://kubernetes.io/docs/admin/kubeadm/) since it's not + reliable yet). This is a step towards making upgrades easier with a planned + `kubeadm upgrade` command. +- Added an experimental command for running only one of kubeadm's functions, + for instance `kubeadm alpha phase certs selfsign` generates all necessary + certificates for a cluster. You can use these building block functions in + your own, custom deployment. In future versions, this command will mature and + be moved to beta quality. +- kubeadm now uses a new [bootstrap and discovery mechanism](https://kubernetes.io/docs/admin/bootstrap-tokens/) + ([proposal](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/bootstrap-discovery.md)) which is baked into the Kubernetes API server, rather than a separate - service. The API for TLS bootstrapping is now baked in to the Kubernetes API - server also. + service. - kubeadm now supports managing the bootstrap tokens active in your cluster using the new [`kubeadm token`](https://kubernetes.io/docs/admin/bootstrap-tokens/) subcommands. - Many other bug-fixes and improvements have been contributed that make using `kubeadm` a more pleasant and reliable experience. ### Breaking Changes +- A new label and taint is used for marking the master. The label is + `node-role.kubernetes.io/master=""` and the taint has the effect + `NoSchedule`. If you want to schedule a workload one the master (a networking + DaemonSet for example), you must tolerate the + `node-role.kubernetes.io/master="":NoSchedule` taint +- The kubelet API is now secured, only cluster admins are allowed to access it. +- Insecure access to the API Server over `localhost:8080` is disabled by default. +- The control plane components now talk securely to each other. The API + Server talks securely to the kubelets in the cluster. - kubeadm creates RBAC-enabled clusters. This means that some add-ons which worked previously won't work without having their YAML configuration updated. - For example, to use Weave Net with a Kubernetes 1.6 cluster created by - kubeadm 1.6, you must use the - [latest YAML](https://www.weave.works/documentation/net-latest-kube-addon/). + Please see each vendor's own documentation to check that the add-ons you are + using will work with Kubernetes 1.6. +- The kube-discovery Deployment isn't used anymore when creating a kubeadm + cluster and shouldn't be used anywhere else either due to its insecure nature. +- The Certificates API has graduated from alpha to beta. This is a + backwards-incompatible change since the alpha support is dropped, and + therefore kubeadm v1.5 and v1.6 don't work together (for example kubeadm v1.5 + can't join a kubeadm v1.6 cluster). +- Supporting only etcd3, with 3.0.14 as the minimum version. ## Cluster Ops ### New Features From 7ef3b3da1d1d0fc688d5fe6a546b2eaa7184c8ff Mon Sep 17 00:00:00 2001 From: Saad Ali Date: Mon, 20 Mar 2017 10:52:27 -0700 Subject: [PATCH 15/30] Rearrange/reword storage release notes for 1.6 --- release-1.6/release-notes-draft.md | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index e0fca94fc6f..0d9566e3196 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -171,18 +171,21 @@ The features described above are now specified using fields rather than annotati ## Storage ### New Features +- Volume plugin for ScaleIO enabling pods to seamlessly access and use data stored on Dell EMC ScaleIO volumes. +- Volume plugin for Portworx added capability to use [Portworx](http://www.portworx.com) as a storage provider for Kubernetes clusters. Portworx pools server capacity and turns servers or cloud instances into converged, highly available compute and storage nodes. +- Volume plugin capable of projecting secrets, configmaps, and downward API items into the same directory. +- Added support for mount options in persistent volumes. +- Added ability to populate environment variables from a configmap or secret. +- Add support to use NFSv3, NFSv4, and GlusterFS on GCE/GKE GCI image based clusters. +- Support for user-written/run dynamic PV provisioners. See github.com/kubernetes-incubator/external-storage for a golang library and examples. +- StorageClass API is promoted to v1 (storage.k8s.io/v1). ### Notable Changes -- A new volume driver capable of projecting secrets, configmaps, and downward API items into the same directory. -- Flex volume plugin is updated to support attach/detach interfaces. It broke backward compatibility. Please update your drivers and implement the new callouts. -- Added support for mount options in persistent volumes -- StorageClass API is promoted to v1 (storage.k8s.io/v1) -- Default storage classes are deployed during installation on Azure, AWS, GCE, OpenStack and vSphere -- Populate environment variables from a configmap or secret. -- Support for user-written/run dynamic PV provisioners. See github.com/kubernetes-incubator/external-storage for a golang library and examples -- ScaleIO Kubernetes Volume Plugin added enabling pods to seamlessly access and use data stored on ScaleIO volumes. -- Portworx Volume Plugin added capability to use [Portworx](http://www.portworx.com) as a storage provider for Kubernetes clusters. Portworx pools your servers capacity and turns your servers or cloud instances into converged, highly available compute and storage nodes. -- Add support to use NFSv3, NFSv4, and GlusterFS on GCI image cluster +- StorageClass pre-installed and set as default on Azure, AWS, GCE, OpenStack, and vSphere. + - This is something to pay close attention to if you’ve been using Kubernetes for a while, because it changes the default behavior of PersistentVolumeClaim objects on these clouds. + - Marking a StorageClass as default makes it so that even a PersistentVolumeClaim without a StorageClass specified will trigger dynamic provisioning (instead of binding to an existing pool of PVs). + - If you depend on the old behavior of volumes binding to existing pool of PersistentVolume objects then modify the StorageClass object and set `storageclass.beta.kubernetes.io/is-default-class` to `false`. ### Breaking Changes +- Flex volume plugin is updated to support attach/detach interfaces. This broke backward compatibility. Please update Flex drivers to ensure they implement the new callouts. ## Testing ### New Features From 03d42c26ca586f2e13fdf8b2a72154ab1bbb1d1e Mon Sep 17 00:00:00 2001 From: caleb miles Date: Mon, 20 Mar 2017 12:21:50 -0700 Subject: [PATCH 16/30] use link to point to external storage examples github.com/kubernetes-incubator/external-storage --- release-1.6/release-notes-draft.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index 0d9566e3196..576c20c8560 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -177,7 +177,7 @@ The features described above are now specified using fields rather than annotati - Added support for mount options in persistent volumes. - Added ability to populate environment variables from a configmap or secret. - Add support to use NFSv3, NFSv4, and GlusterFS on GCE/GKE GCI image based clusters. -- Support for user-written/run dynamic PV provisioners. See github.com/kubernetes-incubator/external-storage for a golang library and examples. +- Support for user-written/run dynamic PV provisioners. The Kubernetes Incubator contains [a golang library and examples](https://github.com/kubernetes-incubator/external-storage). - StorageClass API is promoted to v1 (storage.k8s.io/v1). ### Notable Changes - StorageClass pre-installed and set as default on Azure, AWS, GCE, OpenStack, and vSphere. From 6dea8b0fe37b89b782c282ada11cb8879596fac6 Mon Sep 17 00:00:00 2001 From: Janet Kuo Date: Mon, 20 Mar 2017 15:52:34 -0700 Subject: [PATCH 17/30] Update release note draft 1. Move DaemonSet update from notable changes to new feature 2. Add a known breaking issue for kubectl edit deployment --- release-1.6/release-notes-draft.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index e0fca94fc6f..aaad7f1855e 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -17,8 +17,8 @@ ### New Features - Introduce apps/v1beta1.Deployments resource with modified defaults compared to extensions/v1beta1.Deployments. ([#39683](https://github.com/kubernetes/kubernetes/pull/39683), [@soltysh](https://github.com/soltysh)) - Introduce new generator for apps/v1beta1 deployments ([#42362](https://github.com/kubernetes/kubernetes/pull/42362), [@soltysh](https://github.com/soltysh)) +- Introduce the rolling update feature for DaemonSet. See [Performing a Rolling Update on a DaemonSet](https://deploy-preview-2878--kubernetes-io-master-staging.netlify.com/docs/tasks/manage-daemon/update-daemon-set/). ### Notable Changes -- Implement the update feature for DaemonSet. - Deployments that cannot make progress in rolling out the newest version will now indicate via the API they are blocked - kubectl: respect deployment strategy parameters for rollout status ([#41809](https://github.com/kubernetes/kubernetes/pull/41809), [@kargakis](https://github.com/kargakis)) - kubectl logs allows getting logs directly from deployment, job and statefulset ([#40927](https://github.com/kubernetes/kubernetes/pull/40927), [@soltysh](https://github.com/soltysh)) @@ -29,6 +29,7 @@ - ReplicaSet has onwer ref of the Deployment that created it ([#35676](https://github.com/kubernetes/kubernetes/pull/35676), [@krmayankk](https://github.com/krmayankk)) ### Breaking Changes - Remove extensions/v1beta1 Jobs resource, and job/v1beta1 generator. ([#38614](https://github.com/kubernetes/kubernetes/pull/38614), [@soltysh](https://github.com/soltysh)) +- 1.5 kubectl can't do `kubectl edit deployment` on 1.6 server ([#42392](https://github.com/kubernetes/kubernetes/issues/42392)). Current workaround is to either upgrade to 1.6 kubectl, or run `kubectl edit deployment.extensions` instead of `kubectl edit deployment`. ## Auth ### New Features From 49f7840b8b8a7c643be12137f6c7c9627c0ebc4e Mon Sep 17 00:00:00 2001 From: Bowei Du Date: Tue, 21 Mar 2017 13:51:55 -0700 Subject: [PATCH 18/30] Update release notes with networking features --- release-1.6/release-notes-draft.md | 32 ++++++++++++++++++------------ 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index aaad7f1855e..61f465c2335 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -94,7 +94,26 @@ ## Network ### New Features +* Adds support for configurable DNS stub domains and upstream nameservers to kube-dns. + The following configuration options have been added to the `kube-system:kube-dns` ConfigMap: + ``` + "stubDomains": { + "acme.local": ["1.2.3.4"] + }, + ``` + is a map of domain to list of nameservers for the domain. This is used + to inject private DNS domains into the kube-dns namespace. In the above + example, any DNS requests for *.acme.local will be served by the + nameserver 1.2.3.4. + ``` + "upstreamNameservers": ["8.8.8.8", "8.8.4.4"] + ``` + is a list of upstreamNameservers to use, overriding the configuration + specified in /etc/resolv.conf. + ### Notable Changes +* An empty `kube-system:kube-dns` will be created for the cluster if one did not already exist. + ### Breaking Changes ## Node @@ -256,19 +275,6 @@ The features described above are now specified using fields rather than annotati * restored normalization of custom `--etcd-prefix` when `--storage-backend` is set to etcd3 ([#42506](https://github.com/kubernetes/kubernetes/pull/42506), [@liggitt](https://github.com/liggitt)) * kubelet created cgroups follow lowercase naming conventions ([#42497](https://github.com/kubernetes/kubernetes/pull/42497), [@derekwaynecarr](https://github.com/derekwaynecarr)) * Support whitespace in command path for gcp auth plugin ([#41653](https://github.com/kubernetes/kubernetes/pull/41653), [@jlowdermilk](https://github.com/jlowdermilk)) -* Updates the dnsmasq cache/mux layer to be managed by dnsmasq-nanny. ([#41826](https://github.com/kubernetes/kubernetes/pull/41826), [@bowei](https://github.com/bowei)) - * dnsmasq-nanny manages dnsmasq based on values from the - * kube-system:kube-dns configmap: - * "stubDomains": { - * "acme.local": ["1.2.3.4"] - * }, - * is a map of domain to list of nameservers for the domain. This is used - * to inject private DNS domains into the kube-dns namespace. In the above - * example, any DNS requests for *.acme.local will be served by the - * nameserver 1.2.3.4. - * "upstreamNameservers": ["8.8.8.8", "8.8.4.4"] - * is a list of upstreamNameservers to use, overriding the configuration - * specified in /etc/resolv.conf. * kubelet exports metrics for cgroup management ([#41988](https://github.com/kubernetes/kubernetes/pull/41988), [@sjenning](https://github.com/sjenning)) * Remove cmd/kube-discovery from the tree since it's not necessary anymore ([#42070](https://github.com/kubernetes/kubernetes/pull/42070), [@luxas](https://github.com/luxas)) * kubeadm: Hook up kubeadm against the BootstrapSigner ([#41417](https://github.com/kubernetes/kubernetes/pull/41417), [@luxas](https://github.com/luxas)) From e61e35c3a8d932a9b5795ff12119b691d4268833 Mon Sep 17 00:00:00 2001 From: Bowei Du Date: Tue, 21 Mar 2017 13:53:43 -0700 Subject: [PATCH 19/30] Update release-notes-draft.md --- release-1.6/release-notes-draft.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index 61f465c2335..9c3fcb81769 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -112,7 +112,7 @@ specified in /etc/resolv.conf. ### Notable Changes -* An empty `kube-system:kube-dns` will be created for the cluster if one did not already exist. +* An empty `kube-system:kube-dns` ConfigMap will be created for the cluster if one did not already exist. ### Breaking Changes From 821533aa453fd480bb116ec68d17cc2b3c3e843b Mon Sep 17 00:00:00 2001 From: Yu-Ju Hong Date: Wed, 22 Mar 2017 09:38:31 -0700 Subject: [PATCH 20/30] Update release-notes-draft.md --- release-1.6/release-notes-draft.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index aaad7f1855e..b32cada7272 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -239,7 +239,7 @@ The features described above are now specified using fields rather than annotati * Cluster federation servers have changed the location in etcd where federated services are stored, so existing federated services must be deleted and recreated. Before upgrading, export all federated services from the federation server and delete the services. After upgrading the cluster, recreate the federated services from the exported data. ([#37770](https://github.com/kubernetes/kubernetes/pull/37770), [@enj](https://github.com/enj)) * etcd2: watching from 0 returns all initial states as ADDED events ([#38079](https://github.com/kubernetes/kubernetes/pull/38079), [@hongchaodeng](https://github.com/hongchaodeng)) * Kubelet: The Docker-CRI implementation is not compatible with containers created by older Kubelets. It is recommended to drain your node before upgrade. If you choose to perform an in-place upgrade, the Kubelet will automatically restart all Kubernetes-managed containers on the node. -* Kubelet: The Docker-CRI implementation is not compatible with CNI plugins that do not conform to the [error handling behavior in the spec](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration-list-error-handling). The plugins are being updated to resolve this issue ([#43014](https://github.com/kubernetes/kubernetes/issues/43014)). You can disable CRI explicity (`--enable-cri=false`) as a temporary workaround. +* Kubelet: The Docker-CRI implementation is not compatible with CNI plugins that do not conform to the [error handling behavior in the spec](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration-list-error-handling). The plugins are being updated to resolve this issue ([#43488](https://github.com/kubernetes/kubernetes/issues/43488)). You can disable CRI explicity (`--enable-cri=false`) as a temporary workaround. ### Other notable changes From c3325034188627a4e2f1a3a7a46c467d8469d78a Mon Sep 17 00:00:00 2001 From: Luke Marsden Date: Wed, 22 Mar 2017 20:12:29 +0000 Subject: [PATCH 21/30] Address some review comments --- release-1.6/release-notes-draft.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index 4ad65bcafe5..4566f3e5584 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -60,7 +60,9 @@ ## Cluster Lifecycle ### New Features - kubeadm is a simple tool for creating Kubernetes clusters on existing - machines. kubeadm's command line interface is now in beta. Using it is + machines. kubeadm's command line interface is now in beta. Other parts of + kubeadm, including subcommands under `kubeadm alpha`, are + [still in alpha](https://kubernetes.io/docs/admin/kubeadm/). Using it is considered safe, although note that upgrades and HA are not yet supported. Please [try it out](https://kubernetes.io/docs/getting-started-guides/kubeadm/) and [give us feedback](https://kubernetes.io/docs/getting-started-guides/kubeadm/#feedback)! @@ -69,15 +71,17 @@ ### Notable Changes - kubeadm 1.6 now sets up a more secure cluster by default. It uses - [RBAC](https://kubernetes.io/) to grant limited privileges to workloads - running on the cluster. By default, service accounts and users are - restricted from accessing the API. They can be granted permissions + [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/) to grant limited + privileges to workloads running on the cluster. By default, service accounts + and users are restricted from accessing the API. They can be granted + permissions [using the RBAC command-line utilities](https://kubernetes.io/docs/admin/authorization/rbac/#command-line-utilities). - Added experimental self-hosting support (only available via [config file](https://kubernetes.io/docs/admin/kubeadm/) since it's not reliable yet). This is a step towards making upgrades easier with a planned `kubeadm upgrade` command. -- Added an experimental command for running only one of kubeadm's functions, +- Added an experimental command for running only one of + [kubeadm's phases](https://github.com/kubernetes/kubeadm/pull/156), for instance `kubeadm alpha phase certs selfsign` generates all necessary certificates for a cluster. You can use these building block functions in your own, custom deployment. In future versions, this command will mature and From 7ac82a37b99521ddf26e044c4ce2d5b22ea3efc3 Mon Sep 17 00:00:00 2001 From: Joe Beda Date: Wed, 22 Mar 2017 14:36:51 -0700 Subject: [PATCH 22/30] Tweak kubeadm release notes Signed-off-by: Joe Beda --- release-1.6/release-notes-draft.md | 28 +++++++++++++++++++++------- 1 file changed, 21 insertions(+), 7 deletions(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index 4566f3e5584..c557bcb5966 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -68,6 +68,9 @@ [give us feedback](https://kubernetes.io/docs/getting-started-guides/kubeadm/#feedback)! - Debian packages are additionally built for ppc64le and s390x architectures. - kubeadm now deploys etcd3, and no longer supports etcd2. +- kubeadm now supports managing tokens, including time based expiration, after + the cluster is launched. See [kubeadm reference + docs](https://kubernetes.io/docs/admin/kubeadm/#manage-tokens) for details. ### Notable Changes - kubeadm 1.6 now sets up a more secure cluster by default. It uses @@ -90,19 +93,25 @@ ([proposal](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/bootstrap-discovery.md)) which is baked into the Kubernetes API server, rather than a separate service. -- kubeadm now supports managing the bootstrap tokens active in your cluster - using the new [`kubeadm token`](https://kubernetes.io/docs/admin/bootstrap-tokens/) subcommands. - Many other bug-fixes and improvements have been contributed that make using `kubeadm` a more pleasant and reliable experience. +- The config file (still in alpha) now supports passing arbitrary flags to the + control plane components. ### Breaking Changes +- Quite a few flags been renamed or removed. Those options that are removed as + flags can still be accessed via the config file. Most noteably this includes + external etcd settings and the option for setting the cloud provider on the + API server. The [kubeadm reference + documentation](https://kubernetes.io/docs/admin/kubeadm/) is up to date with + the new flags. - A new label and taint is used for marking the master. The label is `node-role.kubernetes.io/master=""` and the taint has the effect `NoSchedule`. If you want to schedule a workload one the master (a networking DaemonSet for example), you must tolerate the `node-role.kubernetes.io/master="":NoSchedule` taint - The kubelet API is now secured, only cluster admins are allowed to access it. -- Insecure access to the API Server over `localhost:8080` is disabled by default. +- Insecure access to the API Server over `localhost:8080` is now disabled. - The control plane components now talk securely to each other. The API Server talks securely to the kubelets in the cluster. - kubeadm creates RBAC-enabled clusters. This means that some add-ons which @@ -116,6 +125,11 @@ therefore kubeadm v1.5 and v1.6 don't work together (for example kubeadm v1.5 can't join a kubeadm v1.6 cluster). - Supporting only etcd3, with 3.0.14 as the minimum version. +- `kubeadm reset` will no longer drain nodes automatically. This is because the + credentials on nodes do not have permission to perform this operation. We + have documented an [alternate + procedure](https://kubernetes.io/docs/getting-started-guides/kubeadm/#tear-down), + driven from the API server/master. ## Cluster Ops ### New Features @@ -161,7 +175,7 @@ ``` is a list of upstreamNameservers to use, overriding the configuration specified in /etc/resolv.conf. - + ### Notable Changes * An empty `kube-system:kube-dns` ConfigMap will be created for the cluster if one did not already exist. @@ -200,7 +214,7 @@ ### Notable Changes -- The [multiple schedulers](https://kubernetes.io/docs/admin/multiple-schedulers/) feature is now in Beta. This feature allows you to run multiple schedulers in parallel, each responsible for different sets of pods. When using multiple schedulers, the scheduler name is now specified in a new-in-1.6 `schedulerName` field of the PodSpec rather than using the `scheduler.alpha.kubernetes.io/name` annotation on the Pod. When you upgrade to 1.6, the Kubernetes default scheduler will start using the `schedulerName` field of the PodSpec and will ignore the annotation. +- The [multiple schedulers](https://kubernetes.io/docs/admin/multiple-schedulers/) feature is now in Beta. This feature allows you to run multiple schedulers in parallel, each responsible for different sets of pods. When using multiple schedulers, the scheduler name is now specified in a new-in-1.6 `schedulerName` field of the PodSpec rather than using the `scheduler.alpha.kubernetes.io/name` annotation on the Pod. When you upgrade to 1.6, the Kubernetes default scheduler will start using the `schedulerName` field of the PodSpec and will ignore the annotation. - [Node affinity/anti-affinity](https://kubernetes.io/docs/user-guide/node-selection/) and [pod affinity/anti-affinity](https://kubernetes.io/docs/user-guide/node-selection/) are now in Beta. Node affinity/anti-affinity allow you to specify rules for restricting which node(s) a pod can schedule onto, based on the labels on the node. Pod affinity/anti-affinity allow you to specify rules for spreading and packing pods relative to one another, across arbitrary topologies (node, zone, etc.) These affinity rules are now be specified in a new-in-1.6 `affinity` field of the PodSpec. Kubernetes 1.6 continues to support the alpha `scheduler.alpha.kubernetes.io/affinity` annotation on the Pod if you explicitly enable the alpha feature "AffinityInAnnotations", but it will be removed in a future release. When you upgrade to 1.6, if you have not enabled the alpha feature, then the scheduler will use the `affinity` field in PodSpec and will ignore the annotation. If you have enabled the alpha feature, then the scheduler will use the `affinity` field in PodSpec if it is present, and otherwise will use the annotation. @@ -212,7 +226,7 @@ The features described above are now specified using fields rather than annotati - **Multiple schedulers** - Modify your PodSpecs that currently use the `scheduler.alpha.kubernetes.io/name` annotation on Pod, to instead use the `schedulerName` field in the PodSpec. - - Modify any custom scheduler(s) you have written so that they read the `schedulerName` field on Pod instead of the `scheduler.alpha.kubernetes.io/name` annotation. + - Modify any custom scheduler(s) you have written so that they read the `schedulerName` field on Pod instead of the `scheduler.alpha.kubernetes.io/name` annotation. - Note that you can only start using the `schedulerName` field **after** you upgrade to 1.6; it is not recognized in 1.5. - **Node affinity/anti-affinity and pod affinity/anti-affinity** @@ -222,7 +236,7 @@ The features described above are now specified using fields rather than annotati - The `--failure-domains` scheduler command line-argument is not supported in the beta vesion of the feature. - **Taints** - - You will need to use `kubectl taint` to re-create all of your taints after kubectl and the master are upgraded to 1.6. Between the time the master is upgraded to 1.6 and when you do this, your existing taints will have no effect. + - You will need to use `kubectl taint` to re-create all of your taints after kubectl and the master are upgraded to 1.6. Between the time the master is upgraded to 1.6 and when you do this, your existing taints will have no effect. - You can find out what taints you have in place on a node while you are still running Kubernetes 1.5 by doing `kubectl describe node `; the `Taints` section will show the taints you have in place. To see the taints that were created under 1.5 when you are running 1.6, do `kubectl get node -o yaml` and look for the "Annotation" section with the annotation key `scheduler.alpha.kubernetes.io/taints` - You can remove the "old" taints (stored internally as annotations) at any time after the upgrade by doing `kubectl annotate nodes scheduler.alpha.kubernetes.io/taints-` (note the minus at the end, which means "delete the taint with this key") From caa90fe54dc244088c7e30ab985af7556d6153a9 Mon Sep 17 00:00:00 2001 From: Joe Beda Date: Wed, 22 Mar 2017 14:39:13 -0700 Subject: [PATCH 23/30] Revert "Tweak kubeadm release notes" This reverts commit 7ac82a37b99521ddf26e044c4ce2d5b22ea3efc3. --- release-1.6/release-notes-draft.md | 28 +++++++--------------------- 1 file changed, 7 insertions(+), 21 deletions(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index c557bcb5966..4566f3e5584 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -68,9 +68,6 @@ [give us feedback](https://kubernetes.io/docs/getting-started-guides/kubeadm/#feedback)! - Debian packages are additionally built for ppc64le and s390x architectures. - kubeadm now deploys etcd3, and no longer supports etcd2. -- kubeadm now supports managing tokens, including time based expiration, after - the cluster is launched. See [kubeadm reference - docs](https://kubernetes.io/docs/admin/kubeadm/#manage-tokens) for details. ### Notable Changes - kubeadm 1.6 now sets up a more secure cluster by default. It uses @@ -93,25 +90,19 @@ ([proposal](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/bootstrap-discovery.md)) which is baked into the Kubernetes API server, rather than a separate service. +- kubeadm now supports managing the bootstrap tokens active in your cluster + using the new [`kubeadm token`](https://kubernetes.io/docs/admin/bootstrap-tokens/) subcommands. - Many other bug-fixes and improvements have been contributed that make using `kubeadm` a more pleasant and reliable experience. -- The config file (still in alpha) now supports passing arbitrary flags to the - control plane components. ### Breaking Changes -- Quite a few flags been renamed or removed. Those options that are removed as - flags can still be accessed via the config file. Most noteably this includes - external etcd settings and the option for setting the cloud provider on the - API server. The [kubeadm reference - documentation](https://kubernetes.io/docs/admin/kubeadm/) is up to date with - the new flags. - A new label and taint is used for marking the master. The label is `node-role.kubernetes.io/master=""` and the taint has the effect `NoSchedule`. If you want to schedule a workload one the master (a networking DaemonSet for example), you must tolerate the `node-role.kubernetes.io/master="":NoSchedule` taint - The kubelet API is now secured, only cluster admins are allowed to access it. -- Insecure access to the API Server over `localhost:8080` is now disabled. +- Insecure access to the API Server over `localhost:8080` is disabled by default. - The control plane components now talk securely to each other. The API Server talks securely to the kubelets in the cluster. - kubeadm creates RBAC-enabled clusters. This means that some add-ons which @@ -125,11 +116,6 @@ therefore kubeadm v1.5 and v1.6 don't work together (for example kubeadm v1.5 can't join a kubeadm v1.6 cluster). - Supporting only etcd3, with 3.0.14 as the minimum version. -- `kubeadm reset` will no longer drain nodes automatically. This is because the - credentials on nodes do not have permission to perform this operation. We - have documented an [alternate - procedure](https://kubernetes.io/docs/getting-started-guides/kubeadm/#tear-down), - driven from the API server/master. ## Cluster Ops ### New Features @@ -175,7 +161,7 @@ ``` is a list of upstreamNameservers to use, overriding the configuration specified in /etc/resolv.conf. - + ### Notable Changes * An empty `kube-system:kube-dns` ConfigMap will be created for the cluster if one did not already exist. @@ -214,7 +200,7 @@ ### Notable Changes -- The [multiple schedulers](https://kubernetes.io/docs/admin/multiple-schedulers/) feature is now in Beta. This feature allows you to run multiple schedulers in parallel, each responsible for different sets of pods. When using multiple schedulers, the scheduler name is now specified in a new-in-1.6 `schedulerName` field of the PodSpec rather than using the `scheduler.alpha.kubernetes.io/name` annotation on the Pod. When you upgrade to 1.6, the Kubernetes default scheduler will start using the `schedulerName` field of the PodSpec and will ignore the annotation. +- The [multiple schedulers](https://kubernetes.io/docs/admin/multiple-schedulers/) feature is now in Beta. This feature allows you to run multiple schedulers in parallel, each responsible for different sets of pods. When using multiple schedulers, the scheduler name is now specified in a new-in-1.6 `schedulerName` field of the PodSpec rather than using the `scheduler.alpha.kubernetes.io/name` annotation on the Pod. When you upgrade to 1.6, the Kubernetes default scheduler will start using the `schedulerName` field of the PodSpec and will ignore the annotation. - [Node affinity/anti-affinity](https://kubernetes.io/docs/user-guide/node-selection/) and [pod affinity/anti-affinity](https://kubernetes.io/docs/user-guide/node-selection/) are now in Beta. Node affinity/anti-affinity allow you to specify rules for restricting which node(s) a pod can schedule onto, based on the labels on the node. Pod affinity/anti-affinity allow you to specify rules for spreading and packing pods relative to one another, across arbitrary topologies (node, zone, etc.) These affinity rules are now be specified in a new-in-1.6 `affinity` field of the PodSpec. Kubernetes 1.6 continues to support the alpha `scheduler.alpha.kubernetes.io/affinity` annotation on the Pod if you explicitly enable the alpha feature "AffinityInAnnotations", but it will be removed in a future release. When you upgrade to 1.6, if you have not enabled the alpha feature, then the scheduler will use the `affinity` field in PodSpec and will ignore the annotation. If you have enabled the alpha feature, then the scheduler will use the `affinity` field in PodSpec if it is present, and otherwise will use the annotation. @@ -226,7 +212,7 @@ The features described above are now specified using fields rather than annotati - **Multiple schedulers** - Modify your PodSpecs that currently use the `scheduler.alpha.kubernetes.io/name` annotation on Pod, to instead use the `schedulerName` field in the PodSpec. - - Modify any custom scheduler(s) you have written so that they read the `schedulerName` field on Pod instead of the `scheduler.alpha.kubernetes.io/name` annotation. + - Modify any custom scheduler(s) you have written so that they read the `schedulerName` field on Pod instead of the `scheduler.alpha.kubernetes.io/name` annotation. - Note that you can only start using the `schedulerName` field **after** you upgrade to 1.6; it is not recognized in 1.5. - **Node affinity/anti-affinity and pod affinity/anti-affinity** @@ -236,7 +222,7 @@ The features described above are now specified using fields rather than annotati - The `--failure-domains` scheduler command line-argument is not supported in the beta vesion of the feature. - **Taints** - - You will need to use `kubectl taint` to re-create all of your taints after kubectl and the master are upgraded to 1.6. Between the time the master is upgraded to 1.6 and when you do this, your existing taints will have no effect. + - You will need to use `kubectl taint` to re-create all of your taints after kubectl and the master are upgraded to 1.6. Between the time the master is upgraded to 1.6 and when you do this, your existing taints will have no effect. - You can find out what taints you have in place on a node while you are still running Kubernetes 1.5 by doing `kubectl describe node `; the `Taints` section will show the taints you have in place. To see the taints that were created under 1.5 when you are running 1.6, do `kubectl get node -o yaml` and look for the "Annotation" section with the annotation key `scheduler.alpha.kubernetes.io/taints` - You can remove the "old" taints (stored internally as annotations) at any time after the upgrade by doing `kubectl annotate nodes scheduler.alpha.kubernetes.io/taints-` (note the minus at the end, which means "delete the taint with this key") From a0a96c5b2bfe310050984fefb672adcdbe3694f9 Mon Sep 17 00:00:00 2001 From: Joe Beda Date: Wed, 22 Mar 2017 14:42:30 -0700 Subject: [PATCH 24/30] Tweak kubeadm release notes Signed-off-by: Joe Beda --- release-1.6/release-notes-draft.md | 28 +++++++++++++++++++++------- 1 file changed, 21 insertions(+), 7 deletions(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index 4566f3e5584..c557bcb5966 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -68,6 +68,9 @@ [give us feedback](https://kubernetes.io/docs/getting-started-guides/kubeadm/#feedback)! - Debian packages are additionally built for ppc64le and s390x architectures. - kubeadm now deploys etcd3, and no longer supports etcd2. +- kubeadm now supports managing tokens, including time based expiration, after + the cluster is launched. See [kubeadm reference + docs](https://kubernetes.io/docs/admin/kubeadm/#manage-tokens) for details. ### Notable Changes - kubeadm 1.6 now sets up a more secure cluster by default. It uses @@ -90,19 +93,25 @@ ([proposal](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/bootstrap-discovery.md)) which is baked into the Kubernetes API server, rather than a separate service. -- kubeadm now supports managing the bootstrap tokens active in your cluster - using the new [`kubeadm token`](https://kubernetes.io/docs/admin/bootstrap-tokens/) subcommands. - Many other bug-fixes and improvements have been contributed that make using `kubeadm` a more pleasant and reliable experience. +- The config file (still in alpha) now supports passing arbitrary flags to the + control plane components. ### Breaking Changes +- Quite a few flags been renamed or removed. Those options that are removed as + flags can still be accessed via the config file. Most noteably this includes + external etcd settings and the option for setting the cloud provider on the + API server. The [kubeadm reference + documentation](https://kubernetes.io/docs/admin/kubeadm/) is up to date with + the new flags. - A new label and taint is used for marking the master. The label is `node-role.kubernetes.io/master=""` and the taint has the effect `NoSchedule`. If you want to schedule a workload one the master (a networking DaemonSet for example), you must tolerate the `node-role.kubernetes.io/master="":NoSchedule` taint - The kubelet API is now secured, only cluster admins are allowed to access it. -- Insecure access to the API Server over `localhost:8080` is disabled by default. +- Insecure access to the API Server over `localhost:8080` is now disabled. - The control plane components now talk securely to each other. The API Server talks securely to the kubelets in the cluster. - kubeadm creates RBAC-enabled clusters. This means that some add-ons which @@ -116,6 +125,11 @@ therefore kubeadm v1.5 and v1.6 don't work together (for example kubeadm v1.5 can't join a kubeadm v1.6 cluster). - Supporting only etcd3, with 3.0.14 as the minimum version. +- `kubeadm reset` will no longer drain nodes automatically. This is because the + credentials on nodes do not have permission to perform this operation. We + have documented an [alternate + procedure](https://kubernetes.io/docs/getting-started-guides/kubeadm/#tear-down), + driven from the API server/master. ## Cluster Ops ### New Features @@ -161,7 +175,7 @@ ``` is a list of upstreamNameservers to use, overriding the configuration specified in /etc/resolv.conf. - + ### Notable Changes * An empty `kube-system:kube-dns` ConfigMap will be created for the cluster if one did not already exist. @@ -200,7 +214,7 @@ ### Notable Changes -- The [multiple schedulers](https://kubernetes.io/docs/admin/multiple-schedulers/) feature is now in Beta. This feature allows you to run multiple schedulers in parallel, each responsible for different sets of pods. When using multiple schedulers, the scheduler name is now specified in a new-in-1.6 `schedulerName` field of the PodSpec rather than using the `scheduler.alpha.kubernetes.io/name` annotation on the Pod. When you upgrade to 1.6, the Kubernetes default scheduler will start using the `schedulerName` field of the PodSpec and will ignore the annotation. +- The [multiple schedulers](https://kubernetes.io/docs/admin/multiple-schedulers/) feature is now in Beta. This feature allows you to run multiple schedulers in parallel, each responsible for different sets of pods. When using multiple schedulers, the scheduler name is now specified in a new-in-1.6 `schedulerName` field of the PodSpec rather than using the `scheduler.alpha.kubernetes.io/name` annotation on the Pod. When you upgrade to 1.6, the Kubernetes default scheduler will start using the `schedulerName` field of the PodSpec and will ignore the annotation. - [Node affinity/anti-affinity](https://kubernetes.io/docs/user-guide/node-selection/) and [pod affinity/anti-affinity](https://kubernetes.io/docs/user-guide/node-selection/) are now in Beta. Node affinity/anti-affinity allow you to specify rules for restricting which node(s) a pod can schedule onto, based on the labels on the node. Pod affinity/anti-affinity allow you to specify rules for spreading and packing pods relative to one another, across arbitrary topologies (node, zone, etc.) These affinity rules are now be specified in a new-in-1.6 `affinity` field of the PodSpec. Kubernetes 1.6 continues to support the alpha `scheduler.alpha.kubernetes.io/affinity` annotation on the Pod if you explicitly enable the alpha feature "AffinityInAnnotations", but it will be removed in a future release. When you upgrade to 1.6, if you have not enabled the alpha feature, then the scheduler will use the `affinity` field in PodSpec and will ignore the annotation. If you have enabled the alpha feature, then the scheduler will use the `affinity` field in PodSpec if it is present, and otherwise will use the annotation. @@ -212,7 +226,7 @@ The features described above are now specified using fields rather than annotati - **Multiple schedulers** - Modify your PodSpecs that currently use the `scheduler.alpha.kubernetes.io/name` annotation on Pod, to instead use the `schedulerName` field in the PodSpec. - - Modify any custom scheduler(s) you have written so that they read the `schedulerName` field on Pod instead of the `scheduler.alpha.kubernetes.io/name` annotation. + - Modify any custom scheduler(s) you have written so that they read the `schedulerName` field on Pod instead of the `scheduler.alpha.kubernetes.io/name` annotation. - Note that you can only start using the `schedulerName` field **after** you upgrade to 1.6; it is not recognized in 1.5. - **Node affinity/anti-affinity and pod affinity/anti-affinity** @@ -222,7 +236,7 @@ The features described above are now specified using fields rather than annotati - The `--failure-domains` scheduler command line-argument is not supported in the beta vesion of the feature. - **Taints** - - You will need to use `kubectl taint` to re-create all of your taints after kubectl and the master are upgraded to 1.6. Between the time the master is upgraded to 1.6 and when you do this, your existing taints will have no effect. + - You will need to use `kubectl taint` to re-create all of your taints after kubectl and the master are upgraded to 1.6. Between the time the master is upgraded to 1.6 and when you do this, your existing taints will have no effect. - You can find out what taints you have in place on a node while you are still running Kubernetes 1.5 by doing `kubectl describe node `; the `Taints` section will show the taints you have in place. To see the taints that were created under 1.5 when you are running 1.6, do `kubectl get node -o yaml` and look for the "Annotation" section with the annotation key `scheduler.alpha.kubernetes.io/taints` - You can remove the "old" taints (stored internally as annotations) at any time after the upgrade by doing `kubectl annotate nodes scheduler.alpha.kubernetes.io/taints-` (note the minus at the end, which means "delete the taint with this key") From 1296f36da9a2d5423e0da7a6b17c8cfe8946aff9 Mon Sep 17 00:00:00 2001 From: Bowei Du Date: Wed, 22 Mar 2017 17:22:56 -0700 Subject: [PATCH 25/30] Update with CNI + CRI compatibility caveat --- release-1.6/release-notes-draft.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index c557bcb5966..0c5c16d9308 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -177,6 +177,15 @@ specified in /etc/resolv.conf. ### Notable Changes +* Container Network Interface ([CNI](https://github.com/containernetworking/cni)) integration with Container + Runtime Interface (CRI) is enabled by default. + * The standard `bridge` plugin have been validated to interoperate with the new CRI + CNI code path. + * If you are using plugins other than `bridge`, make sure you have updated custom plugins to the + latest version that is compatible. + * If you encounter any issues involving CNI plugins, CRI can be disabled with setting the `--enable-cri` + flag to `false`. + * [Associated action required notes](#CNIcompat). + * An empty `kube-system:kube-dns` ConfigMap will be created for the cluster if one did not already exist. ### Breaking Changes @@ -326,7 +335,7 @@ The features described above are now specified using fields rather than annotati * Cluster federation servers have changed the location in etcd where federated services are stored, so existing federated services must be deleted and recreated. Before upgrading, export all federated services from the federation server and delete the services. After upgrading the cluster, recreate the federated services from the exported data. ([#37770](https://github.com/kubernetes/kubernetes/pull/37770), [@enj](https://github.com/enj)) * etcd2: watching from 0 returns all initial states as ADDED events ([#38079](https://github.com/kubernetes/kubernetes/pull/38079), [@hongchaodeng](https://github.com/hongchaodeng)) * Kubelet: The Docker-CRI implementation is not compatible with containers created by older Kubelets. It is recommended to drain your node before upgrade. If you choose to perform an in-place upgrade, the Kubelet will automatically restart all Kubernetes-managed containers on the node. -* Kubelet: The Docker-CRI implementation is not compatible with CNI plugins that do not conform to the [error handling behavior in the spec](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration-list-error-handling). The plugins are being updated to resolve this issue ([#43488](https://github.com/kubernetes/kubernetes/issues/43488)). You can disable CRI explicity (`--enable-cri=false`) as a temporary workaround. +* Kubelet: The Docker-CRI implementation is not compatible with CNI plugins that do not conform to the [error handling behavior in the spec](https://github.com/containernetworking/cni/blob/master/SPEC.md#network-configuration-list-error-handling). The plugins are being updated to resolve this issue ([#43488](https://github.com/kubernetes/kubernetes/issues/43488)). You can disable CRI explicity (`--enable-cri=false`) as a temporary workaround. See [this section](#CNIcompatdetails) for more details. ### Other notable changes From a0faa4d825a8c258065a520ff58feec9cfe893f3 Mon Sep 17 00:00:00 2001 From: Vish Kannan Date: Wed, 22 Mar 2017 17:27:22 -0700 Subject: [PATCH 26/30] Mention eviction thresholds being part of node allocatable --- release-1.6/release-notes-draft.md | 1 + 1 file changed, 1 insertion(+) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index c557bcb5966..86efc7e9eda 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -186,6 +186,7 @@ ### Notable Changes - The Docker-CRI implementation is Beta and is enabled by default in kubelet. You can disable it by --enable-cri=false. See [notes on the new implementation]( https://github.com/kubernetes/community/blob/master/contributors/devel/container-runtime-interface.md#kubernetes-v16-release-docker-cri-integration-beta) for more details. - kubelet launches pods in a new cgroup hierarchy to better enforce quality of service. Operators must drain all pods from their nodes prior to upgrade of the kubelet. They must ensure the configured cgroup driver matches their associated container runtime cgroup driver. +- Hard eviction thresholds will be subtracted from the Allocatable capacity on nodes to improve node reliability. This *may* break existing clusters since the overall schedulable capacity would reduce after upgrading to v1.6. You can opt-out of this feature by specifying --experimental-allocatable-ignore-eviction=true. ### Breaking Changes ## On Prem From d97afd012b338043f566f8ed88999e9ba6f9cd65 Mon Sep 17 00:00:00 2001 From: ymqytw Date: Wed, 22 Mar 2017 16:42:09 -0700 Subject: [PATCH 27/30] update CLI section in release note --- release-1.6/release-notes-draft.md | 83 ++++++++++++++++++++++++++++-- 1 file changed, 78 insertions(+), 5 deletions(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index c557bcb5966..f87524a53ae 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -50,13 +50,86 @@ ### Breaking Changes ## CLI -### New Features -- Add new command "kubectl set selector" ([#38966](https://github.com/kubernetes/kubernetes/pull/38966), [@kargakis](https://github.com/kargakis)) -- Add kubectl create poddisruptionbudget command ([#36646](https://github.com/kubernetes/kubernetes/pull/36646), [@kargakis](https://github.com/kargakis)) -### Notable Changes -- `kubectl edit` now edits objects exactly as they were retrieved from the API. This allows using `kubectl edit` with third-party resources and extension API servers. Because client-side conversion is no longer done, the `--output-version` option is deprecated for `kubectl edit`. To edit using a particular API version, fully-qualify the resource, version, and group used to fetch the object (for example, `job.v1.batch/myjob`) ([#41304](https://github.com/kubernetes/kubernetes/pull/41304), [@liggitt](https://github.com/liggitt)) + ### Breaking Changes +**Action required** +- Running `kubectl taint` (alpha in 1.5) against a 1.6 server requires upgrading kubectl to version 1.6 +- Running `kubectl taint` (alpha in 1.5) against a 1.5 server requires a kubectl version of 1.5 +- `create secret` no longer accepts passing multiple values to a single --from-literal flag using comma separation + - Update command invocations to pass separate --from-literal flags for each value + +### New Commands +- `apply set-last-applied` *updates the applied-applied-configuration annotation* ([#41694](https://github.com/kubernetes/kubernetes/pull/41694), [@shiywang](https://github.com/shiywang)) +- `kubectl apply view-last-applied` *viewing the last configuration file applied* ([#41146](https://github.com/kubernetes/kubernetes/pull/41146), [@shiywang](https://github.com/shiywang)) +Create subcommands + - `create poddisruptionbudget` ([#36646](https://github.com/kubernetes/kubernetes/pull/36646), [@kargakis](https://github.com/kargakis)) + - `create clusterrole` ([#41538](https://github.com/kubernetes/kubernetes/pull/41538), [@xingzhou](https://github.com/xingzhou)) + - `create role` ([#39852](https://github.com/kubernetes/kubernetes/pull/39852), [@xingzhou](https://github.com/xingzhou)) + - `create clusterrolebinding` ([#37098](https://github.com/kubernetes/kubernetes/pull/37098), [@deads2k](https://github.com/deads2k)) + - `create service externalname` ([#34789](https://github.com/kubernetes/kubernetes/pull/34789), [@adohe](https://github.com/adohe)) +- `set selector` - update selector labels ([#38966](https://github.com/kubernetes/kubernetes/pull/38966), [@kargakis](https://github.com/kargakis)) +- `can-i` to see if you can perform an action ([#41077](https://github.com/kubernetes/kubernetes/pull/41077), [@deads2k](https://github.com/deads2k)) + +### Updates to existing commands +- Printing and output + - Import a numeric ordering sorting library and use it in the sorting printer. ([#40746](https://github.com/kubernetes/kubernetes/pull/40746), [@matthyx](https://github.com/matthyx)) + - DaemonSet get and describe show status fields. ([#42843](https://github.com/kubernetes/kubernetes/pull/42843), [@janetkuo](https://github.com/janetkuo)) + - Pods describe shows tolerationSeconds ([#42162](https://github.com/kubernetes/kubernetes/pull/42162), [@kevin-wangzefeng](https://github.com/kevin-wangzefeng)) + - Node describe contains closing paren ([#39217](https://github.com/kubernetes/kubernetes/pull/39217), [@luksa](https://github.com/luksa)) + - Display pod node selectors with kubectl describe. ([#36396](https://github.com/kubernetes/kubernetes/pull/36396), [@aveshagarwal](https://github.com/aveshagarwal)) + - Add Version to the resource printer for 'get nodes' ([#37943](https://github.com/kubernetes/kubernetes/pull/37943), [@ailusazh](https://github.com/ailusazh)) + - Added support for printing in all supported `--output` formats to `kubectl create ...` and `kubectl apply ...` ([#38112](https://github.com/kubernetes/kubernetes/pull/38112), [@juanvallejo](https://github.com/juanvallejo)) + - Add three more columns to `kubectl get deploy -o wide` output. ([#39240](https://github.com/kubernetes/kubernetes/pull/39240), [@xingzhou](https://github.com/xingzhou)) + - Fix kubectl get -f -o so it prints all items in the file ([#39038](https://github.com/kubernetes/kubernetes/pull/39038), [@ncdc](https://github.com/ncdc)) + - kubectl describe no longer prints the last-applied-configuration annotation for secrets. ([#34664](https://github.com/kubernetes/kubernetes/pull/34664), [@ymqytw](https://github.com/ymqytw)) + - Completed pods should not be hidden when requested by name via `kubectl get`. ([#42216](https://github.com/kubernetes/kubernetes/pull/42216), [@smarterclayton](https://github.com/smarterclayton)) + - Improve formatting of EventSource in kubectl get and kubectl describe ([#40073](https://github.com/kubernetes/kubernetes/pull/40073), [@matthyx](https://github.com/matthyx)) +- Attach now supports multiple types ([#40365](https://github.com/kubernetes/kubernetes/pull/40365), [@shiywang](https://github.com/shiywang)) +- Create now accepts the label selector flag for filtering objects to create ([#40057](https://github.com/kubernetes/kubernetes/pull/40057), [@MrHohn](https://github.com/MrHohn)) +- Top now accepts short forms for "node" and "pod" ("no", "po") ([#39218](https://github.com/kubernetes/kubernetes/pull/39218), [@luksa](https://github.com/luksa)) +- Begin paths for internationalization in kubectl ([#36802](https://github.com/kubernetes/kubernetes/pull/36802), [@brendandburns](https://github.com/brendandburns)) + - Add initial french translations for kubectl ([#40645](https://github.com/kubernetes/kubernetes/pull/40645), [@brendandburns](https://github.com/brendandburns)) + +### Updates to apply +- New command `apply set-last-applied` *updates the applied-applied-configuration annotation* ([#41694](https://github.com/kubernetes/kubernetes/pull/41694), [@shiywang](https://github.com/shiywang)) +- New command `apply view-last-applied` *command for viewing the last configuration file applied* ([#41146](https://github.com/kubernetes/kubernetes/pull/41146), [@shiywang](https://github.com/shiywang)) +- `apply` now supports explicitly clearing values by setting them to null ([#40630](https://github.com/kubernetes/kubernetes/pull/40630), [@liggitt](https://github.com/liggitt)) +- Warn user when using `apply` on a resource lacking the `LastAppliedConfig` annotation ([#36672](https://github.com/kubernetes/kubernetes/pull/36672), [@ymqytw](https://github.com/ymqytw)) +- PATCH (i.e. apply and edit) now supports merging lists of primitives ([#38665](https://github.com/kubernetes/kubernetes/pull/38665), [@ymqytw](https://github.com/ymqytw)) +- `apply` falls back to generic 3-way JSON merge patch for Third Party Resource or unregistered types ([#40666](https://github.com/kubernetes/kubernetes/pull/40666), [@ymqytw](https://github.com/ymqytw)) + +### Updates to edit +- `edit` now supports Third party resources and extension API servers. ([#41304](https://github.com/kubernetes/kubernetes/pull/41304), [@liggitt](https://github.com/liggitt)) + - Now to edit a particular API version, provide the fully-qualify the resource, version, and group used to fetch the object (for example, `job.v1.batch/myjob`) + - Client-side conversion is no longer done, and the `--output-version` option is deprecated for `kubectl edit`. +- `edit` works as intended with apply and will not change the annotation + - No longer updates the last-applied-configuration annotation when --save-config is unspecified or false. ([#41924](https://github.com/kubernetes/kubernetes/pull/41924), [@ymqytw](https://github.com/ymqytw)) + - Fixes issue that caused apply to revert changes made by edit + +### Notable Changes +- The api server will publish the extensions/Deployments API as preferred over the apps/Deployment (introduced in 1.6). ([#43553](https://github.com/kubernetes/kubernetes/pull/43553), [@liggitt](https://github.com/liggitt)) + - This will ensure certain commands in 1.5 versions of kubectl continue function when run against a 1.6 server. (e.g. `kubectl edit deployment`) +- Taint + - The `taint` command will not function in a skewed 1.5 / 1.6 environment - client and server versions must match (See Action required section above) + - Change taints/tolerations to api fields ([#38957](https://github.com/kubernetes/kubernetes/pull/38957), [@aveshagarwal](https://github.com/aveshagarwal)) + - Make kubectl taint command respect effect NoExecute ([#42120](https://github.com/kubernetes/kubernetes/pull/42120), [@kevin-wangzefeng](https://github.com/kevin-wangzefeng)) +- Allow drain --force to remove pods whose managing resource is deleted. ([#41864](https://github.com/kubernetes/kubernetes/pull/41864), [@marun](https://github.com/marun)) +- `--output-version` is ignored for all commands except `kubectl convert`. This is consistent with the generic nature of `kubectl` CRUD commands and the previous removal of `--api-version`. Specific versions can be specified in the resource field: `resource.version.group`, `jobs.v1.batch`. ([#41576](https://github.com/kubernetes/kubernetes/pull/41576), [@deads2k](https://github.com/deads2k)) +- Allow missing keys in templates by default ([#39486](https://github.com/kubernetes/kubernetes/pull/39486), [@ncdc](https://github.com/ncdc)) +- Add error message when trying to use clusterrole with namespace in kubectl ([#36424](https://github.com/kubernetes/kubernetes/pull/36424), [@xilabao](https://github.com/xilabao)) +- When deleting an object with `--grace-period=0`, the client will begin a graceful deletion and wait until the resource is fully deleted. To force deletion, use the `--force` flag. ([#37263](https://github.com/kubernetes/kubernetes/pull/37263), [@smarterclayton](https://github.com/smarterclayton)) + +### Bug fixes +- Fixed --save-config in create subcommand to save the annotation ([#40289](https://github.com/kubernetes/kubernetes/pull/40289), [@xilabao](https://github.com/xilabao)) +- Fixed an issue where 'kubectl get --sort-by=' would return an error if the specified fields in sort were not specified in one or more of the returned objects. ([#40541](https://github.com/kubernetes/kubernetes/pull/40541), [@fabianofranz](https://github.com/fabianofranz)) + - Previously this would cause the command to fail regardless of whether or not the field was present in the object model + - Now the command will succeed even if the sort-by field is missing from one or more of the objects +- Fixed issue with kubectl proxy so it will now proxy an empty path - e.g. http://localhost:8001 ([#39226](https://github.com/kubernetes/kubernetes/pull/39226), [@luksa](https://github.com/luksa)) +- Fixed an issue where commas were not accepted in --from-literal flags for the creation of secrets. ([#35191](https://github.com/kubernetes/kubernetes/pull/35191), [@SamiHiltunen](https://github.com/SamiHiltunen)) + - Passing multiple values separated by a comma in a single --from-literal flag is no longer supported. Please use multiple --from-literal flags to provide multiple values. +- Fixed a bug where the --server, --token, and --certificate-authority flags were not overriding the related in-cluster configs when provided in a `kubectl` call inside a cluster. ([#39006](https://github.com/kubernetes/kubernetes/pull/39006), [@fabianofranz](https://github.com/fabianofranz)) + ## Cluster Lifecycle ### New Features - kubeadm is a simple tool for creating Kubernetes clusters on existing From 0e3befb5a7c0a24262fb50c80fe5b14a8b8dce6c Mon Sep 17 00:00:00 2001 From: Janet Kuo Date: Thu, 23 Mar 2017 16:46:04 -0700 Subject: [PATCH 28/30] Remove the breaking change that's fixed https://github.com/kubernetes/kubernetes/issues/42392 is fixed --- release-1.6/release-notes-draft.md | 1 - 1 file changed, 1 deletion(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index 51e3fa13801..055a6c3be99 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -29,7 +29,6 @@ - ReplicaSet has onwer ref of the Deployment that created it ([#35676](https://github.com/kubernetes/kubernetes/pull/35676), [@krmayankk](https://github.com/krmayankk)) ### Breaking Changes - Remove extensions/v1beta1 Jobs resource, and job/v1beta1 generator. ([#38614](https://github.com/kubernetes/kubernetes/pull/38614), [@soltysh](https://github.com/soltysh)) -- 1.5 kubectl can't do `kubectl edit deployment` on 1.6 server ([#42392](https://github.com/kubernetes/kubernetes/issues/42392)). Current workaround is to either upgrade to 1.6 kubectl, or run `kubectl edit deployment.extensions` instead of `kubectl edit deployment`. ## Auth ### New Features From 4a8edeaeddd0747655b87e3e242be921ee58bb3b Mon Sep 17 00:00:00 2001 From: liyao Date: Fri, 24 Mar 2017 09:43:19 +0800 Subject: [PATCH 29/30] add issue link and author to HPA features add issue link and author to HPA features --- release-1.6/release-notes-draft.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index 2bf1b0a0d99..376be3b6430 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -39,8 +39,8 @@ ## Autoscaling ### New Features ### Notable Changes -- The Horizontal Pod Autoscaler now supports drawing metrics throug the API server aggregator. -- The Horizontal Pod Autoscaler now supports scaling on multiple, custom metrics. +- The Horizontal Pod Autoscaler now supports drawing metrics throug the API server aggregator.([#118](https://github.com/kubernetes/features/issues/118), [@DirectXMan12](https://github.com/DirectXMan12)) +- The Horizontal Pod Autoscaler now supports scaling on multiple, custom metrics.([#117](https://github.com/kubernetes/features/issues/117),[@DirectXMan12](https://github.com/DirectXMan12)) ### Breaking Changes ## Big Data From b53e6662f09a3e761b00fcd6fdce3036edf48b87 Mon Sep 17 00:00:00 2001 From: liyao Date: Fri, 24 Mar 2017 10:07:59 +0800 Subject: [PATCH 30/30] Update release-notes-draft.md --- release-1.6/release-notes-draft.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release-1.6/release-notes-draft.md b/release-1.6/release-notes-draft.md index 376be3b6430..1cbefb8a371 100644 --- a/release-1.6/release-notes-draft.md +++ b/release-1.6/release-notes-draft.md @@ -39,7 +39,7 @@ ## Autoscaling ### New Features ### Notable Changes -- The Horizontal Pod Autoscaler now supports drawing metrics throug the API server aggregator.([#118](https://github.com/kubernetes/features/issues/118), [@DirectXMan12](https://github.com/DirectXMan12)) +- The Horizontal Pod Autoscaler now supports drawing metrics through the API server aggregator.([#118](https://github.com/kubernetes/features/issues/118), [@DirectXMan12](https://github.com/DirectXMan12)) - The Horizontal Pod Autoscaler now supports scaling on multiple, custom metrics.([#117](https://github.com/kubernetes/features/issues/117),[@DirectXMan12](https://github.com/DirectXMan12)) ### Breaking Changes