karmada v1.3.5 release

Changes since v1.3.4

Bug Fixes

• karmada-search: support pod subresource (attach, exec, port-forward) through global proxy. (#3116, @XiShanYongYe-Chang)
• karmada-search: Fixed the problem that ResourceVersion base64 encrypted repeatedly when starting multiple informers to watch resource. (#3391, @niuyueyang1996)
• karmada-search: Fixed http response add content-type header. (#3515, @callmeoldprince)
• karmada-search: Fiexed paging list in karmada search proxy in large-scale member clusters. (#3451, @ikaven1024)
• karmada-controller-manager: Fixed the issue that RBAC resources whose name contains uppercase characters can not be propagated. (#3214, @whitewindmills)
• karmada-controller-manager: Fixed the problem that ServiceAccount can be interpreted as a dependency, but may not be delivered. (#3222, @whitewindmills)
• karmadactl: Fixed unable to view the options of karmadactl addons enable/disable issue. (#3307, @lonelyCZ)

karmada v1.5.0 release

What's New

Multiple Scheduling Groups

Users are now able to declare multiple groups of clusters to both PropagationPolicy and ClusterPropagationPolicy by leveraging the newly introduced ClusterAffinities field. The scheduler will evaluate these groups one by one in the order they appear in the specification until it finds the one that satisfies scheduling restrictions.

This feature allows the Karmada scheduler to first schedule applications to lower-cost clusters or migrate applications from a primary cluster to backup clusters in the case of cluster failure.

See multiple scheduling group proposal for more info.

(Feature contributor: @XiShanYongYe-Chang @RainbowMango)

New Way to Customize Scheduler

The default scheduler is now able to work with any number of third-party customized schedulers. Similar to Kubernetes, the workloads will be scheduled by the default scheduler if the scheduler name is not declared in PropagationPolicy or ClusterPropagationPolicy.

See customize scheduler for more details.

(Feature contributor: @Poor12)

Other Notable Changes

API Changes

• Introduced Affinities to both PropagationPolicy and ClusterPropagationPolicy. (#3105, @RainbowMango)
• Introduced Placement to the ResoureBinding/ClusterResourceBinding API. (#2702, @Poor12)
• Introduced SchedulerObservedAffinityName to both ResourceBinding and ClusterResourceBinding. (#3163, @RainbowMango)

Bug Fixes

• karmadactl: Fixed the issue that karmada-agent installed by the register command cannot delete works due to lack of permissions. (#2902, @lonelyCZ)
• karmadactl: Fixed the issue that the default ValidatingWebhookConfiguration for resourceinterpreterwebhook was not working. (#2915, @whitewindmills)
• karmadactl: Fixed the issue that the default ValidatingWebhookConfiguration for resourceinterpretercustomizations was not working. (#2916, @chaunceyjiang)
• karmadactl: Fixed the error of resources whose name contains colons failed to be created when using karmadactl apply. (#2919, @Poor12)
• karmadactl init: Granted karmada-agent permissions to access resourceinterpretercustomizations. (#2984, @jwcesign)
• karmada-controller-manager: Generated PolicyRules from given subjects for impersonation deduplicate. (#2911, @yanfeng1992)
• karmada-controller-manager/karmada-agent: Fixed the failure to sync work status due to the informer being accidentally shut down. (#2930, @Poor12)
• karmada-controller-manager/karmada-agent: Fixed misjudgment of deployment and statefuleset health status. (#2928, @Fish-pro)
• karmada-controller-manager: Fixed LabelsOverrider and AnnotationsOverrider failures to add new items in the case of null label/annotation. (#2971, @chaunceyjiang)
• karmada-controller-manager: labelsOverrider/annotationsOverrider supports composed-labels, like testannotation/projectId: . (#3037, @chaunceyjiang)
• karmada-controller-manager: Fixed the issue that RBAC resources whose name contains uppercase characters cannot be propagated. (#3201, @whitewindmills)
• karmada-agent: Check whether the resource exists before creating it. Sometimes the resource is created in advance, giving less privilege to Karmada. (#2988, @jwcesign)
• karmada-scheduler: Fixed a corner case that re-scheduling was skipped in the case that the cluster becomes not fit. (#2912, @jwcesign)
• karmada-search: Filtered out not-ready clusters. (#3010, @yanfeng1992)
• karmada-search: Avoided proxy request block when member clusters were down. (#3027, @ikaven1024)
• karmada-webhook: Validated replicaSchedulingType and replicaDivisionPreference. (#3014, @chaunceyjiang)
• karmada-webhook: Fixed the issue that the InterpretDependency operation cannot be registered. (#3052, @whitewindmills)
• karmada-search: Supported pod subresource (attach, exec, port-forward) through global proxy. (#3098, @ikaven1024)

Security

• golang.org/x/net updates to v0.6.0 to fix CVE-2022-41717. (#3048, @fengshunli)

Features & Enhancements

• karmadactl: Introduced --kube-image-tag flag to the init command to specify the Kubernetes image version. (#2840, @helen-frank)
• karmadactl: The --cluster-context flag of join command now takes current-context by default. (#2956, @helen-frank)
• karmadactl: Added edit mode for interpret commands. (#2831, @ikaven1024)
• karmadactl: Introduced --cert-validity-period for init to make the validity period of cert configurable. (#3156, @lonelyCZ)
• karmada-controller-manager: Now the OverridePolicy and ClusterOverridePolicy will be applied by implicit priority order. The one with the lower priority will be applied before the one with the higher priority. (#2609)
• karmada-controller-manager: Users are now able to apply multiple dependencies interpreter configurations. (#2884, @XiShanYongYe-Chang)
• karmada-controller-manager: Built-in interpreter supports StatefulSets. (#3009, @chaunceyjiang)
• karmada-controller-manager: Retained the labels added to resources by member clusters. (#3088, @chaunceyjiang)
• karmada-controller-manager: Default interpreter supports CronJob aggregated status. (#3129, @chaunceyjiang)
• karmada-controller-manager: Supports PodDisruptionBudget resource in default interpreter. (#2997, @a7i)
• karmada-controller-manager: Support for removing annotations/labels propagated through karmada. (#3099, @chaunceyjiang)
• karmada-webhook: Added validation for policy.spec.placement.orderedClusterAffinities. (#3164, @XiShanYongYe-Chang)
• karmada-webhook: Validated the fieldSelector of overridepolicy. (#3193, @chaunceyjiang)

Other

Dependencies

• Kubernetes images will now be pulled from registry.k8s.io instead of k8s.gcr.io, to be in alignment with current community initiatives. (#2882, @Zhuzhenghao)
• Karmada is now built with Golang 1.19.4. (#2908, @qingwave)
• Karmada is now built with Golang 1.19.5. (#3067, @yanggangtony)
• Karmada is now built with Kubernetes v1.26.1 dependencies. (#3080, @RainbowMango)
• The base image alpine now has been promoted from alpine:3.15.1 to alpine:3.17.1. (#3045, @fengshunli)

Helm Chart

• Fixed helm template missing yaml directive marker to separate API Service resources. (#2963, @a7i)
• Fixed missing karmada-search helm template strategy. (#2994, @a7i)
• Fixed karmada-agent helm template strategy indentation. (#2993, @a7i)
• Chart: karmada-search installation supports specifying an external etcd. (#3120, @my-git9)
• Chart: Supports custom labels variable for etcd. (#3138, @my-git9)

Instrumentation

• Instrumentation: Introduced the pool_get_operation_total, pool_put_operation_total metrics to karmada-controller-manager and karmada-agent. (#2883, @ikaven1024)

Contributors

Thank you to everyone who contributed to this release!

Users whose commits are in this release (alphabetically by username)

• @a7i
• @calvin0327
• @carlory
• @chaunceyjiang
• @fengshunli
• @Fish-pro
• @Garrybest
• @helen-frank
• @ikaven1024
• @jwcesign
• @lonelyCZ
• @maoyangLiu
• @my-git9
• @Poor12
• @qingwave
• @RainbowMango
• @VedRatan
• @Wang-Kai
• @whitewindmills
• @wlp1153468871
• @wongearl
• @XiShanYongYe-Chang
• @yanfeng1992
• @yanggangtony
• @Zhuzhenghao

karmada v1.2.7 release

Changes since v1.2.6

Bug Fixes

• karmada-controller-manager: Fixed the issue that RBAC resources whose name contains uppercase characters can not be propagated. (#3123, @whitewindmills)

karmada v1.3.4 release

Changes since v1.3.3

Bug Fixes

• karmada-search: filter out not ready clusters. (#3019, @yanfeng1992)
• karmada-search: avoid proxy request block when member cluster down. (#3031, @ikaven1024)
• karmada-webhook: Fixed the issue that the InterpretDependency operation can't be registered. (#3076, @XiShanYongYe-Chang)

karmada v1.2.6 release

Changes since v1.2.5

Bug Fixes

• karmada-search: filter out not ready clusters. (#3018, @yanfeng1992)
• karmada-webhook: Fixed the issue that the InterpretDependency operation can't be registered. (#3077, @XiShanYongYe-Chang)

karmada v1.4.2 release

Changes since v1.4.1

Bug Fixes

• karmada-controller-manager: Fixed LabelsOverrider and AnnotationsOverrider failed to add new items issue in case of label/annotation is nil. (#2972, @chaunceyjiang)
• karmada-controller-manager: labelsOverrider/annotationsOverrider supports composed-labels, like testannotation/projectId: . (#3047, @chaunceyjiang)
• karmadactl: Grant karmada-agent permission to access resourceinterpretercustomizations for init command. (#2986, @jwcesign)
• karmada-agent: Check if the resource exists before creating it. Sometimes the resource is created in advance, to give less privilege to Karmada. (#3002, @jwcesign)
• karmada-search: filter out not ready clusters. (#3016, @yanfeng1992)
• karmada-search: avoid proxy request block when member cluster down. (#3030, @ikaven1024)
• karmada-webhook: Fixed the issue that the InterpretDependency operation can't be registered. (#3074, @XiShanYongYe-Chang)

karmada v1.3.3 release

Changes since v1.3.2

Bug Fixes

• karmada-scheduler: Fixed a corner case that re-schedule be skipped in case of the cluster becomes not fit. (#2959, @jwcesign)
• karmada-controller-manager/karmada-agent: Fixed failed to sync work status issue due to the informer being accidentally shut down. (#2938, @Poor12)
• karmada-controller-manager/karmada-agent: Fixed misjudgment of deployment and statefuleset health status. (#2945, @Fish-pro)
• karmadactl: Fixed the error of resources whose name contains colons failing to be created when using karmadactl apply. (#2932, @Poor12)
• karmadactl: Fixed the default ValidatingWebhookConfiguration for resourceinterpreterwebhook not working issue. (#2923, @qingwave)
• karmadactl: Fixed karmada-agent installed by the register command can not delete works due to lack of permission issue. (#2905, @lonelyCZ)
• karmada-webhook: Fixed failed to set resource selector default namespace if the relevant OverridePolicy and PropagationPolicy with namespace unset issue. (#2872, @carlory)

Others

• Karmada is now built with Go1.18.9. (#2909, @XiShanYongYe-Chang)

karmada v1.4.1 release

Changes since v1.4.0

Bug Fixes

• karmadactl: Fixed karmada-agent installed by the register command can not delete works due to lack of permission issue. (#2904, @lonelyCZ)
• karmadactl: Fixed the default ValidatingWebhookConfiguration for resourceinterpreterwebhook not working issue. (#2924, @qingwave)
• karmadactl: Fixed the default ValidatingWebhookConfiguration for resourceinterpretercustomizations not working issue. (#2927, @chaunceyjiang)
• karmadactl: Fixed the error of resources whose name contains colons failing to be created when using karmadactl apply. (#2931, @Poor12)
• karmada-controller-manager/karmada-agent: Fixed misjudgment of deployment and statefuleset health status. (#2944, @Fish-pro)
• karmada-controller-manager/karmada-agent: Fixed failed to sync work status issue due to the informer being accidentally shut down. (#2937, @Poor12)
• karmada-scheduler: Fixed a corner case that re-schedule be skipped in case of the cluster becomes not fit. (#2955, @jwcesign)

Others

• Karmada is now built with Golang 1.19.4. (#2913, @qingwave)

karmada v1.2.5 release

Changes since v1.2.4

Bug Fixes

• karmada-controller-manager/karmada-agent: Fixed failed to sync work status issue due to the informer being accidentally shut down. (#2939, @Poor12)
• karmadactl: Fixed the error of resources whose name contains colons failing to be created when using karmadactl promote. (#2933, @Poor12)
• karmadactl: Fixed the default ValidatingWebhookConfiguration for resourceinterpreterwebhook not working issue. (#2922, @qingwave)
• karmada-webhook: Fixed failed to set resource selector default namespace if the relevant OverridePolicy and PropagationPolicy with namespace unset issue. (#2873, @carlory )

karmada v1.4.0 release

What's New

Declarative Resource Interpreter

The Interpreter Framework is designed for interpreting the structure of arbitrary resource types. It consists of built-in and customized interpreters, this release introduced another brand-new customized interpreter.

With the newly introduced declarative interpreter, users can quickly customize resource interpreters for both Kubernetes resources and CRD resources by the rules declared in the ResourceInterpreterCustomization API specification. Compared with the interpreter customized by the webhook, it gets the rules from the declarative specifications instead of requiring an additional webhook component.

The new command named interpret in the karmadactl could be used to test the rules before applying them to the system.
Some examples are provided to help users better understand how this interpreter can be used.

(Feature contributor: @jameszhangyukun @ikaven1024 @chaunceyjiang @XiShanYongYe-Chang @RainbowMango)

PropagationPolicy/ClusterPropagationPolicy priority

Users are now able to declare the priorities for both PropagationPolicy and ClusterPropagationPolicy. A policy will be applied for the matched resource templates if there are no other policies with higher priority at the point the resource template be processed.

The system administrator can use priorities to manage and control policies. Refer to Configure PropagationPolicy priority for more details.

(Feature contributor: @Garrybest @jwcesign)

Instrumentation improvement

This release enhanced observability significantly through metrics and events.
The metrics can be queried by the endpoint(/metrics) of each component using an HTTP scrap, and they are served in Prometheus format. Events are reported to the relevant resource objects respectively.

Refer to events and metrics for more details.

(Feature contributor: @Poor12)

Other Notable Changes

API Changes

• Introduces priority to PropagationPolicy. (#2758, @RainbowMango)
• Introduced LabelsOverrider and AnnotationsOverrider for overriding labels and annotations.(#2584, @chaunceyjiang)
• Evoluted PropagateDeps FeatureGate to Beta and enabled it by default. (#2875, @XiShanYongYe-Chang)
• Evoluted Failover/GracefulEviction FeatureGate to Beta and enabled it by default. (#2876, @jwcesign)
• Evoluted CustomizedClusterResourceModeling FeatureGate to Beta and enabled it by default. (#2877, @Poor12)

Bug Fixes

• karmada-search: Fixed concurrent map writes panic while listing objects via proxy. (#2483, @ikaven1024)
• karmada-search: Fixed the issue that the returned ResourceVersion by proxy was not stable. (#2746, @cmicat)
• karmada-controller-manager/karmada-agent: Fixed the issue that pod information cannot be collected when building resource summary. (#2489, @Poor12)
• karmada-controller-manager: Used cluster secret ref namespace in unified-auth-controller when generating a ClusterRoleBinding. (#2516, @XiShanYongYe-Chang)
• karmada-controller-manager: Fixed the failure of creating resources whose name contains colons. (#2549, @XiShanYongYe-Chang)
• karmada-controller-manager: Fixed the panic when cluster ImpersonatorSecretRef is nil. (#2675, @stingshen)
• karmada-controller-manager: Fixed serviceaccount continual regeneration by service account controller. (#2578, @Poor12)
• karmada-controller-manager: Disabled the preemption matching of pp/cpp priority. (#2734, @XiShanYongYe-Chang)
• karmada-controller-manager: Fixed the issue that clusterOverridePolicy and overridePolicy with nil resource selector could not work. (#2771, @wuyingjun-lucky)
• karmada-controlle-managerr: Ignored resources that do not match with a policy before applying the policy. (#2786, @XiShanYongYe-Chang)
• karmada-agent: Fixed the issue that ServiceExport controller cannot report endpointSlices (due to the missing create permission). (#2515, @lonelyCZ)
• karmadactl: Fixed init cannot honor IPv6 addresses when generating the kubeconfig file. (#2450, @duanmengkk)
• karmadactl: Fixed --karmada-data directory was not initialized in init command. (#2548, @jwcesign)
• karmadactl: Fixed the issue that init commands print incorrect register command. (#2707, @Songjoy)
• karmadactl: Fixed the issue that the namespace already exists. (#2505, @cleverhu)
• karmada-webhook: Fixed the failure to set resource selector default namespace when the relevant OverridePolicy and PropagationPolicy are not configured with namespaces. (#2858, @carlory)

Security

• Security: Added limitReader to io.ReadAll which could limit the memory request and avoid DoS attacks. (#2765, @Poor12)

Features & Enhancements

• karmadactl: Improved karmada init help output. (#2342, @my-git9)
• karmadactl: karmadactl prohibits input extra arguments for init command. (#2497, @helen-frank)
• karmadactl/chart: The init no longer creates redundant ServiceAccounts for components except karmada-agent. (#2523, @carlory)
• karmadactl: Fixed the issue that options of deinit cannot be shown. (#2540, @helen-frank)
• karmadactl/chart: If Karmada is installed by using karmadactl or a Helm chart with default configurations, the image tag of karmada-kube-controller-manager/karmada-apiserver will be kube-controller-manager:v1.25.2/kube-apiserver:v1.25.2. (#2539, @jwcesign)
• karmadactl: Introduced --karmada-apiserver-advertise-address flag to specify Karmada APIserver's address to the init sub-command. (#2550, @wuyingjun-lucky)
• karmadactl: Introduced --enable-cert-rotation option to register commands. (#2596, @lonelyCZ)
• karmadactl: Added dryrun for uncordon. (#2760, @helen-frank)
• karmadactl: Added validation on cluster existence for karmadactl get. (#2787, @helen-frank)
• karmadactl: Added liveness probe into the kube-controller-manager component. (#2817, @carlory)
• karmadactl: Added --image-registry flags for init. (#2655, @helen-frank)
• karmadactl: Added interpreter command for resource interpretation customizations. (#2750, @ikaven1024)
• karmadactl: Added execute mod for the interpret command. (#2824, @ikaven1024)
• karmada-search: Objects returned by proxy will have resource.karmada.io/cached-from-cluster annotation to indicate which member cluster it comes from. (#2469, @ikaven1024)
• karmada-search: Users can get the real resource request metrics while using the proxy. (#2481, @ikaven1024)
• karmada-search: Users now can use --disable-search and --disable-proxy options to disable search and proxy features (both enabled by default). (#2650, @ikaven1024)
• karmada-controller-manager: Added implicit priority for PropagationPolicy. (#2267, @Garrybest)
• karmada-controller-manager: Introduced resource label namespace.karmada.io/skip-auto-propagation: "true" for namespaces that should be skipped from auto propagation. (#2696, @jwcesign)
• karmada-controller-manager: Allowed users to update the .spec.resourceSelectors field of PropagationPolicy/ClusterPropagationPolicy. (#2562, @XiShanYongYe-Chang)
• karmada-controller-manager`: Introduced priority to PropagationPolicy. (#2767, @jwcesign)
• karmada-scheduler-estimator: Leveraged scheduler cache to estimate replicas. (#2704, @Garrybest)
• `karmada-controller...