Add extended scheduler to operator #145
Conversation
| @@ -13,6 +13,7 @@ spec: | |||
| timezone: {{ .Values.timezone | default "UTC" }} | |||
| services: | |||
| {{ toYaml .Values.services | indent 4 }} | |||
| schedulerName: {{ .Values.schedulerName | default "default-scheduler" }} | |||
There was a problem hiding this comment.
why add schedulerName on here
There was a problem hiding this comment.
This schedulerName used at here: https://github.com/weekface/tidb-operator/blob/449176ebcaa90b37be63156eb72a27fee5266a01/pkg/manager/member/pd_member_manager.go#L513
| continue | ||
| } | ||
|
|
||
| nodeMap[nodeName] = append(nodeMap[nodeName], podName1) |
There was a problem hiding this comment.
if the nodeName is not contain in nodeMap, there will happen panic ,we should add logic here
if nodeMap[nodeName] ==nil{
continue
}
| } | ||
|
|
||
| var min int | ||
| var minInitialized bool |
There was a problem hiding this comment.
the minInitialized is not used,what is the minInitialized mean
* address comment
…into scheduler
| command: | ||
| {{- if .Values.scheduler.kubeSchedulerImage }} | ||
| - kube-scheduler | ||
| {{- else }} |
There was a problem hiding this comment.
hyperkube image also has kube-scheduler in $PATH, so no need to set the command separately. So it's unnecessary to define two variables for scheduler image.
| {{- end }} | ||
| - --port=10261 | ||
| - --leader-elect=true | ||
| - --lock-object-name=tidb-scheduler |
There was a problem hiding this comment.
The lock should be customizable and I think it should be the same as schedulerName.
| - --lock-object-namespace={{ .Release.Namespace }} | ||
| - --scheduler-name={{ .Values.scheduler.schedulerName }} | ||
| - --v={{ .Values.scheduler.logLevel }} | ||
| - --policy-configmap=tidb-scheduler-policy |
| helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }} | ||
| data: | ||
| policy.cfg: |- | ||
| { |
There was a problem hiding this comment.
This JSON doesn't need to be templated, so using .Files.Get makes it more easy to maintain.
There was a problem hiding this comment.
OK, waiting for: #149 merged.
charts/tidb-operator/values.yaml
Outdated
| requests: | ||
| cpu: 80m | ||
| memory: 50Mi | ||
| # pd replicas |
There was a problem hiding this comment.
The replicas are cluster specific, it should not be defined in the scheduler. And since PD is single raft cluster, its replica is the same as its member count. But TiKV replica is configured in PD configuration, this can be retrieved by PD API.
However, for the first version, use 3 for both PD and TiKV is enough.
pkg/scheduler/scheduler.go
Outdated
| glog.Infof("scheduling pod: %s/%s", ns, podName) | ||
| var err error | ||
| for _, predicate := range s.predicates { | ||
| glog.Infof("entering predicate: %s, nodes: %v", predicate.Name(), getNodeNames(kubeNodes)) |
There was a problem hiding this comment.
This log should have a higher log level since it's much more verbose.
pkg/scheduler/scheduler.go
Outdated
| } | ||
|
|
||
| // Filter selects a set of nodes from *schedulerapiv1.ExtenderArgs.Nodes when this is a pd or tikv pod | ||
| // else return the original nodes. |
There was a problem hiding this comment.
otherwise, returns the original nodes.
pkg/scheduler/scheduler.go
Outdated
| }, nil | ||
| } | ||
|
|
||
| // We didn't pass `prioritizeVerb` to kubernetes scheduler extender's config file, this method will not be called. |
pkg/scheduler/predicates/ha.go
Outdated
| var component string | ||
| var exist bool | ||
| if component, exist = pod.Labels[label.ComponentLabelKey]; !exist { | ||
| return nil, fmt.Errorf("can't find component in pod labels: %s/%s", ns, podName) |
pkg/scheduler/predicates/ha.go
Outdated
| } | ||
|
|
||
| if ordinal < replicas && min != 0 { | ||
| return nil, fmt.Errorf("the first %d pods can't scheduled to the same node", replicas) |
…into scheduler
…into scheduler
|
/run-e2e-tests |
3 similar comments
|
/run-e2e-tests |
|
/run-e2e-tests |
|
/run-e2e-tests |
|
/run-e2e-tests |
|
/run-e2e-tests |
|
/run-e2e-tests |
| pdReplicas: 3 | ||
| # tikv replicas | ||
| tikvReplicas: 3 | ||
| hyperkubeImage: quay.io/coreos/hyperkube:v1.10.4_coreos.0 |
|
/run-e2e-tests |
|
/run-e2e-tests |
1 similar comment
|
/run-e2e-tests |
* scheduler: add extended scheduler to operator
* README: update doc version and branch info * readme: update format of branch description and add note * minor update * readme: update description Co-authored-by: Lilian Lee <lilin@pingcap.com>
This PR adds an extended scheduler to TiDB Operator, and fixes #138:
@tennix @onlymellb @xiaojingchen @gregwebs PTAL