Skip to content
This repository has been archived by the owner on May 22, 2020. It is now read-only.

Machine class #659

Closed
wants to merge 3 commits into from
Closed

Machine class #659

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2fc8ca5
Add a machine class resource.
roberthbailey Mar 19, 2018
ecdb9c6
Generated code for MachineClass.
roberthbailey Mar 23, 2018
da230c7
Add a sample machine class yaml file.
roberthbailey Mar 23, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion cluster-api/pkg/apis/cluster/v1alpha1/machine_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,24 @@ type ProviderConfig struct {
// ProviderConfigSource represents a source for the provider-specific
// node configuration.
type ProviderConfigSource struct {
// TODO(roberthbailey): Fill these in later

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it better to inline or reference the MachineClass and not ProviderConfig?
Inline:

apiVersion: cluster.k8s.io/v1alpha1
kind: Machine
metadata:
  name: machine-example
  namespace: baz
spec:
  class:
    value:
      capacity:
        cpu: 2
        memory: 7.5Gi
        storage: 20Gi
      allocatable:
        cpu: 2
        memory: 7.5Gi
        storage: 20Gi
      providerConfig:
        apiVersion: gceproviderconfig/v1alpha1
        kind: GCEProviderConfig
        project: GCLOUD_PROJECT_FOO
        zone: us-central1-f
        machineType: n1-standard-2
        image: projects/ubuntu-os-cloud/global/images/family/ubuntu-1604-lts
  versions:
    kubelet: 1.8.3
    containerRuntime:
      name: docker
      version: 1.12.0
  roles:
  - Node

Reference:

apiVersion: cluster.k8s.io/v1alpha1
kind: MachineClass
metadata:
  name: small
  namespace: default
capacity:
  cpu: 2
  memory: 7.5Gi
  storage: 20Gi
allocatable:
  cpu: 2
  memory: 7.5Gi
  storage: 20Gi
providerConfig:
  apiVersion: gceproviderconfig/v1alpha1
  kind: GCEProviderConfig
  project: GCLOUD_PROJECT_FOO
  zone: us-central1-f
  machineType: n1-standard-2
  image: projects/ubuntu-os-cloud/global/images/family/ubuntu-1604-lts
---
apiVersion: cluster.k8s.io/v1alpha1
kind: Machine
metadata:
  name: machine-example
  namespace: baz
spec:
  class:
    ref:
      name: small
      namespace: baz
  versions:
    kubelet: 1.8.3
    containerRuntime:
      name: docker
      version: 1.12.0
  roles:
  - Node

I find that more flexible.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it should be necessary to specify the capacity / allocatable on a single machine if you inline the provider config. I think of the class is a separate way to bring the provider specific data, not something inherent to the spec of the machine.

// No more than one of the following may be specified.

// The machine class from which the provider config should be sourced.
// +optional
MachineClass *MachineClassRef `json:machineClass,omitempty`
}

type MachineClassRef struct {
// The name of the MachineClass.
Name string `json:name`

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about namespace?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Storage classes aren't namespaced, and I used that same pattern here. But it's worth discussing whether they should be part of a namespace.

// Parameters allow basic substitution to be applied to
// a MachineClass (where supported).
// Keys must not be empty. The maximum number of
// parameters is 512, with a cumulative max size of 256K.
// +optional
Parameters map[string]string `json:parameters,omitempty`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What kinds of things is this meant to override? Capacity? Allocatable? Or stuff in the provider config? All of these seem like they could be deeply nested structures that a map would not be able to fully express.

If ProviderConfig, perhaps we consider json patches?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thinking was in the provider config (e.g. parameterize the zone of a class to stamp out identical machine sets in multiple zones).

}

// MachineStatus defines the observed state of Machine
Expand Down
64 changes: 64 additions & 0 deletions cluster-api/pkg/apis/cluster/v1alpha1/machineclass_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@

/*
Copyright 2018 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/


package v1alpha1

import (
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/runtime"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

// +genclient
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

// MachineClass can be used to templatize and re-use provider configuration
// across multiple Machines / MachineSets / MachineDeployments.
// +k8s:openapi-gen=true
// +resource:path=machineclasses
type MachineClass struct {
metav1.TypeMeta `json:",inline"`
// +optional
metav1.ObjectMeta `json:"metadata,omitempty"`

// The total capacity available on this machine type (cpu/memory/disk).
//
// WARNING: It is up to the creator of the MachineClass to ensure that
// this field is consistent with the underlying machine that will
// be provisioned when this class is used, to inform higher level
// automation (e.g. the cluster autoscaler).
Capacity corev1.ResourceList `json:"capacity"`

// How much capacity is actually allocatable on this machine.
// Must be equal to or less than the capacity, and when less
// indicates the resources reserved for system overhead.
//
// WARNING: It is up to the creator of the MachineClass to ensure that
// this field is consistent with the underlying machine that will
// be provisioned when this class is used, to inform higher level
// automation (e.g. the cluster autoscaler).
Allocatable corev1.ResourceList `json:"allocatable"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I understand, this is the capacity minus the amount that kubelet is going to reserve on the node. I don't think you can know what that reserve is going to be just given a machine class. I know for GKE, we vary the reserve for each release, and possibly by machine size.

Perhaps the best way to represent this would be to officially model the Kubelet reserved resources in the MachineSpec (and therefore MachineSet and MachineDeployment). If we did that, we could drop Allocatable here, and autoscalers of deployments could pick the capacity from machine class and subtract reserved from machine spec.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @krousey. There is no way to calculate in advance the allocated resources - the Machine class doesn't have a knowledge of which kubelet version / container runtime you are going to use and those can affect the kubelet's --kube-reserved and --system-reserved flags.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The intent is that this would indicate to the cluster autoscaler how much actual capacity would exist once the "node allocatable" overhead was subtracted. The fact that the overhead varies by version makes putting this variable here... difficult, since it would tightly couple a machineclass to a specific k8s version if you wanted to adhere to the warning text.

@krousey - Is your suggestion of putting reserved in the machine spec to put it next to the kubelet version, since it is tightly coupled with it?


// Provider-specific configuration to use during node creation.
ProviderConfig runtime.RawExtension `json:"providerConfig"`

// TODO: should this use an api.ObjectReference to a 'MachineTemplate' instead?
// A link to the MachineTemplate that will be used to create provider
// specific configuration for Machines of this class.
// MachineTemplate corev1.ObjectReference `json:machineTemplate`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this comment as an alternate design -- one thing we'd thought about was splitting this data across two objects (class + template) and the template could potentially be used directly by machine in addition to machine class. Not sure if that's valuable or not though, so I put this here to foster discussion.

}
40 changes: 40 additions & 0 deletions cluster-api/pkg/apis/cluster/v1alpha1/zz_generated.api.register.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,13 @@ var (
func() runtime.Object { return &MachineList{} }, // Register versioned resource list
&MachineStrategy{builders.StorageStrategySingleton},
)
clusterMachineClassStorage = builders.NewApiResource( // Resource status endpoint
cluster.InternalMachineClass,
MachineClassSchemeFns{},
func() runtime.Object { return &MachineClass{} }, // Register versioned resource
func() runtime.Object { return &MachineClassList{} }, // Register versioned resource list
&cluster.MachineClassStrategy{builders.StorageStrategySingleton},
)
clusterMachineSetStorage = builders.NewApiResource( // Resource status endpoint
cluster.InternalMachineSet,
MachineSetSchemeFns{},
Expand All @@ -63,6 +70,13 @@ var (
func() runtime.Object { return &Machine{} }, // Register versioned resource
func() runtime.Object { return &MachineList{} }, // Register versioned resource list
&MachineStatusStrategy{builders.StatusStorageStrategySingleton},
), clusterMachineClassStorage,
builders.NewApiResource( // Resource status endpoint
cluster.InternalMachineClassStatus,
MachineClassSchemeFns{},
func() runtime.Object { return &MachineClass{} }, // Register versioned resource
func() runtime.Object { return &MachineClassList{} }, // Register versioned resource list
&cluster.MachineClassStatusStrategy{builders.StatusStorageStrategySingleton},
), clusterMachineSetStorage,
builders.NewApiResource( // Resource status endpoint
cluster.InternalMachineSetStatus,
Expand Down Expand Up @@ -143,6 +157,32 @@ type MachineList struct {
Items []Machine `json:"items"`
}

//
// MachineClass Functions and Structs
//
// +k8s:deepcopy-gen=false
type MachineClassSchemeFns struct {
builders.DefaultSchemeFns
}

// +k8s:deepcopy-gen=false
type MachineClassStrategy struct {
builders.DefaultStorageStrategy
}

// +k8s:deepcopy-gen=false
type MachineClassStatusStrategy struct {
builders.DefaultStatusStorageStrategy
}

// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

type MachineClassList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty"`
Items []MachineClass `json:"items"`
}

//
// MachineSet Functions and Structs
//
Expand Down
122 changes: 122 additions & 0 deletions cluster-api/pkg/apis/cluster/v1alpha1/zz_generated.conversion.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,16 @@ func RegisterConversions(scheme *runtime.Scheme) error {
Convert_cluster_ContainerRuntimeInfo_To_v1alpha1_ContainerRuntimeInfo,
Convert_v1alpha1_Machine_To_cluster_Machine,
Convert_cluster_Machine_To_v1alpha1_Machine,
Convert_v1alpha1_MachineClass_To_cluster_MachineClass,
Convert_cluster_MachineClass_To_v1alpha1_MachineClass,
Convert_v1alpha1_MachineClassList_To_cluster_MachineClassList,
Convert_cluster_MachineClassList_To_v1alpha1_MachineClassList,
Convert_v1alpha1_MachineClassRef_To_cluster_MachineClassRef,
Convert_cluster_MachineClassRef_To_v1alpha1_MachineClassRef,
Convert_v1alpha1_MachineClassStatusStrategy_To_cluster_MachineClassStatusStrategy,
Convert_cluster_MachineClassStatusStrategy_To_v1alpha1_MachineClassStatusStrategy,
Convert_v1alpha1_MachineClassStrategy_To_cluster_MachineClassStrategy,
Convert_cluster_MachineClassStrategy_To_v1alpha1_MachineClassStrategy,
Convert_v1alpha1_MachineList_To_cluster_MachineList,
Convert_cluster_MachineList_To_v1alpha1_MachineList,
Convert_v1alpha1_MachineSet_To_cluster_MachineSet,
Expand Down Expand Up @@ -346,6 +356,116 @@ func Convert_cluster_Machine_To_v1alpha1_Machine(in *cluster.Machine, out *Machi
return autoConvert_cluster_Machine_To_v1alpha1_Machine(in, out, s)
}

func autoConvert_v1alpha1_MachineClass_To_cluster_MachineClass(in *MachineClass, out *cluster.MachineClass, s conversion.Scope) error {
out.ObjectMeta = in.ObjectMeta
out.Capacity = *(*v1.ResourceList)(unsafe.Pointer(&in.Capacity))
out.Allocatable = *(*v1.ResourceList)(unsafe.Pointer(&in.Allocatable))
out.ProviderConfig = in.ProviderConfig
return nil
}

// Convert_v1alpha1_MachineClass_To_cluster_MachineClass is an autogenerated conversion function.
func Convert_v1alpha1_MachineClass_To_cluster_MachineClass(in *MachineClass, out *cluster.MachineClass, s conversion.Scope) error {
return autoConvert_v1alpha1_MachineClass_To_cluster_MachineClass(in, out, s)
}

func autoConvert_cluster_MachineClass_To_v1alpha1_MachineClass(in *cluster.MachineClass, out *MachineClass, s conversion.Scope) error {
out.ObjectMeta = in.ObjectMeta
out.Capacity = *(*v1.ResourceList)(unsafe.Pointer(&in.Capacity))
out.Allocatable = *(*v1.ResourceList)(unsafe.Pointer(&in.Allocatable))
out.ProviderConfig = in.ProviderConfig
return nil
}

// Convert_cluster_MachineClass_To_v1alpha1_MachineClass is an autogenerated conversion function.
func Convert_cluster_MachineClass_To_v1alpha1_MachineClass(in *cluster.MachineClass, out *MachineClass, s conversion.Scope) error {
return autoConvert_cluster_MachineClass_To_v1alpha1_MachineClass(in, out, s)
}

func autoConvert_v1alpha1_MachineClassList_To_cluster_MachineClassList(in *MachineClassList, out *cluster.MachineClassList, s conversion.Scope) error {
out.ListMeta = in.ListMeta
out.Items = *(*[]cluster.MachineClass)(unsafe.Pointer(&in.Items))
return nil
}

// Convert_v1alpha1_MachineClassList_To_cluster_MachineClassList is an autogenerated conversion function.
func Convert_v1alpha1_MachineClassList_To_cluster_MachineClassList(in *MachineClassList, out *cluster.MachineClassList, s conversion.Scope) error {
return autoConvert_v1alpha1_MachineClassList_To_cluster_MachineClassList(in, out, s)
}

func autoConvert_cluster_MachineClassList_To_v1alpha1_MachineClassList(in *cluster.MachineClassList, out *MachineClassList, s conversion.Scope) error {
out.ListMeta = in.ListMeta
out.Items = *(*[]MachineClass)(unsafe.Pointer(&in.Items))
return nil
}

// Convert_cluster_MachineClassList_To_v1alpha1_MachineClassList is an autogenerated conversion function.
func Convert_cluster_MachineClassList_To_v1alpha1_MachineClassList(in *cluster.MachineClassList, out *MachineClassList, s conversion.Scope) error {
return autoConvert_cluster_MachineClassList_To_v1alpha1_MachineClassList(in, out, s)
}

func autoConvert_v1alpha1_MachineClassRef_To_cluster_MachineClassRef(in *MachineClassRef, out *cluster.MachineClassRef, s conversion.Scope) error {
out.Name = in.Name
out.Parameters = *(*map[string]string)(unsafe.Pointer(&in.Parameters))
return nil
}

// Convert_v1alpha1_MachineClassRef_To_cluster_MachineClassRef is an autogenerated conversion function.
func Convert_v1alpha1_MachineClassRef_To_cluster_MachineClassRef(in *MachineClassRef, out *cluster.MachineClassRef, s conversion.Scope) error {
return autoConvert_v1alpha1_MachineClassRef_To_cluster_MachineClassRef(in, out, s)
}

func autoConvert_cluster_MachineClassRef_To_v1alpha1_MachineClassRef(in *cluster.MachineClassRef, out *MachineClassRef, s conversion.Scope) error {
out.Name = in.Name
out.Parameters = *(*map[string]string)(unsafe.Pointer(&in.Parameters))
return nil
}

// Convert_cluster_MachineClassRef_To_v1alpha1_MachineClassRef is an autogenerated conversion function.
func Convert_cluster_MachineClassRef_To_v1alpha1_MachineClassRef(in *cluster.MachineClassRef, out *MachineClassRef, s conversion.Scope) error {
return autoConvert_cluster_MachineClassRef_To_v1alpha1_MachineClassRef(in, out, s)
}

func autoConvert_v1alpha1_MachineClassStatusStrategy_To_cluster_MachineClassStatusStrategy(in *MachineClassStatusStrategy, out *cluster.MachineClassStatusStrategy, s conversion.Scope) error {
out.DefaultStatusStorageStrategy = in.DefaultStatusStorageStrategy
return nil
}

// Convert_v1alpha1_MachineClassStatusStrategy_To_cluster_MachineClassStatusStrategy is an autogenerated conversion function.
func Convert_v1alpha1_MachineClassStatusStrategy_To_cluster_MachineClassStatusStrategy(in *MachineClassStatusStrategy, out *cluster.MachineClassStatusStrategy, s conversion.Scope) error {
return autoConvert_v1alpha1_MachineClassStatusStrategy_To_cluster_MachineClassStatusStrategy(in, out, s)
}

func autoConvert_cluster_MachineClassStatusStrategy_To_v1alpha1_MachineClassStatusStrategy(in *cluster.MachineClassStatusStrategy, out *MachineClassStatusStrategy, s conversion.Scope) error {
out.DefaultStatusStorageStrategy = in.DefaultStatusStorageStrategy
return nil
}

// Convert_cluster_MachineClassStatusStrategy_To_v1alpha1_MachineClassStatusStrategy is an autogenerated conversion function.
func Convert_cluster_MachineClassStatusStrategy_To_v1alpha1_MachineClassStatusStrategy(in *cluster.MachineClassStatusStrategy, out *MachineClassStatusStrategy, s conversion.Scope) error {
return autoConvert_cluster_MachineClassStatusStrategy_To_v1alpha1_MachineClassStatusStrategy(in, out, s)
}

func autoConvert_v1alpha1_MachineClassStrategy_To_cluster_MachineClassStrategy(in *MachineClassStrategy, out *cluster.MachineClassStrategy, s conversion.Scope) error {
out.DefaultStorageStrategy = in.DefaultStorageStrategy
return nil
}

// Convert_v1alpha1_MachineClassStrategy_To_cluster_MachineClassStrategy is an autogenerated conversion function.
func Convert_v1alpha1_MachineClassStrategy_To_cluster_MachineClassStrategy(in *MachineClassStrategy, out *cluster.MachineClassStrategy, s conversion.Scope) error {
return autoConvert_v1alpha1_MachineClassStrategy_To_cluster_MachineClassStrategy(in, out, s)
}

func autoConvert_cluster_MachineClassStrategy_To_v1alpha1_MachineClassStrategy(in *cluster.MachineClassStrategy, out *MachineClassStrategy, s conversion.Scope) error {
out.DefaultStorageStrategy = in.DefaultStorageStrategy
return nil
}

// Convert_cluster_MachineClassStrategy_To_v1alpha1_MachineClassStrategy is an autogenerated conversion function.
func Convert_cluster_MachineClassStrategy_To_v1alpha1_MachineClassStrategy(in *cluster.MachineClassStrategy, out *MachineClassStrategy, s conversion.Scope) error {
return autoConvert_cluster_MachineClassStrategy_To_v1alpha1_MachineClassStrategy(in, out, s)
}

func autoConvert_v1alpha1_MachineList_To_cluster_MachineList(in *MachineList, out *cluster.MachineList, s conversion.Scope) error {
out.ListMeta = in.ListMeta
out.Items = *(*[]cluster.Machine)(unsafe.Pointer(&in.Items))
Expand Down Expand Up @@ -727,6 +847,7 @@ func Convert_cluster_ProviderConfig_To_v1alpha1_ProviderConfig(in *cluster.Provi
}

func autoConvert_v1alpha1_ProviderConfigSource_To_cluster_ProviderConfigSource(in *ProviderConfigSource, out *cluster.ProviderConfigSource, s conversion.Scope) error {
out.MachineClass = (*cluster.MachineClassRef)(unsafe.Pointer(in.MachineClass))
return nil
}

Expand All @@ -736,6 +857,7 @@ func Convert_v1alpha1_ProviderConfigSource_To_cluster_ProviderConfigSource(in *P
}

func autoConvert_cluster_ProviderConfigSource_To_v1alpha1_ProviderConfigSource(in *cluster.ProviderConfigSource, out *ProviderConfigSource, s conversion.Scope) error {
out.MachineClass = (*MachineClassRef)(unsafe.Pointer(in.MachineClass))
return nil
}

Expand Down
Loading