Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix Karpenter addon not return null for instanceStorePolicy as default #960

Merged
merged 1 commit into from
Mar 15, 2024
Merged

Fix Karpenter addon not return null for instanceStorePolicy as default #960

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 39 additions & 39 deletions lib/addons/karpenter/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ export type Ec2NodeClassSpec = {
* Required for Alpha CRDS
*/
securityGroupSelector?: Values,

/**
* Subnet selector terms (subnet id or tags) used for Beta CRDs
* Required for Beta CRDS
Expand Down Expand Up @@ -88,7 +88,7 @@ export type Ec2NodeClassSpec = {
* Karpenter will automatically query the appropriate EKS optimized AMI via AWS Systems Manager
*/
amiFamily?: "AL2" | "Bottlerocket" | "Ubuntu" | "Windows2019" | "Windows2022"

/**
* Optional field to control how instance store volumes are handled. Set it to RAID0
* for faster ephemeral storage
Expand All @@ -100,7 +100,7 @@ export type Ec2NodeClassSpec = {
* i.e. custom scripts or pass-through custom configurations at start-up
*/
userData?: string,

/**
* Optional field to use the name of the IAM Instance profile,
* instead of the role generated by Karpenter.
Expand All @@ -110,7 +110,7 @@ export type Ec2NodeClassSpec = {

/**
* Tags adds tags to all resources created, including EC2 Instances, EBS volumes and Launch Templates.
* Karpenter allows overrides of the default "Name" tag but does not allow overrides to restricted domains
* Karpenter allows overrides of the default "Name" tag but does not allow overrides to restricted domains
* (such as "karpenter.sh", "karpenter.k8s.aws", and "kubernetes.io/cluster").
* This ensures that Karpenter is able to correctly auto-discover machines that it owns.
*/
Expand Down Expand Up @@ -167,38 +167,38 @@ export type NodePoolSpec = {
requirements?: NodePoolRequirementValues,

/**
* Enables consolidation which attempts to reduce cluster cost by both removing un-needed nodes and down-sizing those that can't be removed.
* Enables consolidation which attempts to reduce cluster cost by both removing un-needed nodes and down-sizing those that can't be removed.
* Mutually exclusive with the ttlSecondsAfterEmpty parameter.
*
*
* Replaced with disruption.consolidationPolicy for versions v0.32.x and later
*/
consolidation?: {
enabled: boolean,
}

/**
* If omitted, the feature is disabled and nodes will never expire.
* If set to less time than it requires for a node to become ready,
* If omitted, the feature is disabled and nodes will never expire.
* If set to less time than it requires for a node to become ready,
* the node may expire before any pods successfully start.
*
*
* Replaced with disruption.expireAfter for versions v0.32.x and later
*/
ttlSecondsUntilExpired?: number,

/**
* How many seconds Karpenter will wailt until it deletes empty/unnecessary instances (in seconds).
* Mutually exclusive with the consolidation parameter.
*
*
* Replaced with disruption.consolidationPolicy and disruption.consolidateAfter for versions v0.32.x and later
*/
ttlSecondsAfterEmpty?: number,

/**
/**
* Disruption section which describes the ways in which Karpenter can disrupt and replace Nodes
* Configuration in this section constrains how aggressive Karpenter can be with performing operations
* like rolling Nodes due to them hitting their maximum lifetime (expiry) or scaling down nodes to reduce cluster cost
* Only applicable for versions v0.32 or later
*
*
* @param consolidationPolicy consolidation policy - will default to WhenUnderutilized if not provided
* @param consolidateAfter How long Karpenter waits to consolidate nodes - cannot be set when the policy is WhenUnderutilized
* @param expireAfter How long Karpenter waits to expire nodes
Expand Down Expand Up @@ -241,20 +241,20 @@ export type NodePoolSpec = {
*/
export interface KarpenterAddOnProps extends HelmAddOnUserProps {
/**
* This is the top level nodepool specification. Nodepools launch nodes in response to pods that are unschedulable.
* A single nodepool is capable of managing a diverse set of nodes.
* This is the top level nodepool specification. Nodepools launch nodes in response to pods that are unschedulable.
* A single nodepool is capable of managing a diverse set of nodes.
* Node properties are determined from a combination of nodepool and pod scheduling constraints.
*/
nodePoolSpec?: NodePoolSpec,

/**
* This is the top level spec for the AWS Karpenter Provider
* It contains configuration necessary to launch instances in AWS.
* It contains configuration necessary to launch instances in AWS.
*/
ec2NodeClassSpec?: Ec2NodeClassSpec,

/**
* Flag for enabling Karpenter's native interruption handling
* Flag for enabling Karpenter's native interruption handling
*/
interruptionHandling?: boolean,
}
Expand Down Expand Up @@ -294,7 +294,7 @@ export class KarpenterAddOn extends HelmAddOn {
const endpoint = cluster.clusterEndpoint;
const name = cluster.clusterName;
const partition = cluster.stack.partition;

const stackName = cluster.stack.stackName;
const region = cluster.stack.region;

Expand Down Expand Up @@ -324,9 +324,9 @@ export class KarpenterAddOn extends HelmAddOn {
const amiFamily = this.options.ec2NodeClassSpec?.amiFamily;
const amiSelector = this.options.ec2NodeClassSpec?.amiSelector || {};
const amiSelectorTerms = this.options.ec2NodeClassSpec?.amiSelectorTerms;
const instanceStorePolicy = this.options.ec2NodeClassSpec?.instanceStorePolicy || "";
const instanceStorePolicy = this.options.ec2NodeClassSpec?.instanceStorePolicy || null;
const userData = this.options.ec2NodeClassSpec?.userData || "";
const instanceProf = this.options.ec2NodeClassSpec?.instanceProfile;
const instanceProf = this.options.ec2NodeClassSpec?.instanceProfile;
const tags = this.options.ec2NodeClassSpec?.tags || {};
const metadataOptions = this.options.ec2NodeClassSpec?.metadataOptions || {
httpEndpoint: "enabled",
Expand All @@ -336,12 +336,12 @@ export class KarpenterAddOn extends HelmAddOn {
};
const blockDeviceMappings = this.options.ec2NodeClassSpec?.blockDeviceMappings || [];
const detailedMonitoring = this.options.ec2NodeClassSpec?.detailedMonitoring || false;

// Check Kubernetes and Karpenter version compatibility for warning
this.isCompatible(version, clusterInfo.version);

// Version feature checks for errors
this.versionFeatureChecksForError(clusterInfo, version, disruption, consol, ttlSecondsAfterEmpty, ttlSecondsUntilExpired,
this.versionFeatureChecksForError(clusterInfo, version, disruption, consol, ttlSecondsAfterEmpty, ttlSecondsUntilExpired,
this.options.ec2NodeClassSpec, amiFamily);

// Set up the node role and instance profile
Expand Down Expand Up @@ -438,16 +438,16 @@ export class KarpenterAddOn extends HelmAddOn {
};

if (semver.lt(version, '0.32.0')){
globalSettings = merge(globalSettings, {
globalSettings = merge(globalSettings, {
defaultInstanceProfile: karpenterInstanceProfile.instanceProfileName,
interruptionQueueName: interruption ? stackName : ""
interruptionQueueName: interruption ? stackName : ""
});
} else {
globalSettings = merge(globalSettings, {
interruptionQueue: interruption ? stackName : ""
globalSettings = merge(globalSettings, {
interruptionQueue: interruption ? stackName : ""
});
}

if (semver.lt(version, '0.32.0')){
utils.setPath(values, "settings.aws", merge(globalSettings, values?.settings?.aws ?? {}));
} else {
Expand Down Expand Up @@ -548,7 +548,7 @@ export class KarpenterAddOn extends HelmAddOn {
detailedMonitoring: detailedMonitoring,
},
};

// Provide custom Instance Profile to replace role if provided, else use the role created with the addon
if (instanceProf) {
ec2Node = merge(ec2Node, { spec: { instanceProfile: instanceProf }});
Expand Down Expand Up @@ -580,7 +580,7 @@ export class KarpenterAddOn extends HelmAddOn {
},
};

// Add EC2 Detailed Monitoring for v0.22.0 and up
// Add EC2 Detailed Monitoring for v0.22.0 and up
if (semver.gte(version, '0.22.0')){
ec2Node = merge(ec2Node, { spec: { detailedMonitoring: detailedMonitoring}});
}
Expand All @@ -594,7 +594,7 @@ export class KarpenterAddOn extends HelmAddOn {
}

/**
* Helper function to convert a key-pair values (with an operator)
* Helper function to convert a key-pair values (with an operator)
* of spec configurations to appropriate json format for addManifest function
* @param reqs
* @returns newReqs
Expand All @@ -618,7 +618,7 @@ export class KarpenterAddOn extends HelmAddOn {
/**
* Helper function to ensure right features are added as part of the configuration
* for the right version of the add-on
* @param clusterInfo
* @param clusterInfo
* @param version version of the add-on
* @param disruption disruption feature available with the Beta CRDs
* @param consolidation consolidation setting available with the Alpha CRDs
Expand All @@ -628,9 +628,9 @@ export class KarpenterAddOn extends HelmAddOn {
* @param amiFamily AMI Family
* @returns
*/
private versionFeatureChecksForError(clusterInfo: ClusterInfo, version: string, disruption: any, consolidation: any, ttlSecondsAfterEmpty: any, ttlSecondsUntilExpired: any,
private versionFeatureChecksForError(clusterInfo: ClusterInfo, version: string, disruption: any, consolidation: any, ttlSecondsAfterEmpty: any, ttlSecondsUntilExpired: any,
ec2NodeClassSpec: any, amiFamily: any): void {

// EC2 Detailed Monitoring is only available in versions 0.23.0 and above
if (semver.lt(version, '0.23.0') && ec2NodeClassSpec){
assert(ec2NodeClassSpec["detailedMonitoring"] === undefined, "Detailed Monitoring is not available in this version of Karpenter. Please upgrade to at least 0.23.0.");
Expand All @@ -647,7 +647,7 @@ export class KarpenterAddOn extends HelmAddOn {
if (semver.gte(version, '0.32.0')){
// Consolidation features don't exist in beta CRDs
assert(!consolidation && !ttlSecondsAfterEmpty && !ttlSecondsUntilExpired, 'Consolidation features are only available for previous versions of Karpenter.');

// consolidateAfter cannot be set if policy is set to WhenUnderutilized
if (disruption && disruption["consolidationPolicy"] == "WhenUnderutilized"){
assert(!disruption["consolidateAfter"], 'You cannot set consolidateAfter value if the consolidation policy is set to Underutilized.');
Expand All @@ -662,7 +662,7 @@ export class KarpenterAddOn extends HelmAddOn {
}

// version check errors for v0.31.x and down (alpha CRDs)
// Includes checks for consolidation and disruption features
// Includes checks for consolidation and disruption features
if (semver.lt(version, '0.32.0')){
if (consolidation){
assert(!(consolidation.enabled && ttlSecondsAfterEmpty) , 'Consolidation and ttlSecondsAfterEmpty must be mutually exclusive.');
Expand Down Expand Up @@ -709,7 +709,7 @@ export class KarpenterAddOn extends HelmAddOn {
instanceProfileName: `KarpenterNodeInstanceProfile-${instanceProfileName}`,
path: '/'
});

const clusterId = Names.uniqueId(cluster);

//Cfn output for Node Role in case of needing to add additional policies
Expand All @@ -719,10 +719,10 @@ export class KarpenterAddOn extends HelmAddOn {
exportName: clusterId+"KarpenterNodeRoleName",
});
//Cfn output for Instance Profile for creating additional provisioners
new CfnOutput(cluster.stack, 'Karpenter Instance Profile name', {
new CfnOutput(cluster.stack, 'Karpenter Instance Profile name', {
value: karpenterInstanceProfile ? karpenterInstanceProfile.instanceProfileName! : "none",
description: "Karpenter add-on Instance Profile name",
exportName: clusterId+"KarpenterInstanceProfileName",
exportName: clusterId+"KarpenterInstanceProfileName",
});

// Map Node Role to aws-auth
Expand Down Expand Up @@ -750,6 +750,6 @@ export class KarpenterAddOn extends HelmAddOn {
const compatibleVersion = versionMap.get(kubeVersion) as string;
if (semver.gt(compatibleVersion, karpenterVersion)) {
console.warn(`Please use minimum Karpenter version for this Kubernetes Version: ${compatibleVersion}, otherwise you will run into compatibility issues.`);
}
}
}
}
Loading