Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Broker node EBS provisioned throughput settings #101

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Broker node EBS provisioned throughput settings #101

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
385f1f8
Broker node EBS provisioned throughput settings
angryhamsterx Dec 24, 2023
9b1994e
Merge branch 'main' into broker_ebs_provisioned_throughput
Nuru Jul 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,15 @@ resource "aws_msk_cluster" "default" {
storage_info {
ebs_storage_info {
volume_size = var.broker_volume_size

dynamic "provisioned_throughput" {
Copy link

@sbogdanovic sbogdanovic Jan 30, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case we want to explicitly disable provisioned throughput, to produce for example:

provisioned_throughput {
    enabled = false
}

the module with the change you suggested would try to set volume_throughput to the default value (250).

this would cover that case, and fix this bug:

dynamic "provisioned_throughput" {
  for_each = var.broker_ebs_provisioned_throughput_enabled && var.broker_ebs_provisioned_throughput_state ? [1] : []

  content {
    enabled           = true
    volume_throughput = var.broker_ebs_provisioned_throughput_value
  }
}

dynamic "provisioned_throughput" {
  for_each = var.broker_ebs_provisioned_throughput_enabled && !var.broker_ebs_provisioned_throughput_state ? [1] : []

  content {
    enabled           = false
  }
}

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@angryhamsterx would you mind if we pull your PR changes into a seperate PR from us, with cloudposse'isms covered, and drive the PR forwards?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@angryhamsterx https://github.com/InfareSolutions/terraform-aws-msk-apache-kafka-cluster/tree/broker_ebs_provisioned_throughput

for_each = var.broker_ebs_provisioned_throughput_enabled ? [1] : []

content {
enabled = var.broker_ebs_provisioned_throughput_state
volume_throughput = var.broker_ebs_provisioned_throughput_value
}
}
Comment on lines +149 to +156
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
dynamic "provisioned_throughput" {
for_each = var.broker_ebs_provisioned_throughput_enabled ? [1] : []
content {
enabled = var.broker_ebs_provisioned_throughput_state
volume_throughput = var.broker_ebs_provisioned_throughput_value
}
}
provisioned_throughput {
enabled = var.broker_ebs_provisioned_throughput.enabled
volume_throughput = var.broker_ebs_provisioned_throughput.volume_throughput
}

}
}

Expand Down
25 changes: 25 additions & 0 deletions variables.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,31 @@ variable "broker_volume_size" {
nullable = false
}

variable "broker_ebs_provisioned_throughput_enabled" {
type = bool
default = false
description = "Enable broker node EBS provisioned throughput for broker type kafka.m5.4xlarge or larger"
nullable = false
}

variable "broker_ebs_provisioned_throughput_state" {
type = bool
default = false
description = "Enable or disable provisioned throughput of the EBS volumes for the data drive on kafka broker nodes"
nullable = false
}

variable "broker_ebs_provisioned_throughput_value" {
type = number
default = 250
description = "Provisioned throughput value of the EBS volumes for the data drive on kafka broker nodes in MiB per second"
validation {
condition = var.broker_ebs_provisioned_throughput_value >= 250
error_message = "broker_ebs_provisioned_throughput_value must be minimum 250."
}
nullable = false
}

Comment on lines +34 to +58
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
variable "broker_ebs_provisioned_throughput_enabled" {
type = bool
default = false
description = "Enable broker node EBS provisioned throughput for broker type kafka.m5.4xlarge or larger"
nullable = false
}
variable "broker_ebs_provisioned_throughput_state" {
type = bool
default = false
description = "Enable or disable provisioned throughput of the EBS volumes for the data drive on kafka broker nodes"
nullable = false
}
variable "broker_ebs_provisioned_throughput_value" {
type = number
default = 250
description = "Provisioned throughput value of the EBS volumes for the data drive on kafka broker nodes in MiB per second"
validation {
condition = var.broker_ebs_provisioned_throughput_value >= 250
error_message = "broker_ebs_provisioned_throughput_value must be minimum 250."
}
nullable = false
}
variable "broker_ebs_provisioned_throughput" {
type = object({
enabled = optional(bool, false)
volume_throughput = optional(number)
})
description = "Enable broker node EBS provisioned throughput for broker type kafka.m5.4xlarge or larger"
default = {}
nullable = false
}

variable "subnet_ids" {
type = list(string)
description = "Subnet IDs for Client Broker"
Expand Down