Broker node EBS provisioned throughput settings

[angryhamsterx]

what

Ability to enable EBS volume provisioned throughput for broker nodes kafka.m5.4xlarge or larger.
You can add configuration block for this by setting broker_ebs_provisioned_throughput_enabled, and then change enable/disable state by broker_ebs_provisioned_throughput_state.

Caveat:
Once provisioned_throughput block edded, it can not be removed, so there is variable broker_ebs_provisioned_throughput_state to change state of provisioned throughput.

why

• Support missed configuration options.
• Additional flexibility in setting up a clusters using this module.

references

• https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/msk_cluster#ebs_storage_info-provisioned_throughput-argument-reference

[sbogdanovic]

In case we want to explicitly disable provisioned throughput, to produce for example:

provisioned_throughput {
    enabled = false
}

the module with the change you suggested would try to set volume_throughput to the default value (250).

this would cover that case, and fix this bug:

dynamic "provisioned_throughput" {
  for_each = var.broker_ebs_provisioned_throughput_enabled && var.broker_ebs_provisioned_throughput_state ? [1] : []

  content {
    enabled           = true
    volume_throughput = var.broker_ebs_provisioned_throughput_value
  }
}

dynamic "provisioned_throughput" {
  for_each = var.broker_ebs_provisioned_throughput_enabled && !var.broker_ebs_provisioned_throughput_state ? [1] : []

  content {
    enabled           = false
  }
}

[tommij]

@angryhamsterx would you mind if we pull your PR changes into a seperate PR from us, with cloudposse'isms covered, and drive the PR forwards?

[tommij]

@angryhamsterx https://github.com/InfareSolutions/terraform-aws-msk-apache-kafka-cluster/tree/broker_ebs_provisioned_throughput

[hans-d]

@angryhamsterx can you update the pr (see also the failing readme check)

[mergify]

Thanks @angryhamsterx for creating this pull request!

A maintainer will review your changes shortly. Please don't be discouraged if it takes a while.

While you wait, make sure to review our contributor guidelines.

Tip

Need help or want to ask for a PR review to be expedited?

Join us on Slack in the #pr-reviews channel.

[Nuru]

Also, to fix #117 restore the

  lifecycle {
    ignore_changes = [
      # Ignore changes to ebs_volume_size in favor of autoscaling policy
broker_node_group_info[0].storage_info[0].ebs_storage_info[0].volume_size,
    ]
  }

We do NOT want to ignore changes to provisioned throughput, as that would make it impossible to update.

[Nuru]

Suggested change

	variable "broker_ebs_provisioned_throughput_enabled" {
	type = bool
	default = false
	description = "Enable broker node EBS provisioned throughput for broker type kafka.m5.4xlarge or larger"
	nullable = false
	}
	variable "broker_ebs_provisioned_throughput_state" {
	type = bool
	default = false
	description = "Enable or disable provisioned throughput of the EBS volumes for the data drive on kafka broker nodes"
	nullable = false
	}
	variable "broker_ebs_provisioned_throughput_value" {
	type = number
	default = 250
	description = "Provisioned throughput value of the EBS volumes for the data drive on kafka broker nodes in MiB per second"
	validation {
	condition = var.broker_ebs_provisioned_throughput_value >= 250
	error_message = "broker_ebs_provisioned_throughput_value must be minimum 250."
	}
	nullable = false
	}
	variable "broker_ebs_provisioned_throughput" {
	type = object({
	enabled = optional(bool, false)
	volume_throughput = optional(number)
	})
	description = "Enable broker node EBS provisioned throughput for broker type kafka.m5.4xlarge or larger"
	default = {}
	nullable = false
	}

[Nuru]

Suggested change

	dynamic "provisioned_throughput" {
	for_each = var.broker_ebs_provisioned_throughput_enabled ? [1] : []
	content {
	enabled = var.broker_ebs_provisioned_throughput_state
	volume_throughput = var.broker_ebs_provisioned_throughput_value
	}
	}
	provisioned_throughput {
	enabled = var.broker_ebs_provisioned_throughput.enabled
	volume_throughput = var.broker_ebs_provisioned_throughput.volume_throughput
	}