parsing failure when "set {}" value contains a comma and manifest experiment is enabled

[anapsix]

Terraform, Provider, Kubernetes and Helm Versions

Terraform version: v1.3.6
Provider version: v2.8.0 (and earlier)
Kubernetes version: v1.22.3

Affected Resource(s)

• helm_release

Terraform Configuration Files

provider "kubernetes" {}

provider "helm" {
  experiments {
    manifest = true
  }
}

resource "helm_release" "example" {
  name       = "my-redis-release"
  repository = "https://charts.bitnami.com/bitnami"
  chart      = "redis"

  set {
    name  = "commonAnnotations.customAnnotation"
    value = "something,else"
  }
}

Debug Output

Full debug output: https://gist.github.com/anapsix/2902b39e28dd718be91b0dcab89e34c4

Steps to Reproduce

1. configure helm provider with manifest = true experiments enabled
2. use set {} in helm_release resource with value containing a comma
3. terraform plan

Expected Behavior

❯ terraform plan

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # helm_release.example will be created
  + resource "helm_release" "example" {
      + atomic                     = false
      + chart                      = "redis"
      + cleanup_on_fail            = false
      + create_namespace           = false
      + dependency_update          = false
      + disable_crd_hooks          = false
      + disable_openapi_validation = false
      + disable_webhooks           = false
      + force_update               = false
      + id                         = (known after apply)
      + lint                       = false
      + manifest                   = (known after apply)
      + max_history                = 0
      + metadata                   = (known after apply)
      + name                       = "my-redis-release"
      + namespace                  = "default"
      + pass_credentials           = false
      + recreate_pods              = false
      + render_subchart_notes      = true
      + replace                    = false
      + repository                 = "https://charts.bitnami.com/bitnami"
      + reset_values               = false
      + reuse_values               = false
      + skip_crds                  = false
      + status                     = "deployed"
      + timeout                    = 300
      + verify                     = false
      + version                    = (known after apply)
      + wait                       = true
      + wait_for_jobs              = false

      + set {
          + name  = "commonAnnotations.customAnnotation"
          + value = "something,else"
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Actual Behavior

❯ terraform plan
╷
│ Error: error getting values: failed parsing key "commonAnnotations.customAnnotation" with value something,else, key "else" has no value
│
│   with helm_release.example,
│   on main.tf line 9, in resource "helm_release" "example":
│    9: resource "helm_release" "example" {
│
╵

Important Factoids

Enabling manifest = true Helm provider experiment appears to cause an issue when set {} value contains a comma. While passing a similar construct via values = [...] is trouble free:

values = [
  yamlencode({
    commonAnnotations = {
      customAnnotation = "something, else"
    }
  })
]

This behavior is observed with Helm provider version 2.8.0 and v2.7.x during "plan", while 2.6.x, 2.5.x, 2.4.x, 2.3.x, 2.2.x, and 2.1.x produce the same error during "apply". I have not tested earlier provider versions since manifest = true experiment is not available before 2.1.0

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[yaroslav-nakonechnikov]

same problem raised with ingress chart:
example:

  set {
    name  = "controller.service.annotations.service\\.beta\\.kubernetes\\.io/aws-load-balancer-subnets"
    type  = "auto"
    value = "subnet-0fd26432879bc21xx, subnet-0ba383f86c94539xx, subnet-0ebb77e75a9904dxx"
  }

and it produces:

Error: failed parsing key "controller.service.annotations.service\\.beta\\.kubernetes\\.io/aws-load-balancer-subnets" with value subnet-0fd26432879bc21xx, subnet-0ba383f86c94539xx, subnet-0ebb77e75a9904dxx, key " subnet-0ba383f86c94539xx" has no value (cannot end with ,)

  with helm_release.ingress_nginx,
  on main.tf line 553, in resource "helm_release" "ingress_nginx":
 553: resource "helm_release" "ingress_nginx" {

[yaroslav-nakonechnikov]

this is incredible how not explained "feature" breaks things...

i found old #495 with nice comment: #495 (comment)

and it solved.

Need to escape comma!

[anapsix]

Indeed, @iaroslav-nakonechnikov, the following syntax works. Thanks!

• escaping comma

  resource "helm_release" "example" {
    name       = "my-redis-release"
    repository = "https://charts.bitnami.com/bitnami"
    chart      = "redis"
  
    set {
      name  = "commonAnnotations.customAnnotation"
      value = "something\\,else"
    }
  }

• verifying

  ❯ kubectl get sts my-redis-release-master -o yaml | oq -i yaml -r '.metadata.annotations'
  {
    "customAnnotation": "something,else",
    "meta.helm.sh/release-name": "my-redis-release",
    "meta.helm.sh/release-namespace": "default"
  }
  

Turns out the bitnami/redis is less than ideal example, since it keeps changing "checksum" annotations, hitting another manifest experiment Helm provider bug (I believe, the issue is described in #805).. so, changing commonAnnotations.customAnnotation value from something\\,else to something (or making any other change) produces the This is a bug in the provider, which should be reported in the provider's own issue tracker error when Helm provider has manifest = true.

Switching to bitnami/nginx chart for testing the escape workaround is more successful.

• applying

  resource "helm_release" "example" {
    name       = "my-nginx-release"
    repository = "https://charts.bitnami.com/bitnami"
    chart      = "nginx"
  
    set {
      name  = "commonAnnotations.customAnnotation"
      value = "something\\,\\ else"
    }
  
    set {
      name  = "service.type"
      value = "NodePort"
    }
  }

• verifying

  ❯ kubectl get deploy my-nginx-release -o yaml | oq -i yaml -r '.metadata.annotations'
  {
    "customAnnotation": "something, else",
    "deployment.kubernetes.io/revision": "1",
    "meta.helm.sh/release-name": "my-nginx-release",
    "meta.helm.sh/release-namespace": "default"
  }
  

• applying change

  // ..snip
    set {
      name  = "commonAnnotations.customAnnotation"
      value = "something\\,\\ else\\,\\ entirely"
    }
  // snip..

• verifying

  ❯ kubectl get deploy my-nginx-release -o yaml | oq -i yaml -r '.metadata.annotations'
  {
    "customAnnotation": "something, else, entirely",
    "deployment.kubernetes.io/revision": "1",
    "meta.helm.sh/release-name": "my-nginx-release",
    "meta.helm.sh/release-namespace": "default"
  }
  

Kinda annoying having to escape commas, spaces, etc.. So, I'm sticking with yamlencode workaround:

// ..snip
values = [
  yamlencode({
    commonAnnotations = {
      customAnnotation = "something, else, entirely, sort of"
    }
  })
]
// snip..

[anapsix]

For the record, I was unable to use the type = "string" (as mentioned in #495 (comment)) and make it work without having to escape the comma

[arybolovlev]

It seems to be a valid behavior. Helm treats the comma as a separator when you pass it to set so the provider works in the same way:

--set stringArray    set values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)

As was mentioned here already, you either need to escape the comma or use values.

I will go ahead and close this issue.

Thank you.