Inconsistent final plan when local file contaent is unknown before apply using kube secret binary data

[tschneider-aneo]

Terraform Version

Terraform v1.8.5
on linux_amd64
+ provider registry.terraform.io/hashicorp/kubernetes v2.21.1
+ provider registry.terraform.io/hashicorp/local v2.5.1

Terraform Configuration Files

provider "kubernetes" {
  config_path = "~/.kube/config"
}

terraform {
  required_version = ">= 1.6"
  required_providers {
    local = {
      source  = "hashicorp/local"
      version = ">= 2.4.0"
    }
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = "~> 2.21.1"
    }
  }
}


resource "kubernetes_secret" "my_kube_secret" {
  metadata {
    name = "secret-name"
  }
  data = {
    username = "foo"
    password = "bar"
  }
  type = "kubernetes.io/basic-auth"
}

data "kubernetes_secret" "my_kube_secret_data" {
  metadata {
    name = kubernetes_secret.my_kube_secret.metadata[0].name
  }
  binary_data = {
    username = ""
    password = ""
  }
}

resource "local_file" "my_local_file" {
  content = data.kubernetes_secret.my_kube_secret_data.binary_data["username"]
  filename = "${path.root}/generated/foo.txt"
}

Debug Output

https://gist.github.com/tschneider-aneo/691a2d49d4ee0a69ec035b00622834a9

Expected Behavior

TF should know that the local file's content will be known during apply and should just wait for the data source to retrieve the kube secret data.

Actual Behavior

TF seems to be converting an unknown value to a null value, producing the inconsistent final plan error.

Steps to Reproduce

1. terraform init
2. terraform apply

Additional Context

The kube secret syntax for retrieving base64-encoded data seems to be problematic

References

No response

[liamcervante]

Hi @tschneider-aneo, thanks for filing this!

I think this is likely to be an issue with the kubernetes provider rather than Terraform core. There is a separate repository for the Kubernetes provider: https://github.com/hashicorp/terraform-provider-kubernetes

As an aside, is there a reason you can't reference the kubernetes_secret resource directly? You shouldn't need the intermediary data source, the data you are passing into the local_file should be available directly from the resource. Potentially, this could solve the problem for you as there may be some weirdness happening with race conditions in which the Kubernetes cluster hasn't finished creating the secret before Terraform attempts to read it via the data source. This would depend on implementation details within the provider.

I will close this issue here, but feel free to raise this directly with the Kubernetes team.

Thanks!

[tschneider-aneo]

Hi @tschneider-aneo, thanks for filing this!

I think this is likely to be an issue with the kubernetes provider rather than Terraform core. There is a separate repository for the Kubernetes provider: https://github.com/hashicorp/terraform-provider-kubernetes

As an aside, is there a reason you can't reference the kubernetes_secret resource directly? You shouldn't need the intermediary data source, the data you are passing into the local_file should be available directly from the resource. Potentially, this could solve the problem for you as there may be some weirdness happening with race conditions in which the Kubernetes cluster hasn't finished creating the secret before Terraform attempts to read it via the data source. This would depend on implementation details within the provider.

I will close this issue here, but feel free to raise this directly with the Kubernetes team.

Thanks!

Hi @liamcervante, thank you for your answer.

I'm sorry I didn't mention it, but the reason why I did not reference the kubernetes secret resource directly is because I encountered this bug while trying to access a kube secret created by a Helm release resource (deployed with TF). But deploying a Helm release is quite tedious and not necessary to reproduce the bug.

[lemaitre-aneo]

Hello @liamcervante, I am a colleague of @tschneider-aneo, and I want to highlight the problem.
The problem is that the value that should be unknown is somehow transformed into a null value.
At first, we thought it might be a terraform issue as small variations made it work, but we will check back with the kubernetes provider first.

[liamcervante]

Thanks for the extra context! I've tried to replicate this with other (non-kubernetes) generated resources and haven't been able to which is what makes me think the Kubernetes resources are introducing the problem. If you can replicate with something other than Kubernetes than we could more likely show this is a Terraform Core problem.

[liamcervante]

Note, I've found hashicorp/terraform-provider-kubernetes#2444. This is marked as acknowledged, so I think this is a known bug the provider team is working on.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.