[Bug] Subsequent terraform apply fail to connect to deployment's endpoint

[frconil]

Describe the bug
When using a combination of the ec provider and the elasticstack provider, runs after the initial apply fail with connection issues, as the endpoint is replaced with the default (localhost:9200):

 Error: dial tcp [::1]:9200: connect: connection refused
│
│   with elasticstack_elasticsearch_security_user.user,
│   on main.tf line 46, in resource "elasticstack_elasticsearch_security_user" "user":
│   46: resource "elasticstack_elasticsearch_security_user" "user" {
│

This is likely a consequence of elastic/terraform-provider-ec#599 as I cannot reproduce the bug with terraform <1.4 (tested with 1.3.0 and 1.3.9)

To Reproduce
Steps to reproduce the behavior:

1. TF configuration used:

terraform {
  required_version = ">= 1.0.0"
  required_providers {
    ec = {
      source  = "elastic/ec"
    }
     elasticstack = {
      source  = "elastic/elasticstack"
   }
  }
}

provider "ec" {
apikey = "[redacted]"
}

resource "ec_deployment" "custom-deployment-id" {
  name                   = "My deployment identifier"
  region                 = "gcp-europe-west3"
  version                = "8.8.0"
  deployment_template_id = "gcp-memory-optimized-v2"

 elasticsearch = {
    hot = {
      autoscaling = {}
    }
  }
  kibana = {}
}

provider "elasticstack" {
  elasticsearch {
    username = ec_deployment.custom-deployment-id.elasticsearch_username
    password = ec_deployment.custom-deployment-id.elasticsearch_password
  endpoints = ["${ec_deployment.custom-deployment-id.elasticsearch.https_endpoint}"]
  }
}

resource "elasticstack_elasticsearch_security_user" "user" {
  username = "ingest_user"
  password = "mysecretpassword"
  roles    = ["editor"]

  # Set the custom metadata for this user
  metadata = jsonencode({
    "env"    = "testing"
    "open"   = false
    "number" = 49
  })
}

4. TF operations to execute to get the error: terraform plan, terraform apply
5. See the error in the output

terraform plan
ec_deployment.custom-deployment-id: Refreshing state... [id=be69ae8f29bc80d8fd273208e3141402]
elasticstack_elasticsearch_security_user.user: Refreshing state... [id=qaKsUow9SrOXtWhSoJb4fQ/ingest_user]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform planned the following actions, but then encountered a problem:

  # ec_deployment.custom-deployment-id will be updated in-place
  ~ resource "ec_deployment" "custom-deployment-id" {
      + apm_secret_token       = (sensitive value)
      ~ elasticsearch          = {
          ~ cloud_id       = "My_deployment_identifier:[redacted]=" -> (known after apply)
          ~ http_endpoint  = "http://cc23ffd44d7c4890b0dc8994d155f4a5.europe-west3.gcp.cloud.es.io:9200" -> (known after apply)
          ~ https_endpoint = "https://cc23ffd44d7c4890b0dc8994d155f4a5.europe-west3.gcp.cloud.es.io:443" -> (known after apply)
          ~ resource_id    = "cc23ffd44d7c4890b0dc8994d155f4a5" -> (known after apply)
            # (7 unchanged attributes hidden)
        }
        id                     = "be69ae8f29bc80d8fd273208e3141402"
        name                   = "My deployment identifier"
        # (7 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.
╷
│ Error: dial tcp [::1]:9200: connect: connection refused
│
│   with elasticstack_elasticsearch_security_user.user,
│   on main.tf line 46, in resource "elasticstack_elasticsearch_security_user" "user":
│   46: resource "elasticstack_elasticsearch_security_user" "user" {
│
╵

Expected behavior

The same terraform plan runs without issues in 1.3.9:

terraform plan
ec_deployment.custom-deployment-id: Refreshing state... [id=be69ae8f29bc80d8fd273208e3141402]
elasticstack_elasticsearch_security_user.user: Refreshing state... [id=qaKsUow9SrOXtWhSoJb4fQ/ingest_user]

No changes. Your infrastructure matches the configuration.

Versions (:

• OS: macos
• Terraform Version 1.5.1
• Provider version v0.6.0

Additional context
As mentioned above, I believe this is caused by the configuration drift reported in elastic/terraform-provider-ec#599

[nobuhikosekiya]

+1. I ran into the same issue yesterday.

[adopauco]

I have the same issue here with message:
Error: dial tcp [::1]:9200: connect: connection refused
Even with Terraform 1.3.9, it does not work for me.

Versions:

• Terraform versions tested: 1.5.1, 1.4.6, 1.3.9
• ec provider 0.7.0
• elasticstack provider 0.6.2

[adopauco]

I have the same issue here with message: Error: dial tcp [::1]:9200: connect: connection refused Even with Terraform 1.3.9, it does not work for me.

Versions:

• Terraform versions tested: 1.5.1, 1.4.6, 1.3.9
• ec provider 0.7.0
• elasticstack provider 0.6.2

I managed to find a workaround. If there is only one elasticsearch deployment, then you can set the ELASTICSEARCH_ENDPOINTS variable to the elastic cloud elasticsearch https endpoint. This fixes the connection refused error in terraform plan.

[gvozdetsky]

For me this workaround worked:

data "ec_deployment" "ec" {
  id = module.ElasticCloud.id
}

provider "elasticstack" {
  # Use our Elastic Cloud deployment outputs for connection details.
  # This also allows the provider to create the proper relationships between the two resources.
  elasticsearch {
    endpoints = [data.ec_deployment.ec.elasticsearch[0].https_endpoint]
    username  = module.ElasticCloud.elastic_username_output
    password  = module.ElasticCloud.elastic_password_output
  }
}

[jamesagarside]

I too am experiencing this. Initial apply worked but subsequent give

 dial tcp [::1]:9200: connect: connection refused

[jamesagarside]

Terraform Provider

[gvozdetsky]

fixed in https://github.com/elastic/terraform-provider-ec/ 0.8.0

[dlaczeg]

0.8.0 working for me