Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Unable to get a user/role #377

Closed
orlandothoeny opened this issue Jul 4, 2023 · 6 comments
Closed

[Bug] Unable to get a user/role #377

orlandothoeny opened this issue Jul 4, 2023 · 6 comments
Labels
bug Something isn't working

Comments

@orlandothoeny
Copy link

orlandothoeny commented Jul 4, 2023

Hello,

We are running an Elastic Cloud instance, which is managed using Terraform.

We use the elasticstack_elasticsearch_security_user & elasticstack_elasticsearch_security_role resources to create users & roles.
The following error occurs during the state refresh when Elasticsearch users/roles are fetched while running terraform apply: "Security must be explicitly enabled when using a [basic] license. Enable security by setting [xpack.security.enabled] to [true] in the elasticsearch.yml file and restart the node."

Our code was working before, it broke at some point. Unfortunately, I cannot identify the exact reason. Might be due to an elasticstack provider update or an Elasticsearch server update.

The weird thing is, calling the API using curl works, the following returns JSON containing our Elasticsearch users.

curl -u 'USER:PASSWORD' -X GET https://my-instance.es.europe-west4.gcp.elastic-cloud.com/_security/user

I tried setting the xpack.security.enabled: true config explicitly but that's not possible on Elastic Cloud:
Screenshot from 2023-07-04 10-32-48

Terraform errors


│ Error: Unable to get a user.

│ with module.elastic_cloud.elasticstack_elasticsearch_security_user.terraform_user,
│ on ../modules/elastic_cloud/users.tf line 23, in resource "elasticstack_elasticsearch_security_user" "terraform_user":
│ 23: resource "elasticstack_elasticsearch_security_user" "terraform_user" {

│ Failed with: {"error":{"root_cause":[{"type":"exception","reason":"Security must be explicitly enabled when using a [basic] license. Enable security by setting [xpack.security.enabled] to [true] in the elasticsearch.yml file and restart the node."}],"type":"exception","reason":"Security must be explicitly enabled when using a
│ [basic] license. Enable security by setting [xpack.security.enabled] to [true] in the elasticsearch.yml file and restart the node."},"status":500}


│ Error: Unable to get a role.

│ with module.elastic_cloud.elasticstack_elasticsearch_security_role.datadog_role,
│ on ../modules/elastic_cloud/users.tf line 54, in resource "elasticstack_elasticsearch_security_role" "datadog_role":
│ 54: resource "elasticstack_elasticsearch_security_role" "datadog_role" {

│ Failed with: {"error":{"root_cause":[{"type":"exception","reason":"Security must be explicitly enabled when using a [basic] license. Enable security by setting [xpack.security.enabled] to [true] in the elasticsearch.yml file and restart the node."}],"type":"exception","reason":"Security must be explicitly enabled when using a
│ [basic] license. Enable security by setting [xpack.security.enabled] to [true] in the elasticsearch.yml file and restart the node."},"status":500}

Debug output

2023-07-04T11:40:08.395+0200 [TRACE] provider.terraform-provider-elasticstack_v0.5.0: Received request: tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_req_id=c92e2a6e-de56-3ee4-f812-0f64d0542635 tf_resource_type=elasticstack_elasticsearch_security_role @caller=github.com/hashicorp/terraform-plugin-go@v0.14.1/tfprotov5/tf5server/server.go:737 @module=sdk.proto tf_proto_version=5.3 tf_rpc=ReadResource timestamp=2023-07-04T11:40:08.395+0200
2023-07-04T11:40:08.395+0200 [TRACE] provider.terraform-provider-elasticstack_v0.5.0: Sending request downstream: tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_resource_type=elasticstack_elasticsearch_security_role tf_rpc=ReadResource @module=sdk.proto tf_proto_version=5.3 tf_req_id=c92e2a6e-de56-3ee4-f812-0f64d0542635 @caller=github.com/hashicorp/terraform-plugin-go@v0.14.1/tfprotov5/internal/tf5serverlogging/downstream_request.go:17 timestamp=2023-07-04T11:40:08.395+0200
2023-07-04T11:40:08.395+0200 [TRACE] provider.terraform-provider-elasticstack_v0.5.0: Received request: @module=sdk.proto tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_rpc=ReadResource @caller=github.com/hashicorp/terraform-plugin-go@v0.14.1/tfprotov5/tf5server/server.go:737 tf_proto_version=5.3 tf_req_id=80442025-43c8-f436-1eb0-7c157c7d2a9b tf_resource_type=elasticstack_elasticsearch_security_role timestamp=2023-07-04T11:40:08.395+0200
2023-07-04T11:40:08.395+0200 [TRACE] provider.terraform-provider-elasticstack_v0.5.0: calling downstream server: tf_mux_provider=*schema.GRPCProviderServer tf_rpc=ReadResource @caller=github.com/hashicorp/terraform-plugin-mux@v0.7.0/internal/logging/mux.go:16 @module=sdk.mux timestamp=2023-07-04T11:40:08.395+0200
2023-07-04T11:40:08.395+0200 [TRACE] provider.terraform-provider-elasticstack_v0.5.0: Sending request downstream: @caller=github.com/hashicorp/terraform-plugin-go@v0.14.1/tfprotov5/internal/tf5serverlogging/downstream_request.go:17 @module=sdk.proto tf_resource_type=elasticstack_elasticsearch_security_role tf_proto_version=5.3 tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_req_id=80442025-43c8-f436-1eb0-7c157c7d2a9b tf_rpc=ReadResource timestamp=2023-07-04T11:40:08.395+0200
2023-07-04T11:40:08.395+0200 [TRACE] provider.terraform-provider-elasticstack_v0.5.0: calling downstream server: tf_mux_provider=*schema.GRPCProviderServer tf_rpc=ReadResource @caller=github.com/hashicorp/terraform-plugin-mux@v0.7.0/internal/logging/mux.go:16 @module=sdk.mux timestamp=2023-07-04T11:40:08.395+0200
2023-07-04T11:40:08.395+0200 [TRACE] provider.terraform-provider-elasticstack_v0.5.0: Received request: tf_provider_addr=registry.terraform.io/elastic/elasticstack @module=sdk.proto tf_proto_version=5.3 tf_req_id=432bb24d-5a2a-04dd-4338-6bb76efc33e2 tf_resource_type=elasticstack_elasticsearch_security_role tf_rpc=ReadResource @caller=github.com/hashicorp/terraform-plugin-go@v0.14.1/tfprotov5/tf5server/server.go:737 timestamp=2023-07-04T11:40:08.395+0200
2023-07-04T11:40:08.395+0200 [TRACE] provider.terraform-provider-elasticstack_v0.5.0: Sending request downstream: @module=sdk.proto tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_resource_type=elasticstack_elasticsearch_security_role @caller=github.com/hashicorp/terraform-plugin-go@v0.14.1/tfprotov5/internal/tf5serverlogging/downstream_request.go:17 tf_proto_version=5.3 tf_req_id=432bb24d-5a2a-04dd-4338-6bb76efc33e2 tf_rpc=ReadResource timestamp=2023-07-04T11:40:08.395+0200
2023-07-04T11:40:08.395+0200 [TRACE] provider.terraform-provider-elasticstack_v0.5.0: calling downstream server: @caller=github.com/hashicorp/terraform-plugin-mux@v0.7.0/internal/logging/mux.go:16 @module=sdk.mux tf_mux_provider=*schema.GRPCProviderServer tf_rpc=ReadResource timestamp=2023-07-04T11:40:08.395+0200
2023-07-04T11:40:08.396+0200 [TRACE] provider.terraform-provider-elasticstack_v0.5.0: Calling downstream: tf_rpc=ReadResource tf_resource_type=elasticstack_elasticsearch_security_role @caller=github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/resource.go:1014 @module=sdk.helper_schema tf_mux_provider=*schema.GRPCProviderServer tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_req_id=c92e2a6e-de56-3ee4-f812-0f64d0542635 timestamp=2023-07-04T11:40:08.395+0200
2023-07-04T11:40:08.396+0200 [TRACE] provider.terraform-provider-elasticstack_v0.5.0: Calling downstream: tf_resource_type=elasticstack_elasticsearch_security_role tf_rpc=ReadResource @caller=github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/resource.go:1014 tf_mux_provider=*schema.GRPCProviderServer tf_provider_addr=registry.terraform.io/elastic/elasticstack @module=sdk.helper_schema tf_req_id=80442025-43c8-f436-1eb0-7c157c7d2a9b timestamp=2023-07-04T11:40:08.395+0200
2023-07-04T11:40:08.396+0200 [TRACE] provider.terraform-provider-elasticstack_v0.5.0: Calling downstream: @module=sdk.helper_schema tf_req_id=432bb24d-5a2a-04dd-4338-6bb76efc33e2 tf_resource_type=elasticstack_elasticsearch_security_role @caller=github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/resource.go:1014 tf_mux_provider=*schema.GRPCProviderServer tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_rpc=ReadResource timestamp=2023-07-04T11:40:08.395+0200
2023-07-04T11:40:08.396+0200 [DEBUG] provider.terraform-provider-elasticstack_v0.5.0: elasticsearch API request dump error: &errors.errorString{s:"unsupported protocol scheme """}: tf_mux_provider=*schema.GRPCProviderServer tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_req_id=c92e2a6e-de56-3ee4-f812-0f64d0542635 @caller=github.com/elastic/terraform-provider-elasticstack/internal/clients/debug.go:45 @module=elasticstack tf_resource_type=elasticstack_elasticsearch_security_role tf_rpc=ReadResource timestamp=2023-07-04T11:40:08.395+0200
2023-07-04T11:40:08.396+0200 [DEBUG] provider.terraform-provider-elasticstack_v0.5.0: elasticsearch API request dump error: &errors.errorString{s:"unsupported protocol scheme """}: tf_rpc=ReadResource @caller=github.com/elastic/terraform-provider-elasticstack/internal/clients/debug.go:45 tf_req_id=80442025-43c8-f436-1eb0-7c157c7d2a9b tf_resource_type=elasticstack_elasticsearch_security_role @module=elasticstack tf_mux_provider=*schema.GRPCProviderServer tf_provider_addr=registry.terraform.io/elastic/elasticstack timestamp=2023-07-04T11:40:08.395+0200
2023-07-04T11:40:08.396+0200 [DEBUG] provider.terraform-provider-elasticstack_v0.5.0: elasticsearch API request dump error: &errors.errorString{s:"unsupported protocol scheme """}: @caller=github.com/elastic/terraform-provider-elasticstack/internal/clients/debug.go:45 @module=elasticstack tf_mux_provider=*schema.GRPCProviderServer tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_req_id=432bb24d-5a2a-04dd-4338-6bb76efc33e2 tf_resource_type=elasticstack_elasticsearch_security_role tf_rpc=ReadResource timestamp=2023-07-04T11:40:08.395+0200
2023-07-04T11:40:08.397+0200 [DEBUG] provider.terraform-provider-elasticstack_v0.5.0: elasticsearch API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 500 Internal Server Error
Content-Type: application/json; charset=UTF-8
Warning: 299 Elasticsearch-7.17.4-79878662c54c886ae89206c685d9f1051a9d6411 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.17/security-minimal-setup.html to enable security."
X-Elastic-Product: Elasticsearch

{
"error": {
"root_cause": [
{
"type": "exception",
"reason": "Security must be explicitly enabled when using a [basic] license. Enable security by setting [xpack.security.enabled] to [true] in the elasticsearch.yml file and restart the node."
}
],
"type": "exception",
"reason": "Security must be explicitly enabled when using a [basic] license. Enable security by setting [xpack.security.enabled] to [true] in the elasticsearch.yml file and restart the node."
},
"status": 500
}
-----------------------------------------------------: tf_rpc=ReadResource @caller=github.com/elastic/terraform-provider-elasticstack/internal/clients/debug.go:55 tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_req_id=11d63bca-28ef-7e46-5726-d7dd7fdddfe5 @module=elasticstack tf_mux_provider=*schema.GRPCProviderServer tf_resource_type=elasticstack_elasticsearch_security_role timestamp=2023-07-04T11:40:08.397+0200
2023-07-04T11:40:08.397+0200 [TRACE] provider.terraform-provider-elasticstack_v0.5.0: Called downstream: tf_resource_type=elasticstack_elasticsearch_security_role tf_rpc=ReadResource tf_provider_addr=registry.terraform.io/elastic/elasticstack @module=sdk.helper_schema tf_mux_provider=*schema.GRPCProviderServer tf_req_id=11d63bca-28ef-7e46-5726-d7dd7fdddfe5 @caller=github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/resource.go:1016 timestamp=2023-07-04T11:40:08.397+0200
2023-07-04T11:40:08.397+0200 [TRACE] provider.terraform-provider-elasticstack_v0.5.0: Received downstream response: tf_req_duration_ms=3 tf_resource_type=elasticstack_elasticsearch_security_role tf_rpc=ReadResource diagnostic_error_count=1 diagnostic_warning_count=0 tf_provider_addr=registry.terraform.io/elastic/elasticstack tf_req_id=11d63bca-28ef-7e46-5726-d7dd7fdddfe5 @caller=github.com/hashicorp/terraform-plugin-go@v0.14.1/tfprotov5/internal/tf5serverlogging/downstream_request.go:37 @module=sdk.proto tf_proto_version=5.3 timestamp=2023-07-04T11:40:08.397+0200
2023-07-04T11:40:08.397+0200 [ERROR] provider.terraform-provider-elasticstack_v0.5.0: Response contains error diagnostic: diagnostic_severity=ERROR diagnostic_summary="Unable to get a role." @caller=github.com/hashicorp/terraform-plugin-go@v0.14.1/tfprotov5/internal/diag/diagnostics.go:55 diagnostic_detail="Failed with: {"error":{"root_cause":[{"type":"exception","reason":"Security must be explicitly enabled when using a [basic] license. Enable security by setting [xpack.security.enabled] to [true] in the elasticsearch.yml file and restart the node."}],"type":"exception","reason":"Security must be explicitly enabled when using a [basic] license. Enable security by setting [xpack.security.enabled] to [true] in the elasticsearch.yml file and restart the node."},"status":500}" tf_req_id=11d63bca-28ef-7e46-5726-d7dd7fdddfe5 tf_resource_type=elasticstack_elasticsearch_security_role tf_rpc=ReadResource @module=sdk.proto tf_proto_version=5.3 tf_provider_addr=registry.terraform.io/elastic/elasticstack timestamp=2023-07-04T11:40:08.397+0200

Versions (please complete the following information):

  • OS: Ubuntu 22.04
  • Terraform Version 1.5.1
  • Provider version 0.5.0, also tried with 0.6.2, but get the same error
  • Elasticsearch Version 7.17.9
@orlandothoeny orlandothoeny added the bug Something isn't working label Jul 4, 2023
@Kushmaro
Copy link
Contributor

i'm not clear on where this was tested. On a basic licensed cluster, or on Elastic Cloud?
sounds like the former?

if so, it seems the problem is the basic license.

@orlandothoeny
Copy link
Author

orlandothoeny commented Jul 12, 2023

Hi @Kushmaro,

We're using Elastic Cloud with a single Elasticsearch instance and a Kibana instance.

Our Terraform code was working before. But as mentioned we're not able to identify what specifically broke it.
Probably either a Terraform provider upgrade or an update of our Elastic Cloud instance. Or maybe something about the licensing changed? But I'm able to get the users by calling the API with curl using the Terraform Elasticsearch user. So the Terraform code should also be able to 🤔

@Kushmaro
Copy link
Contributor

@orlandothoeny well its quite hard to determine what broke with the code itself. Can you share the relevant piece?

@orlandothoeny
Copy link
Author

orlandothoeny commented Jul 13, 2023

Here is a simplified version of the code we're using.

  • We create the Elastic Cloud deployment
  • Then use the built-in elastic user to authenticate with the elasticstack provider. (the initial terraform apply is a 2-step process, the deployment needs to be created first since the provider is initialized at the beginning which throws an error if the user is missing. Unlike in the code, we don't access the username & password via the ec_deployment resource but store it inside a GCP secret and then use its value to pass it to the provider. Left that out to make it simpler)
  • We create Elasticsearch users & roles
provider "ec" {
  // ...
}

resource "ec_deployment" "deployment" {
  name  = "my-deployment-dev"
  alias = "my-deployment-dev"

  region                 = "europe-west4"
  version                = "7.17.9"
  deployment_template_id = "gcp-storage-optimized"

  elasticsearch = {
    ref_id    = "main-elasticsearch"
    autoscale = "false"

    hot = {
      autoscaling = {}
    }

    topology = {
      id = "hot_content"

      zone_count = 1

      size          = "2g"
      size_resource = "memory"
    }
  }

  kibana = {
    ref_id = "kibana"

    topology = {
      size          = "1g"
      size_resource = "memory"
    }
  }

  observability = {
    deployment_id = "self"
    logs          = false
    metrics       = true
  }
}

provider "elasticstack" {
  elasticsearch {
    endpoints = [ec_deployment.deployment.elasticsearch.https_endpoint]
    username  = ec_deployment.deployment.elasticsearch_username
    password  = ec_deployment.deployment.elasticsearch_password
  }
}

resource "elasticstack_elasticsearch_security_user" "terraform_user" {
  username = "terraform"

  password = "foo"
  roles    = ["superuser"]
}

resource "elasticstack_elasticsearch_security_user" "datadog_user" {
  username = "datadog"

  password = "bar"
  /* We still need the "monitoring_user" user role, because we cannot assign the Kibana "Stack Monitoring" permission to our own custom "datadog" role.
  These are only available via this built-in role */
  roles = [elasticstack_elasticsearch_security_role.datadog_role.name, "monitoring_user"]
}

resource "elasticstack_elasticsearch_security_role" "datadog_role" {
  name = "datadog"

  indices {
    names      = [".monitoring-*", "metricbeat-*"]
    privileges = ["read", "read_cross_cluster", "monitor"]
  }

  cluster = ["monitor"]
  run_as  = []
}

@orlandothoeny
Copy link
Author

Hi @Kushmaro,

I also tried it by using explicit per-resource definitions (elasticsearch_connection) of credentials. But the same error happens.

provider "ec" {
  // ...
}

resource "ec_deployment" "deployment" {
  name  = "my-deployment-dev"
  alias = "my-deployment-dev"

  region                 = "europe-west4"
  version                = "7.17.9"
  deployment_template_id = "gcp-storage-optimized"

  elasticsearch = {
    ref_id    = "main-elasticsearch"
    autoscale = "false"

    hot = {
      autoscaling = {}
    }

    topology = {
      id = "hot_content"

      zone_count = 1

      size          = "2g"
      size_resource = "memory"
    }
  }

  kibana = {
    ref_id = "kibana"

    topology = {
      size          = "1g"
      size_resource = "memory"
    }
  }

  observability = {
    deployment_id = "self"
    logs          = false
    metrics       = true
  }
}

provider "elasticstack" {
  elasticsearch {}
}

resource "elasticstack_elasticsearch_security_user" "terraform_user" {
  username = "terraform"

  password = "foo"
  roles    = ["superuser"]

elasticsearch_connection {
    endpoints = [ec_deployment.deployment.elasticsearch.https_endpoint]
    username  = ec_deployment.deployment.elasticsearch_username
    password  = ec_deployment.deployment.elasticsearch_password
  }
}

resource "elasticstack_elasticsearch_security_user" "datadog_user" {
  username = "datadog"

  password = "bar"
  /* We still need the "monitoring_user" user role, because we cannot assign the Kibana "Stack Monitoring" permission to our own custom "datadog" role.
  These are only available via this built-in role */
  roles = [elasticstack_elasticsearch_security_role.datadog_role.name, "monitoring_user"]

elasticsearch_connection {
    endpoints = [ec_deployment.deployment.elasticsearch.https_endpoint]
    username  = ec_deployment.deployment.elasticsearch_username
    password  = ec_deployment.deployment.elasticsearch_password
  }
}

resource "elasticstack_elasticsearch_security_role" "datadog_role" {
  name = "datadog"

  indices {
    names      = [".monitoring-*", "metricbeat-*"]
    privileges = ["read", "read_cross_cluster", "monitor"]
  }

  cluster = ["monitor"]
  run_as  = []

elasticsearch_connection {
    endpoints = [ec_deployment.deployment.elasticsearch.https_endpoint]
    username  = ec_deployment.deployment.elasticsearch_username
    password  = ec_deployment.deployment.elasticsearch_password
  }
}

@orlandothoeny
Copy link
Author

Seems like the root cause is actually #370

Managed to get it to work by using the workaround described in that issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants