Add support for AWS "OpenSearch"?

[TonyLovesDevOps]

AWS has officially renamed AWS Elasticsearch to OpenSearch after forking Elasticsearch to OpenSearch and Kibana to OpenSearch Dashboards at opensearch.org, because lawyers and 💰.

There are at least two immediate issues with this provider supporting OpenSearch:

1. OpenSearch returns version.number: "1.0.0" from the default endpoint even though it's really ES 7.10.2 under the hood so the provider returns Error: ElasticSearch is older than 5.0.0! when trying to apply changes. This can be worked around by enabling compatibility mode so the domain returns 7.10 as its version;
2. I'm still looking into why, but when pointing the provider at an AWS OpenSearch domain (even with compatibility mode enabled) it returns several Error: no URLs found: no Elasticsearch node available during a plan operation.

Here's a repro for the first problem:

# docker-compose.yml
version: '3'
services:
  opensearch-node1:
    image: opensearchproject/opensearch:1.0.1
    container_name: opensearch-node1
    environment:
      - discovery.type=single-node
    volumes:
      - opensearch-data1:/usr/share/opensearch/data
    ports:
      - 9200:9200
    networks:
      - opensearch-net
  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:1.0.1
    container_name: opensearch-dashboards
    ports:
      - 5601:5601
    expose:
      - "5601"
    environment:
      OPENSEARCH_HOSTS: '["https://opensearch-node1:9200"]'
    networks:
      - opensearch-net

volumes:
  opensearch-data1:

networks:
  opensearch-net:

# opensearch_test.tf
terraform {
  required_providers {
    elasticsearch = {
      source = "phillbaker/elasticsearch"
      version = "~> 1.6.3"
    }
  }
}

provider "elasticsearch" {
  insecure  = true # to bypass certificate check
  url       = "https://localhost:9200"
  username  = "admin"
  password  = "admin"
}

resource "elasticsearch_index" "foo" {
  name                = "foo"
  number_of_shards    = 1
  number_of_replicas  = 1
}

1. Save the docker-compose.yaml and terraform_object_test.tf files to a temp directory.
2. Run terraform init to set up the providers
3. Run docker compose up to set up the ES/kibana containers
4. Run terraform plan -out plan.out and terraform apply plan.out to create the index.

Expected Behavior
Successful apply.

Actual Behavior
Error: ElasticSearch is older than 5.0.0! returned.

[TonyLovesDevOps]

It seems that the olivere/elastic library intends to support both.

[phillbaker]

Hi @TonyLovesDevOps, thanks for opening an issue.

For (1) for now, I would suggest that compatibility mode is used until we either add support explicitly in the provider for opensearch or there are new features that are opensearch specific and we add support for them. We should document that as the expected process for the time being. Update: looks like the compatibility mode can be enabled via the AWS API: hashicorp/terraform-provider-aws#20853, folks could use that as well.

For (2), this sounds like an authentication issue, can you confirm that you have network access and the correct credentials available to the provider from the location where it's running? Can you try setting sniff and healthcheck to false in the provider config?

[TonyLovesDevOps]

For (1), I had to add this block to my aws_elasticsearch_domain resource block in order to not have the object recreated or the compatibility mode switch disabled:

  lifecycle {
    ignore_changes = [ elasticsearch_version, advanced_options ]
  }

For (2) I'm not sure what was going on, but I "modified" the Access Policy for the domain in the AWS console and saved the exact same policy that was there before, and now everything appears to be working! 🎉 I also made sure I had the latest version (currently 3.58.0) of the aws provider, though I think that's not what fixed it as I see no commits adding opensearch support to that repo.

[ngamber]

An example of features that aren't supported would be data_streams, the index_template resource bombs out if a data_stream is specified (which the cluster supports on opensearch_1.0). .

│ Error: elastic: Error 400 (Bad Request): unknown key [data_stream] in the template [type=parse_exception]

[phillbaker]

Hi @ngamber, can you please share the underlying elasticsearch versions and the resource definition that you're using?

To my knowledge datastreams were only included in XPack until ES v7.10 and so may be included in Opensearch, which forked off of ES v7.10, but not in any opendistro version which is based on earlier versions of ES.

[ngamber]

I'm using 2.0.0-beta.2 and Opensearch-1.0.

Following the opensearch docs, I've tried adding one to an index template with the previous error.

{
  "test" : {
    "order" : 0,
    "index_patterns" : [
      "test*"
    ],
     "data_stream": {},

    "settings" : {
      "index" : {
        "mapping" : {
          "total_fields" : {
            "limit" : "2000"
          }
        },
        "opendistro" : {
          "index_state_management" : {
            "rollover_alias" : "test"
          }
        },
        "search" : {
          "slowlog" : {
            "threshold" : {
              "query" : {
                "warn" : "10s"
              }
            }
          }
        },
        "number_of_shards" : "1",
        "number_of_replicas" : "0"
      }
    },
    "mappings" : {
      "_source" : {
        "enabled" : true
      },
      "properties" : {
        "@timestamp" : {
          "type" : "date"
        },
        "host_name" : {
          "type" : "keyword"
        }
      }
    },
    "aliases" : { }
  }
}

example template

(https://opensearch.org/docs/latest/opensearch/data-streams/)

[ngamber]

actual tf policy:

  "index_patterns": ["test*"],

  "data_stream": {},
  "settings": {
    "index": {
      "number_of_shards": "1",
      "number_of_replicas": "0",
      "mapping.total_fields.limit": 2000,
      "search": {
        "slowlog" : {
          "threshold" : {
            "query" : {
              "warn": "10s"
            }
          }
        }
      }
    },
    "opendistro.index_state_management.rollover_alias": "test"

  },
  "mappings": {
    "_source": {
      "enabled": true
    },
    "properties": {
      "host_name": {
        "type": "keyword"
      },
      "@timestamp": {
        "type": "date"
      }
    }
  }
}

[nelg]

+1

[kushalgangan]

Hi @phillbaker, terraform has added the OpenSearch support in aws provider, could you please fix below issue facing in Elasticsearch provider:

Error: ElasticSearch is older than 5.0.0!

FYI: hashicorp/terraform-provider-aws#20853 (comment)

CC: @lakshmi-enjeti