Support YAML anchors in elasticsearch.yml

[albertzaharovits]

I wonder whether we should support YAML anchors.

Here is an example which I believe is not supported:

xpack:
  security:
    enabled: true
  http:
    ssl:
      enabled: true
      keystore: &keystore
      path: /usr/share/elasticsearch/config/keystore.p12
      type: PKCS12
      password: ${ES_SECRET}
      key_password: ${ES_SECRET}
  transport:
    ssl:
      enabled: true
      verification_mode: certificate
      keystore: *keystore
      truststore:
        path: /usr/share/elasticsearch/config/truststore.p12
        type: PKCS12
        password: ${ES_SECRET}

The example defines the &keystore "anchor" which is then referred to by the *keystore alias.

[elasticmachine]

Pinging @elastic/es-core-infra (:Core/Infra/Settings)

[rjernst]

@albertzaharovits In theory, we should already support anchors, although there may be problems with our implementation since we view the yaml through jackson's json parsing interface, and I also see there seem to be ongoing issues with anchors in jackson (FasterXML/jackson-dataformats-text#98).

However, what error do you get when using the above? It looks to me like the indentation is off (path/type/etc should be indented one more level?).

[albertzaharovits]

@rjernst Aaargh, You're right, the indentation was off. I've corrected it:

xpack:
  security:
    enabled: true
    http:
      ssl:
        enabled: true
        keystore: &ks
          path: elastic-certificates.p12
          type: PKCS12
          password: pass
          key_password: pass
    transport:
      ssl:
        enabled: true
        verification_mode: certificate
        keystore: *ks
        truststore:
          path: elastic-stack-ca.p12
          type: PKCS12
          password: ""

This is the error I get:

org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: unknown setting [xpack.security.transport.ssl.keystore] did you mean any of [xpack.security.transport.ssl.keystore.path, xpack.security.transport.ssl.keystore.type, xpack.security.transport.ssl.key, xpack.security.transport.ssl.key_passphrase, xpack.security.transport.ssl.enabled, xpack.security.transport.ssl.keystore.password, xpack.security.transport.ssl.keystore.algorithm, xpack.security.transport.ssl.certificate, xpack.security.transport.ssl.truststore.path, xpack.security.transport.ssl.truststore.type, xpack.security.transport.ssl.keystore.key_password, xpack.security.transport.ssl.verification_mode, xpack.security.transport.ssl.truststore.password]?
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.5.2.jar:7.5.2]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.5.2.jar:7.5.2]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.5.2.jar:7.5.2]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) ~[elasticsearch-cli-7.5.2.jar:7.5.2]
        at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.5.2.jar:7.5.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.5.2.jar:7.5.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.5.2.jar:7.5.2]
Caused by: java.lang.IllegalArgumentException: unknown setting [xpack.security.transport.ssl.keystore] did you mean any of [xpack.security.transport.ssl.keystore.path, xpack.security.transport.ssl.keystore.type, xpack.security.transport.ssl.key, xpack.security.transport.ssl.key_passphrase, xpack.security.transport.ssl.enabled, xpack.security.transport.ssl.keystore.password, xpack.security.transport.ssl.keystore.algorithm, xpack.security.transport.ssl.certificate, xpack.security.transport.ssl.truststore.path, xpack.security.transport.ssl.truststore.type, xpack.security.transport.ssl.keystore.key_password, xpack.security.transport.ssl.verification_mode, xpack.security.transport.ssl.truststore.password]?
        at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:530) ~[elasticsearch-7.5.2.jar:7.5.2]
        at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:475) ~[elasticsearch-7.5.2.jar:7.5.2]
        at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:446) ~[elasticsearch-7.5.2.jar:7.5.2]
        at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:417) ~[elasticsearch-7.5.2.jar:7.5.2]
        at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:149) ~[elasticsearch-7.5.2.jar:7.5.2]
        at org.elasticsearch.node.Node.<init>(Node.java:352) ~[elasticsearch-7.5.2.jar:7.5.2]
        at org.elasticsearch.node.Node.<init>(Node.java:253) ~[elasticsearch-7.5.2.jar:7.5.2]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:241) ~[elasticsearch-7.5.2.jar:7.5.2]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:241) ~[elasticsearch-7.5.2.jar:7.5.2]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:369) ~[elasticsearch-7.5.2.jar:7.5.2]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.5.2.jar:7.5.2]
        ... 6 more