Block execution engine settings in sql query settings API and add more unit tests

[vamsi-amazon]

Description

Currently SQL Query Settings API acts as a backdoor for changing cluster settings specific to sql engine. We no longer require this for the new settings introduced for spark execution engine.

New Settings Introduced.

plugins.query.datasources.encryption.masterkey
plugins.query.datasources.uri.hosts.denylist
plugins.query.executionengine.spark.config
plugins.query.executionengine.spark.session.limit
plugins.query.executionengine.spark.refresh_job.limit
plugins.query.executionengine.spark.session.index.ttl
plugins.query.executionengine.spark.result.index.ttl
plugins.query.executionengine.spark.auto_index_management.enabled

We also need to rethink the use cases of this API and deprecate if not required.[https://github.com//issues/2409]

This PR covers blocking execution engine settings.

Query Settings API Result before and after:

PUT {{baseUrl}}/_plugins/_query/settings
content-type: application/json

{
  "transient" : {
    "plugins.sql.enabled" : "false",
    "plugins.query.executionengine.spark.config" : "--conf spark.driverEnv.executors=2"
  }
}

{
  "acknowledged": true,
  "persistent": {},
  "transient": {
    "plugins": {
      "query": {
        "executionengine": {
          "spark": {
            "config": "--conf spark.driverEnv.executors=2"
          }
        }
      },
      "sql": {
        "enabled": "false"
      }
    }
  }
}

After the change.

PUT {{baseUrl}}/_plugins/_query/settings
content-type: application/json

{
  "transient" : {
    "plugins.sql.enabled" : "false",
    "plugins.query.executionengine.spark.config" : "--conf spark.driverEnv.executors=2",
    "plugins.query.datasources.uri.hosts.denylist": "127.001.1, 123.100.1"
  }
}

Response:

{
  "acknowledged": true,
  "persistent": {
    "plugins": {
      "sql": {
        "enabled": "false"
      }
    }
  },
  "transient": {}
}

Issues Resolved

#2382

Check List

• New functionality includes testing.
  • All tests pass, including unit test, integration test and doctest
• New functionality has been documented.
  • New functionality has javadoc added
  • New functionality has user manual doc added
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

Merging #2407 (c161e77) into main (6cd2db0) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##               main    #2407   +/-   ##
=========================================
  Coverage     95.54%   95.54%           
  Complexity     4985     4985           
=========================================
  Files           478      478           
  Lines         13883    13883           
  Branches        931      931           
=========================================
  Hits          13265    13265           
  Misses          598      598           
  Partials         20       20           

Flag	Coverage Δ
sql-engine	95.54% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

[vamsi-amazon]

This change is specifically to assert the entry point class jar.

[kavithacm]

lgtm