This repository has been archived by the owner on Mar 30, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 162
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add support for SSL/TLS for Elasticsearch HTTP and Transport layers (#…
…187) Add support for SSL/TLS for Elasticsearch HTTP and Transport layers This commit adds support for securing HTTP and Transport layers by specifying parameters with which PKCS#12 archives containing the certificate and key can be supplied. To configure TLS for the HTTP layer, a PKCS#12 archive containing the HTTP cert and key or an archive containing the HTTP CA cert and key must be provided. When an archive containing the HTTP cert and key is provided, it will take preference, and Kibana will be configured with certificate verification mode. The most likely use case for this is to provide a cert for a CNAME pointing at the external loadbalancer public IP. When an archive containing the HTTP CA cert and key HTTP certs is provided, the CA certs is used to generate a PKCS#12 archive containing a certificate and key for each node, including the node DNS and IP as Subject Alternative Names, as well as the internal loadbalancer public IP. This allows Kibana to be set to full verification mode when communicating internally. A client communicating through the external loadbalancer can verify certificates returned from Elasticsearch against the CA. A client communicating through Application Gateway will use the certificate configured for the Application Gateway. To configure TLS for the Transport layer, a PKCS#12 archive containing the Transport CA cert and key must be provided. The CA cert is used to generate a PKCS#12 archive containing a certificate and key for each node, including the node DNS and IP as Subject Alternative Names. Including the IP allows the cluster to operate in full verification mode for the Transport layer. Other changes: - Use the insecure flag when calling localhost over HTTPS with curl. Since the subject name in certificate used to secure the HTTP layer is most likely not going to match localhost (it's likely to be tied to a public domain name), the --cacert flag for curl cannot be used. - Update integration tests to use the parameters-file argument to pass the parameters to azure cli. Since parameters can now contain base 64 encoded certificates, the input can be longer than the maximum characters allowed in Windows (8192). - Add generated self-signed certificates for use in integration tests. - Add --test parameter to be able to filter integration tests --test <regexp> with a pattern to match the tests to run - Configure Application Gateway to work with TLS on backend pool Add support for TLS from Application Gateway to the backend pool. Application Gateway communicates with the backend pool through the internal loadbalancer, and the public certificate(s) used by the backend pool must be whitelisted by Application Gateway by providing these details to it. This means that for TLS on the HTTP layer in conjunction with Application Gateway, only the single esHttpCertBlob option can be supported. In the case of esHttpCaCertBlob which is used to generate a cert for HTTP layer for each VM/node in the cluster, the public keys for these certs cannot be automatically added to Application Gateway as part of deployment. - Pass Elasticsearch HTTP and HTTP CA archives for Kibana configuration Pass the Elasticsearch HTTP and HTTP CA PKCS#12 archives to Kibana to configure TLS to Elasticsearch from Kibana. When a CA cert is provided, it is used to configure the certificate authority. The presence of an Elasticsearch HTTP archive overrides the presence of a HTTP CA archive. That is, if a HTTP archive is provided, the Certificate Authority will be extracted from this PKCS#12 archive. If no CA cert is present, The verification mode for TLS with Elasticsearch will be set to none. - Remove versions less than 5.3.x from the template. 5.0.x and 5.1.x are now EOL and 5.2.x will be EOL end of July. 5.2.x is removed now because Kibana Console does not work with self-signed certs; Kibana itself works, but not console, responding with Client request error: unable to verify the first certificate Closes #199 - Remove hostname verification in integration tests Remove the hostname verification check used by node's request module. Since all certs used are self-signed, hostname verification will fail. Tests still verify the CA. - Add support for --nodestroy parameter Adds a --nodestroy parameter that when passed, does not delete resource groups after integration tests
- Loading branch information
Showing
32 changed files
with
1,586 additions
and
245 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,8 +1,5 @@ | ||
| { | ||
| "versions": [ | ||
| "5.0.2", | ||
| "5.1.2", | ||
| "5.2.2", | ||
| "5.3.2", | ||
| "5.4.0", | ||
| "5.4.2", | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| { | ||
| "description": "1 data node cluster using temp disk, with additional yaml configuration", | ||
| "isValid" : true, | ||
| "deploy" : true, | ||
| "why" : "", | ||
| "location" : "westeurope", | ||
| "parameters" : { | ||
| "loadBalancerType":{"value":"internal"}, | ||
| "kibana":{"value":"Yes"}, | ||
| "kibanaAdditionalYaml":{"value":"server.name: \"My server\"\nserver.defaultRoute: \"/app/kibana\""}, | ||
| "esAdditionalYaml":{"value":"action.auto_create_index: +.*\nindices.queries.cache.size: 5%"}, | ||
| "jumpbox":{"value":"No"}, | ||
| "vmSizeKibana":{"value":"Standard_DS1_v2"}, | ||
| "vmSizeDataNodes":{"value":"Standard_DS1_v2"}, | ||
| "vmDataNodeCount":{"value":1}, | ||
| "vmDataDiskCount":{"value":0}, | ||
| "vmDataDiskSize":{"value":"Small"}, | ||
| "storageAccountType":{"value":"Default"}, | ||
| "dataNodesAreMasterEligible":{"value":"Yes"}, | ||
| "vmSizeMasterNodes":{"value":"Standard_DS2"}, | ||
| "vmClientNodeCount":{"value":0}, | ||
| "vmSizeClientNodes":{"value":"Standard_D1"}, | ||
| "authenticationType":{"value":"password"}, | ||
| "vNetName": {"value": "es-net"}, | ||
| "vNetClusterSubnetName": {"value": "es-subnet"}, | ||
| "vNetAppGatewaySubnetName": {"value": "es-app-gateway"}, | ||
| "vNetLoadBalancerIp": {"value": "10.0.0.4"}, | ||
| "vNetNewOrExisting": {"value":"new"}, | ||
| "vNetExistingResourceGroup": {"value": ""}, | ||
| "vNetNewAddressPrefix": {"value": "10.0.0.0/24"}, | ||
| "vNetNewClusterSubnetAddressPrefix": {"value": "10.0.0.0/25"}, | ||
| "vNetNewAppGatewaySubnetAddressPrefix": {"value": "10.0.0.128/28"}, | ||
| "appGatewayTier": {"value":"Standard"}, | ||
| "appGatewaySku": {"value":"Small"}, | ||
| "appGatewayCount": {"value":1}, | ||
| "appGatewayCertBlob": {"value":""}, | ||
| "appGatewayCertPassword": {"value":""}, | ||
| "appGatewayWafStatus": {"value":"Disabled"}, | ||
| "appGatewayWafMode": {"value":"Detection"}, | ||
| "userCompany": { "value": "" }, | ||
| "userEmail": { "value": "" }, | ||
| "userFirstName": { "value": "" }, | ||
| "userLastName": { "value": "" }, | ||
| "userJobTitle": { "value": "Other" }, | ||
| "userCountry": { "value": "" } | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| { | ||
| "isValid" : true, | ||
| "deploy" : true, | ||
| "why" : "", | ||
| "location" : "westeurope", | ||
| "parameters" : { | ||
| "loadBalancerType":{"value":"gateway"}, | ||
| "esHttpCertBlob":{"value":"cert-no-password.pfx"}, | ||
| "esTransportCaCertBlob":{"value":"ca-cert-no-password.pfx"}, | ||
| "kibana":{"value":"No"}, | ||
| "jumpbox":{"value":"Yes"}, | ||
| "vmSizeDataNodes":{"value":"Standard_D1"}, | ||
| "vmDataNodeCount":{"value":3}, | ||
| "vmDataDiskCount":{"value":0}, | ||
| "vmDataDiskSize":{"value":"Small"}, | ||
| "storageAccountType":{"value":"Default"}, | ||
| "dataNodesAreMasterEligible":{"value":"Yes"}, | ||
| "vmSizeMasterNodes":{"value":"Standard_DS2"}, | ||
| "vmClientNodeCount":{"value":0}, | ||
| "vmSizeClientNodes":{"value":"Standard_D1"}, | ||
| "authenticationType":{"value":"password"}, | ||
| "vNetName": {"value": "es-net"}, | ||
| "vNetClusterSubnetName": {"value": "es-subnet"}, | ||
| "vNetAppGatewaySubnetName": {"value": "es-app-gateway"}, | ||
| "vNetLoadBalancerIp": {"value": "10.0.0.4"}, | ||
| "vNetNewOrExisting": {"value":"new"}, | ||
| "vNetExistingResourceGroup": {"value": ""}, | ||
| "vNetNewAddressPrefix": {"value": "10.0.0.0/24"}, | ||
| "vNetNewClusterSubnetAddressPrefix": {"value": "10.0.0.0/25"}, | ||
| "vNetNewAppGatewaySubnetAddressPrefix": {"value": "10.0.0.128/28"}, | ||
| "appGatewayTier": {"value":"Standard"}, | ||
| "appGatewaySku": {"value":"Small"}, | ||
| "appGatewayCount": {"value":1}, | ||
| "appGatewayCertBlob": {"value":"cert-with-password.pfx"}, | ||
| "appGatewayCertPassword": {"value":"Password123"}, | ||
| "appGatewayEsHttpCertBlob": {"value":"cert-no-password.crt"}, | ||
| "appGatewayWafStatus": {"value":"Disabled"}, | ||
| "appGatewayWafMode": {"value":"Detection"}, | ||
| "userCompany": { "value": "" }, | ||
| "userEmail": { "value": "" }, | ||
| "userFirstName": { "value": "" }, | ||
| "userLastName": { "value": "" }, | ||
| "userJobTitle": { "value": "Other" }, | ||
| "userCountry": { "value": "" } | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| { | ||
| "isValid" : true, | ||
| "deploy" : true, | ||
| "why" : "", | ||
| "location" : "westeurope", | ||
| "parameters" : { | ||
| "loadBalancerType":{"value":"external"}, | ||
| "esHttpCertBlob":{"value":"cert-no-password.pfx"}, | ||
| "esTransportCaCertBlob":{"value":"ca-cert-no-password.pfx"}, | ||
| "kibana":{"value":"Yes"}, | ||
| "kibanaCertBlob": {"value":"cert-no-password.crt"}, | ||
| "kibanaKeyBlob": {"value":"cert-no-password.key"}, | ||
| "jumpbox":{"value":"No"}, | ||
| "vmSizeKibana":{"value":"Standard_D1"}, | ||
| "vmSizeDataNodes":{"value":"Standard_D1"}, | ||
| "vmDataNodeCount":{"value":3}, | ||
| "vmDataDiskCount":{"value":40}, | ||
| "vmDataDiskSize":{"value":"Small"}, | ||
| "storageAccountType":{"value":"Default"}, | ||
| "dataNodesAreMasterEligible":{"value":"Yes"}, | ||
| "vmSizeMasterNodes":{"value":"Standard_DS2"}, | ||
| "vmClientNodeCount":{"value":0}, | ||
| "vmSizeClientNodes":{"value":"Standard_D1"}, | ||
| "authenticationType":{"value":"password"}, | ||
| "vNetName": {"value": "es-net"}, | ||
| "vNetClusterSubnetName": {"value": "es-subnet"}, | ||
| "vNetAppGatewaySubnetName": {"value": "es-app-gateway"}, | ||
| "vNetLoadBalancerIp": {"value": "10.0.0.4"}, | ||
| "vNetNewOrExisting": {"value":"new"}, | ||
| "vNetExistingResourceGroup": {"value": ""}, | ||
| "vNetNewAddressPrefix": {"value": "10.0.0.0/24"}, | ||
| "vNetNewClusterSubnetAddressPrefix": {"value": "10.0.0.0/25"}, | ||
| "vNetNewAppGatewaySubnetAddressPrefix": {"value": "10.0.0.128/28"}, | ||
| "appGatewayTier": {"value":"Standard"}, | ||
| "appGatewaySku": {"value":"Small"}, | ||
| "appGatewayCount": {"value":1}, | ||
| "appGatewayCertBlob": {"value":""}, | ||
| "appGatewayCertPassword": {"value":""}, | ||
| "appGatewayWafStatus": {"value":"Disabled"}, | ||
| "appGatewayWafMode": {"value":"Detection"}, | ||
| "userCompany": { "value": "" }, | ||
| "userEmail": { "value": "" }, | ||
| "userFirstName": { "value": "" }, | ||
| "userLastName": { "value": "" }, | ||
| "userJobTitle": { "value": "Other" }, | ||
| "userCountry": { "value": "" } | ||
| } | ||
| } |
Oops, something went wrong.