Merge remote-tracking branch 'upstream/main' into kbn-107678-fail-sta…

…rt-on-unknown-so-types

.eslintrc.js

@@ -850,6 +850,10 @@ module.exports = {
                 name: 'semver',
                 message: 'Please use "semver/*/{function}" instead',
               },
+              {
+                name: '@kbn/rule-data-utils',
+                message: `Import directly from @kbn/rule-data-utils/* submodules in public/common code`,
+              },
             ],
           },
         ],

.github/CODEOWNERS

@@ -397,7 +397,7 @@
 # Security Solution sub teams
 /x-pack/plugins/cases @elastic/security-threat-hunting
 /x-pack/plugins/timelines @elastic/security-threat-hunting
-/x-pack/test/case_api_integration @elastic/security-threat-hunting
+/x-pack/test/cases_api_integration @elastic/security-threat-hunting
 /x-pack/plugins/lists @elastic/security-detections-response
 
 ## Security Solution sub teams - security-onboarding-and-lifecycle-mgt
@@ -415,11 +415,15 @@
 /x-pack/plugins/security_solution/scripts/endpoint/trusted_apps/ @elastic/security-onboarding-and-lifecycle-mgt
 /x-pack/test/security_solution_endpoint/apps/endpoint/ @elastic/security-onboarding-and-lifecycle-mgt
 
+## Security Solution sub teams - security-telemetry (Data Engineering)
+x-pack/plugins/security_solution/server/usage/ @elastic/security-telemetry
+x-pack/plugins/security_solution/server/lib/telemetry/ @elastic/security-telemetry
+
 ## Security Solution sub teams - security-engineering-productivity
-x-pack/plugins/security_solution/cypress/ccs_integration
-x-pack/plugins/security_solution/cypress/upgrade_integration
-x-pack/plugins/security_solution/cypress/README.md
-x-pack/test/security_solution_cypress
+x-pack/plugins/security_solution/cypress/ccs_integration @elastic/security-engineering-productivity
+x-pack/plugins/security_solution/cypress/upgrade_integration @elastic/security-engineering-productivity
+x-pack/plugins/security_solution/cypress/README.md @elastic/security-engineering-productivity
+x-pack/test/security_solution_cypress @elastic/security-engineering-productivity
 
 # Security Intelligence And Analytics
 /x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules @elastic/security-intelligence-analytics

dev_docs/contributing/how_we_use_github.mdx

@@ -46,7 +46,7 @@ In order to assist with developer tooling we ask that all Elastic engineers use
  1. Update the git config for your current repository to commit with your `@elastic.co` email:
 
     ```bash
-    git config --local user.email YOUR_ELASTIC_EMAIL@elastic.co
+    git config user.email YOUR_ELASTIC_EMAIL@elastic.co
     ```
 
  1. Create a commit using the new email address

docs/apm/correlations.asciidoc

@@ -38,7 +38,7 @@ in a table below the chart. The table is sorted by correlation coefficients that
 range from 0 to 1. Attributes with higher correlation values are more likely to
 contribute to high latency transactions. By default, the attribute with the
 highest correlation value is added to the chart. To see the latency distribution
-for other attributes, hover over their row in the table.
+for other attributes, select their row in the table.
 
 If a correlated attribute seems noteworthy, use the **Filter** quick links:
 
@@ -47,12 +47,14 @@ the selected value.
 * `-` creates a new query in the {apm-app} to filter out transactions containing
 the selected value.
 
-In this example screenshot, transactions with the field
-`labels.orderPriceRange` and value `large` are skewed to the right with slower
-response times than the overall latency distribution. If you select the `+`
-filter in the appropriate row of the table, it creates a new query in the
-{apm-app} for transactions with this attribute. With the "noise" now filtered
-out, you can begin viewing sample traces to continue your investigation.
+You can also click the icon beside the field name to view and filter its most
+popular values.
+
+In this example screenshot, there are transactions that are skewed to the right
+with slower response times than the overall latency distribution. If you select
+the `+` filter in the appropriate row of the table, it creates a new query in
+the {apm-app} for transactions with this attribute. With the "noise" now
+filtered out, you can begin viewing sample traces to continue your investigation.
 
 [discrete]
 [[correlations-error-rate]]
@@ -67,25 +69,23 @@ is determined by its {ecs-ref}/ecs-event.html#field-event-outcome[event.outcome]
 value. For example, APM agents set the `event.outcome` to `failure` when an HTTP
 transaction returns a `5xx` status code.
 
-// The chart highlights the failed transactions in the overall latency distribution for the transaction group. 
-If there are attributes that have a statistically significant correlation with
-failed transactions, they are listed in a table. The table is sorted by scores,
-which are mapped to high, medium, or low impact levels. Attributes with high
-impact levels are more likely to contribute to failed transactions.
-//  By default, the attribute with the highest score is added to the chart. To see a different attribute in the chart, hover over its row in the table.
+The chart highlights the failed transactions in the overall latency distribution
+for the transaction group. If there are attributes that have a statistically
+significant correlation with failed transactions, they are listed in a table.
+The table is sorted by scores, which are mapped to high, medium, or low impact
+levels. Attributes with high impact levels are more likely to contribute to
+failed transactions. By default, the attribute with the highest score is added
+to the chart. To see a different attribute in the chart, select its row in the
+table. 
 
-For example, in the screenshot below, the field
-`kubernetes.pod.name` and value `frontend-node-59dff47885-fl5lb` has a medium
-impact level and existed in 19% of the failed transactions.
+For example, in the screenshot below, there are attributes such as a specific
+node and pod name that have medium impact on the failed transactions.
 
 [role="screenshot"]
 image::apm/images/correlations-failed-transactions.png[Failed transaction correlations]
 
-TIP: Some details, such as the failure and success percentages, are available
-only when the
-<<observability-enable-inspect-es-queries,observability:enableInspectEsQueries>>
-advanced setting is enabled.
-
 Select the `+` filter to create a new query in the {apm-app} for transactions
-with this attribute. You might do his for multiple attributes--each time
-filtering out more and more noise and bringing you closer to a diagnosis.
+with one or more of these attributes. If you are unfamiliar with a field, click
+the icon beside its name to view its most popular values and optionally filter
+on those values too. Each time that you add another attribute, it is filtering
+out more and more noise and bringing you closer to a diagnosis.

docs/developer/contributing/development-github.asciidoc

@@ -53,7 +53,7 @@ In order to assist with developer tooling we ask that all Elastic engineers use
 +
 ["source","shell"]
 -----------
-git config --local user.email YOUR_ELASTIC_EMAIL@elastic.co
+git config user.email YOUR_ELASTIC_EMAIL@elastic.co
 -----------
  4. Create a commit using the new email address
 +

docs/settings/alert-action-settings.asciidoc

@@ -191,3 +191,6 @@ Specifies the default timeout for the all rule types tasks. The time is formatte
 `<count>[ms,s,m,h,d,w,M,Y]` 
 +
 For example, `20m`, `24h`, `7d`, `1w`. Default: `60s`.
+
+`xpack.alerting.cancelAlertsOnRuleTimeout`::
+Specifies whether to skip writing alerts and scheduling actions if rule execution is cancelled due to timeout. Default: `true`. This setting can be overridden by individual rule types.

docs/user/production-considerations/security-production-considerations.asciidoc

@@ -17,7 +17,10 @@ For more information on {kib}'s security controls, see <<using-kibana-with-secur
 === Enable SSL/TLS
 
 You should use SSL/TLS encryption to ensure that traffic between browsers and the {kib} server cannot be viewed or tampered with by third
-parties. See <<configuring-tls>>.
+parties. See 
+{ref}/security-basic-setup-https.html#encrypt-kibana-http[encrypt HTTP client communications for {kib}].
+
+encrypt-kibana-http
 
 [float]
 [[configuring-kibana-shield]]

docs/user/security/securing-communications/elasticsearch-mutual-tls.asciidoc

@@ -5,36 +5,42 @@
 <titleabbrev>Mutual TLS with {es}</titleabbrev>
 ++++
 
-Secure Sockets Layer (SSL) and Transport Layer Security (TLS) provide encryption for data-in-transit. While these terms are often used
-interchangeably, {kib} supports only TLS, which supersedes the old SSL protocols.
-
-TLS requires X.509 certificates to authenticate the communicating parties and perform encryption of data-in-transit. Each certificate
-contains a public key and has and an associated -- but separate -- private key; these keys are used for cryptographic operations. {kib}
-supports certificates and private keys in PEM or PKCS#12 format.
-
-In a standard TLS configuration, the server presents a signed certificate to authenticate itself to the client. In a mutual TLS
-configuration, the client also presents a signed certificate to authenticate itself to the server.
-
-When {es} {security-features} is enabled on your cluster, each request that {kib} (the client) makes to {es} (the server) must be
-authenticated. Most requests made by end users through {kib} to {es} are authenticated by using the credentials of the logged-in user. There
-are, however, a few internal requests that {kib} needs to make to {es}. For this reason, you must configure credentials for {kib} to use for
-those requests.
-
-If {kib} has `elasticsearch.username` and `elasticsearch.password` configured, it will attempt to use these to authenticate to {es} via the
-{ref}/native-realm.html[native realm]. However, {kib} also supports mutual TLS authentication with {es} via a {ref}/pki-realm.html[Public
-Key Infrastructure (PKI) realm]. To do so, {es} needs to verify the signature on the {kib} client certificate, and it also needs to map the
-client certificate's distinguished name (DN) to the appropriate `kibana_system` role.
-
-NOTE: Using a PKI realm is a gold feature. For a comparison of the Elastic license levels, see https://www.elastic.co/subscriptions[the
-subscription page].
-
-To configure {kib} and {es} to use mutual TLS authentication:
-
-. <<using-kibana-with-security,Set up {kib} to work with {stack} {security-features} with a username and password>>.
-
-. <<configuring-tls-kib-es,Set up TLS encryption between {kib} and {es}>>.
-+
-This entails generating a "server certificate" for {es} to use on the HTTP layer.
+Secure Sockets Layer (SSL) and Transport Layer Security (TLS) provide encryption
+for data-in-transit. While these terms are often used interchangeably, {kib}
+supports only TLS, which supersedes the old SSL protocols.
+
+TLS requires X.509 certificates to authenticate the communicating parties and
+perform encryption of data-in-transit. Each certificate contains a public key
+and has and an associated -- but separate -- private key; these keys are used
+for cryptographic operations. {kib} supports certificates and private keys in
+PEM or PKCS#12 format.
+
+In a standard TLS configuration, the server presents a signed certificate to
+authenticate itself to the client. In a mutual TLS configuration, the client
+also presents a signed certificate to authenticate itself to the server.
+
+{es} {security-features} are enabled on your cluster by default, so each request
+that {kib} (the client) makes to {es} (the server) is authenticated. Most
+requests made by end users through {kib} to {es} are authenticated by using the
+credentials of the logged-in user.
+
+To {ref}/configuring-stack-security.html#stack-start-with-security[enroll {kib} with an {es} cluster], you pass a generated enrollment token. This token 
+configures {kib} to authenticate with {es} using a
+{ref}/service-accounts.html#service-accounts-tokens[service account token].
+{kib} also supports mutual TLS authentication with {es} via a
+{ref}/pki-realm.html[Public Key Infrastructure (PKI) realm]. With this setup,
+{es} needs to verify the signature on the {kib} client certificate, and it also
+needs to map the client certificate's distinguished name (DN) to the appropriate
+`kibana_system` role.
+
+NOTE: Using a PKI realm is a gold feature. For a comparison of the Elastic
+license levels, see https://www.elastic.co/subscriptions[the subscription page].
+
+[discrete]
+==== Configure {kib} and {es} to use mutual TLS authentication
+
+If you haven't already, start {kib} and connect it to {es} using the
+{ref}/configuring-stack-security.html#stack-start-with-security[enrollment token].
 
 . Obtain a client certificate and private key for {kib}.
 +
@@ -43,15 +49,14 @@ This entails generating a "server certificate" for {es} to use on the HTTP layer
 
 NOTE: This is not the same as the <<configuring-tls-browser-kib,server certificate>> that {kib} will present to web browsers.
 
-You may choose to generate a client certificate and private key using the {ref}/certutil.html[`elasticsearch-certutil`] tool. If you
-followed the {es} documentation for {ref}/configuring-tls.html#node-certificates[generating node certificates], then you likely have already
-set up a certificate authority (CA) to sign the {es} server certificate. You may choose to use the same CA to sign the {kib} client
-certificate. For example:
+You may choose to generate a client certificate and private key using the {ref}/certutil.html[`elasticsearch-certutil`] tool. If you followed the {es} documentation for {ref}/security-basic-setup.html#generate-certificates[generating the certificates authority], then you already have a certificate authority (CA) to sign
+the {es} server certificate. You may choose to use the same CA to sign the {kib}
+client certificate. For example:
 
 [source,sh]
---------------------------------------------------------------------------------
+----
 bin/elasticsearch-certutil cert -ca elastic-stack-ca.p12 -name kibana-client -dns <your_kibana_hostname>
---------------------------------------------------------------------------------
+----
 
 This will generate a client certificate and private key in a PKCS#12 file named `kibana-client.p12`. In this example, the client certificate
 has a Common Name (CN) of `"kibana-client"` and a subject alternative name (SAN) of `"<your_kibana_hostname>"`. The SAN may be required if
@@ -67,9 +72,9 @@ If you followed the instructions to generate a client certificate, then you will
 certificate chain from this file. For example:
 
 [source,sh]
---------------------------------------------------------------------------------
+----
 openssl pkcs12 -in kibana-client.p12 -cacerts -nokeys -out kibana-ca.crt
---------------------------------------------------------------------------------
+----
 
 This will produce a PEM-formatted file named `kibana-ca.crt` that contains the CA certificate from the PKCS#12 file.
 --
@@ -81,11 +86,11 @@ By default, {es} provides a native realm for authenticating with a username and
 and a native realm (for end users), you must configure each realm in `elasticsearch.yml`:
 
 [source,yaml]
---------------------------------------------------------------------------------
+----
 xpack.security.authc.realms.pki.realm1.order: 1
 xpack.security.authc.realms.pki.realm1.certificate_authorities: "/path/to/kibana-ca.crt"
 xpack.security.authc.realms.native.realm2.order: 2
---------------------------------------------------------------------------------
+----
 --
 
 . Configure {es} to request client certificates.
@@ -95,9 +100,9 @@ By default, {es} will not request a client certificate when establishing a TLS c
 certificate authentication in `elasticsearch.yml`:
 
 [source,yaml]
---------------------------------------------------------------------------------
+----
 xpack.security.http.ssl.client_authentication: "optional"
---------------------------------------------------------------------------------
+----
 --
 
 . Restart {es}.
@@ -124,16 +129,16 @@ You need to specify the information required to access your client certificate a
 Specify your PKCS#12 file in `kibana.yml`:
 
 [source,yaml]
---------------------------------------------------------------------------------
+----
 elasticsearch.ssl.keystore.path: "/path/to/kibana-client.p12"
---------------------------------------------------------------------------------
+----
 
 If your PKCS#12 file is encrypted, add the decryption password to your <<secure-settings,{kib} keystore>>:
 
 [source,yaml]
---------------------------------------------------------------------------------
+----
 bin/kibana-keystore add elasticsearch.ssl.keystore.password
---------------------------------------------------------------------------------
+----
 
 TIP: If your PKCS#12 file isn't protected with a password, depending on how it was generated, you may need to set
 `elasticsearch.ssl.keystore.password` to an empty string.
@@ -145,17 +150,17 @@ TIP: If your PKCS#12 file isn't protected with a password, depending on how it w
 Specify your certificate and private key in `kibana.yml`:
 
 [source,yaml]
---------------------------------------------------------------------------------
+----
 elasticsearch.ssl.certificate: "/path/to/kibana-client.crt"
 elasticsearch.ssl.key: "/path/to/kibana-client.key"
---------------------------------------------------------------------------------
+----
 
 If your private key is encrypted, add the decryption password to your <<secure-settings,{kib} keystore>>:
 
 [source,yaml]
---------------------------------------------------------------------------------
+----
 bin/kibana-keystore add elasticsearch.ssl.keyPassphrase
---------------------------------------------------------------------------------
+----
 --
 
 . Configure {kib} _not_ to use a username and password for {es}.