Merge branch '7.x' into backport/7.x/pr-65575

docs/apm/machine-learning.asciidoc

@@ -6,22 +6,31 @@
 <titleabbrev>Integrate with machine learning</titleabbrev>
 ++++
 
-The Machine Learning integration will initiate a new job predefined to calculate anomaly scores on transaction response times.
-The response time graph will show the expected bounds and add an annotation when the anomaly score is 75 or above.
-Jobs can be created per transaction type, and based on the average response time.
-Manage jobs in the *Machine Learning jobs management*.
+The Machine Learning integration initiates a new job predefined to calculate anomaly scores on APM transaction durations.
+Jobs can be created per transaction type, and are based on the service's average response time.
+
+After a machine learning job is created, results are shown in two places:
+
+The transaction duration graph will show the expected bounds and add an annotation when the anomaly score is 75 or above.
+
+[role="screenshot"]
+image::apm/images/apm-ml-integration.png[Example view of anomaly scores on response times in the APM app]
+
+Service maps will display a color-coded anomaly indicator based on the detected anomaly score.
 
 [role="screenshot"]
-image::apm/images/apm-ml-integration.png[Example view of anomaly scores on response times in APM app in Kibana]
+image::apm/images/apm-service-map-anomaly.png[Example view of anomaly scores on service maps in the APM app]
 
 [float]
 [[create-ml-integration]]
 === Create a new machine learning job
 
 To enable machine learning anomaly detection, first choose a service to monitor.
 Then, select **Integrations** > **Enable ML anomaly detection** and click **Create job**.
+
 That's it! After a few minutes, the job will begin calculating results;
 it might take additional time for results to appear on your graph.
+Jobs can be managed in *Machine Learning jobs management*.
 
 APM specific anomaly detection wizards are also available for certain Agents.
 See the machine learning {ml-docs}/ootb-ml-jobs-apm.html[APM anomaly detection configurations] for more information.

docs/apm/service-maps.asciidoc

@@ -9,7 +9,9 @@ Please use Chrome or Firefox if available.
 
 A service map is a real-time visual representation of the instrumented services in your application's architecture.
 It shows you how these services are connected, along with high-level metrics like average transaction duration,
-requests per minute, and errors per minute, that allow you to quickly assess the status of your services.
+requests per minute, and errors per minute.
+If enabled, service maps also integrate with machine learning--for real time health indicators based on anomaly detection scores.
+All of these features can help you to quickly and visually assess the status and health of your services.
 
 We currently surface two types of service maps:
 
@@ -52,6 +54,26 @@ Additional filters are not currently available for service maps.
 [role="screenshot"]
 image::apm/images/service-maps-java.png[Example view of service maps with Java highlighted in the APM app in Kibana]
 
+[float]
+[[service-map-anomaly-detection]]
+=== Anomaly detection with machine learning
+
+Machine learning jobs can be created to calculate anomaly scores on APM transaction durations within the selected service.
+When these jobs are active, service maps will display a color-coded anomaly indicator based on the detected anomaly score:
+
+[horizontal]
+image:apm/images/green-service.png[APM green service]:: Max anomaly score **<=25**. Service is healthy.
+image:apm/images/yellow-service.png[APM yellow service]:: Max anomaly score **26-74**. Anomalous activity detected. Service may be degraded.
+image:apm/images/red-service.png[APM red service]:: Max anomaly score **>=75**. Anomalous activity detected. Service is unhealthy.
+
+[role="screenshot"]
+image::apm/images/apm-service-map-anomaly.png[Example view of anomaly scores on service maps in the APM app]
+
+If an anomaly has been detected, click *view anomalies* to view the anomaly detection metric viewier in the Machine learning app.
+This time series analysis will display additional details on the severity and time of the detected anomalies.
+
+To learn how to create a machine learning job, see <<machine-learning-integration,machine learning integration>>.
+
 [float]
 [[service-maps-legend]]
 === Legend

docs/apm/spans.asciidoc

@@ -1,38 +1,53 @@
 [role="xpack"]
 [[spans]]
-=== Span timeline
+=== Trace sample timeline
 
-TIP: A {apm-overview-ref-v}/transaction-spans.html[span] is the duration of a single event.
-Spans are automatically captured by APM agents, and you can also define custom spans.
-Each span has a type and is defined by a different color in the timeline/waterfall visualization.
-
-The span timeline visualization is a bird's-eye view of what your application was doing while it was trying to respond to the request that came in.
+The trace sample timeline visualization is a bird's-eye view of what your application was doing while it was trying to respond to a request.
 This makes it useful for visualizing where the selected transaction spent most of its time.
 
 [role="screenshot"]
 image::apm/images/apm-transaction-sample.png[Example of distributed trace colors in the APM app in Kibana]
 
 View a span in detail by clicking on it in the timeline waterfall.
-When you click on an SQL Select database query,
+For example, when you click on an SQL Select database query,
 the information displayed includes the actual SQL that was executed, how long it took,
 and the percentage of the trace's total time.
 You also get a stack trace, which shows the SQL query in your code.
 Finally, APM knows which files are your code and which are just modules or libraries that you've installed.
 These library frames will be minimized by default in order to show you the most relevant stack trace.
 
+TIP: A {apm-overview-ref-v}/transaction-spans.html[span] is the duration of a single event.
+Spans are automatically captured by APM agents, and you can also define custom spans.
+Each span has a type and is defined by a different color in the timeline/waterfall visualization.
+
 [role="screenshot"]
 image::apm/images/apm-span-detail.png[Example view of a span detail in the APM app in Kibana]
 
-If your span timeline is colorful, it's indicative of a <<distributed-tracing,distributed trace>>.
+[float]
+[[distributed-tracing]]
+==== Distributed tracing
+
+If your trace sample timeline is colorful, it's indicative of a distributed trace.
 Services in a distributed trace are separated by color and listed in the order they occur.
 
 [role="screenshot"]
 image::apm/images/apm-services-trace.png[Example of distributed trace colors in the APM app in Kibana]
 
-Don't forget; a distributed trace includes more than one transaction.
+As application architectures are shifting from monolithic to more distributed, service-based architectures,
+distributed tracing has become a crucial feature of modern application performance monitoring.
+It allows you to trace requests through your service architecture automatically, and visualize those traces in one single view in the APM app.
+From initial web requests to your front-end service, to queries made to your back-end services,
+this makes finding possible bottlenecks throughout your application much easier and faster.
+
+[role="screenshot"]
+image::apm/images/apm-distributed-tracing.png[Example view of the distributed tracing in APM app in Kibana]
+
+Don't forget; by definition, a distributed trace includes more than one transaction.
 When viewing these distributed traces in the timeline waterfall, you'll see this image:apm/images/transaction-icon.png[APM icon] icon,
 which indicates the next transaction in the trace.
 These transactions can be expanded and viewed in detail by clicking on them.
 
 After exploring these traces,
 you can return to the full trace by clicking *View full trace*.
+
+TIP: Distributed tracing is supported by all APM agents, and there's no additional configuration needed.

docs/apm/traces.asciidoc

@@ -4,7 +4,7 @@
 
 TIP: Traces link together related transactions to show an end-to-end performance of how a request was served
 and which services were part of it.
-In addition to the Traces overview, you can view your application traces in the <<spans,span timeline waterfall>>.
+In addition to the Traces overview, you can view your application traces in the <<spans,trace sample timeline waterfall>>.
 
 The *Traces* overview displays the entry transaction for all traces in your application.
 If you're using <<distributed-tracing>>, this view is key to finding the critical paths within your application.
@@ -17,25 +17,3 @@ If there's a particular endpoint you're worried about, you can click on it to vi
 
 [role="screenshot"]
 image::apm/images/apm-traces.png[Example view of the Traces overview in APM app in Kibana]
-
-[float]
-[[distributed-tracing]]
-==== Distributed tracing
-
-Elastic APM supports distributed tracing.
-Distributed tracing is a key feature of modern application performance monitoring as application architectures are shifting from monolithic to more distributed,
-service-based architectures.
-
-Distributed tracing allows APM users to automatically trace requests all the way through the service architecture,
-and visualize those traces in one single view in the APM app.
-This is accomplished by tracing all of the requests, from the initial web request to your front-end service,
-to queries made to your back-end services.
-This makes finding possible bottlenecks throughout your application much easier and faster.
-
-By definition, a distributed trace includes more than one transaction.
-You can use the <<spans,span timeline visualization>> to view a waterfall display of all of the transactions from individual services that are connected in a trace.
-
-[role="screenshot"]
-image::apm/images/apm-distributed-tracing.png[Example view of the distributed tracing in APM app in Kibana]
-
-TIP: Distributed tracing is supported by all APM agents, and there's no additional configuration needed.

docs/apm/transactions.asciidoc

@@ -95,7 +95,7 @@ It's the requests on the right, the ones taking longer than average, that we pro
 
 When you select one of these buckets,
 you're presented with up to ten trace samples.
-Each sample has a span timeline waterfall that shows what a typical request in that bucket was doing.
+Each sample has a trace timeline waterfall that shows what a typical request in that bucket was doing.
 By investigating this timeline waterfall, we can hopefully determine _why_ this request was slow and then implement a fix.
 
 [role="screenshot"]

docs/development/plugins/data/public/kibana-plugin-plugins-data-public.indexpattern.md

@@ -29,7 +29,6 @@ export declare class IndexPattern implements IIndexPattern
 |  [id](./kibana-plugin-plugins-data-public.indexpattern.id.md) |  | <code>string</code> |  |
 |  [intervalName](./kibana-plugin-plugins-data-public.indexpattern.intervalname.md) |  | <code>string &#124; undefined &#124; null</code> |  |
 |  [metaFields](./kibana-plugin-plugins-data-public.indexpattern.metafields.md) |  | <code>string[]</code> |  |
-|  [routes](./kibana-plugin-plugins-data-public.indexpattern.routes.md) |  | <code>{</code><br/><code>        edit: string;</code><br/><code>        addField: string;</code><br/><code>        indexedFields: string;</code><br/><code>        scriptedFields: string;</code><br/><code>        sourceFilters: string;</code><br/><code>    }</code> |  |
 |  [timeFieldName](./kibana-plugin-plugins-data-public.indexpattern.timefieldname.md) |  | <code>string &#124; undefined</code> |  |
 |  [title](./kibana-plugin-plugins-data-public.indexpattern.title.md) |  | <code>string</code> |  |
 |  [type](./kibana-plugin-plugins-data-public.indexpattern.type.md) |  | <code>string</code> |  |

docs/development/plugins/data/public/kibana-plugin-plugins-data-public.indexpatterns.md

@@ -18,7 +18,6 @@ indexPatterns: {
     validate: typeof validateIndexPattern;
     getFromSavedObject: typeof getFromSavedObject;
     flattenHitWrapper: typeof flattenHitWrapper;
-    getRoutes: typeof getRoutes;
     formatHitProvider: typeof formatHitProvider;
 }
 ```

packages/kbn-dev-utils/src/precommit_hook/cli.ts

@@ -0,0 +1,50 @@
+/*
+ * Licensed to Elasticsearch B.V. under one or more contributor
+ * license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright
+ * ownership. Elasticsearch B.V. licenses this file to you under
+ * the Apache License, Version 2.0 (the "License"); you may
+ * not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import Path from 'path';
+import { chmod, writeFile } from 'fs';
+import { promisify } from 'util';
+
+import { run } from '../run';
+import { REPO_ROOT } from '../repo_root';
+import { SCRIPT_SOURCE } from './script_source';
+import { getGitDir } from './get_git_dir';
+
+const chmodAsync = promisify(chmod);
+const writeFileAsync = promisify(writeFile);
+
+run(
+  async ({ log }) => {
+    try {
+      const gitDir = await getGitDir();
+      const installPath = Path.resolve(REPO_ROOT, gitDir, 'hooks/pre-commit');
+
+      log.info(`Registering Kibana pre-commit git hook...`);
+      await writeFileAsync(installPath, SCRIPT_SOURCE);
+      await chmodAsync(installPath, 0o755);
+      log.success(`Kibana pre-commit git hook was installed successfully.`);
+    } catch (e) {
+      log.error(`Kibana pre-commit git hook was not installed as an error occur.`);
+      throw e;
+    }
+  },
+  {
+    description: 'Register git hooks in the local repo',
+  }
+);

src/dev/register_git_hook/index.js → packages/kbn-dev-utils/src/precommit_hook/get_git_dir.ts

@@ -17,4 +17,16 @@
  * under the License.
  */
 
-export { registerPrecommitGitHook } from './register_git_hook';
+import execa from 'execa';
+
+import { REPO_ROOT } from '../repo_root';
+
+// Retrieves the correct location for the .git dir for
+// every git setup (including git worktree)
+export async function getGitDir() {
+  return (
+    await execa('git', ['rev-parse', '--git-common-dir'], {
+      cwd: REPO_ROOT,
+    })
+  ).stdout.trim();
+}