Merge branch 'master' into alerting/index-field-validation

.eslintrc.js

@@ -491,6 +491,7 @@ module.exports = {
         'x-pack/dev-tools/mocha/setup_mocha.js',
         'x-pack/scripts/*.js',
       ],
+      excludedFiles: ['**/integration_tests/**/*'],
       rules: {
         'import/no-commonjs': 'off',
         'prefer-object-spread/prefer-object-spread': 'off',

docs/development/plugins/data/server/kibana-plugin-plugins-data-server.md

@@ -23,6 +23,7 @@
 |  Function | Description |
 |  --- | --- |
 |  [getDefaultSearchParams(config)](./kibana-plugin-plugins-data-server.getdefaultsearchparams.md) |  |
+|  [getTotalLoaded({ total, failed, successful })](./kibana-plugin-plugins-data-server.gettotalloaded.md) |  |
 |  [parseInterval(interval)](./kibana-plugin-plugins-data-server.parseinterval.md) |  |
 |  [plugin(initializerContext)](./kibana-plugin-plugins-data-server.plugin.md) | Static code to be shared externally |
 |  [shouldReadFieldFromDocValues(aggregatable, esType)](./kibana-plugin-plugins-data-server.shouldreadfieldfromdocvalues.md) |  |

docs/management/alerting/alert-details.asciidoc

@@ -0,0 +1,34 @@
+[role="xpack"]
+[[alert-details]]
+=== Alert details
+
+beta[]
+
+The *Alert details* page tells you about the state of the alert and provides granular control over the actions it is taking. 
+
+[role="screenshot"]
+image::images/alerts-details-instances-active.png[Alert details page with three alert instances]
+
+In this example, alerts detect when a site serves more than a threshold number of bytes in a 24 hour period. Three sites are above the threshold. These are called alert instances - occurrences of the condition being detected - and the instance name, status, time of detection, and duration of the condition are shown in this view. 
+
+Upon detection, each instance can trigger one or more actions. If the condition persists, the same actions will trigger either on the next scheduled alert check, or (if defined) after the re-notify period on the alert has passed. To prevent re-notification, you can suppress future actions by clicking on the eye icon to mute an individual alert instance. Muting means that the alert checks continue to run on a schedule, but that instance will not trigger any action.
+
+[role="screenshot"]
+image::images/alerts-details-instance-muting.png[Muting an alert instance]
+
+Alert instances will come and go from the list depending on whether they meet the alert conditions or not - unless they are muted. If a muted instance no longer meets the alert conditions, it will appear as inactive in the list. This prevents an instance from triggering actions if it reappears in the future. 
+
+[role="screenshot"]
+image::images/alerts-details-instances-inactive.png[Alert details page with three inactive alert instances]
+
+If you want to suppress actions on all current and future instances, you can mute the entire alert. Alert checks continue to run and the instance list will update as instances activate or deactivate, but no actions will be triggered.
+
+[role="screenshot"]
+image::images/alerts-details-muting.png[Use the mute toggle to suppress all action on current and future instances]
+
+You can also disable an alert altogether. When disabled, the alert stops running checks altogether and will clear any instances it is tracking. You may want to disable alerts that are not currently needed to reduce the load on {kib} and {es}.
+
+[role="screenshot"]
+image::images/alerts-details-disabling.png[Use the disable toggle to turn off alert checks and clear instances tracked]
+
+* For further information on alerting concepts and examples, see <<alerting-getting-started>>.

docs/management/alerting/alert-management.asciidoc

@@ -0,0 +1,59 @@
+[role="xpack"]
+[[alert-management]]
+=== Managing Alerts
+
+beta[]
+
+The *Alerts* tab provides a cross-app view of alerting. Different {kib} apps like <<xpack-infra, Metrics>>, <<xpack-apm, APM>>, <<xpack-uptime, Uptime>>, and <<xpack-siem, SIEM>> can offer their own alerts, and the *Alerts* tab provides a central place to:
+
+* <<create-edit-alerts, Create and edit>> alerts
+* <<controlling-alerts, Control alerts>> including enabling/disabling, muting/unmuting, and deleting
+* Drill-down to <<alert-details, alert details>>
+
+[role="screenshot"]
+image:management/alerting/images/alerts-and-actions-ui.png[Example alert listing in the Alerts and Actions UI]
+
+For more information on alerting concepts and the types of alerts and actions available, see <<alerting-getting-started>>.
+
+[float]
+==== Finding alerts
+
+The *Alerts* tab lists all alerts in the current space, including summary information about their execution frequency, tags, and type. 
+
+The *search bar* can be used to quickly find alerts by name or tag. 
+
+[role="screenshot"]
+image::images/alerts-filter-by-search.png[Filtering the alerts list using the search bar]
+
+The *type* dropdown lets you filter to a subset of alert types.
+
+[role="screenshot"]
+image::images/alerts-filter-by-type.png[Filtering the alerts list by types of alert]
+
+The *Action type* dropdown lets you filter by the type of action used in the alert. 
+
+[role="screenshot"]
+image::images/alerts-filter-by-action-type.png[Filtering the alert list by type of action]
+
+[float]
+[[create-edit-alerts]]
+==== Creating and editing alerts
+
+Many alerts must be created within the context of a {kib} app like <<xpack-infra, Metrics>>, <<xpack-apm, APM>>, or <<xpack-uptime, Uptime>>, but others are generic. Generic alert types can be created in the *Alerts* management UI by clicking the *Create* button. This will launch a flyout that guides you through selecting an alert type and configuring it's properties. Refer to <<alert-types>> for details on what types of alerts are available and how to configure them. 
+
+After an alert is created, you can re-open the flyout and change an alerts properties by clicking the *Edit* button shown on each row of the alert listing. 
+
+
+[float]
+[[controlling-alerts]]
+==== Controlling alerts
+
+The alert listing allows you to quickly mute/unmute, disable/enable, and delete individual alerts by clicking the action button at the right of each row. 
+
+[role="screenshot"]
+image:management/alerting/images/individual-mute-disable.png[The actions button allows an individual alert to be muted, disabled, or deleted]
+
+These operations can also be performed in bulk by multi-selecting alerts and clicking the *Manage alerts* button:
+
+[role="screenshot"]
+image:management/alerting/images/bulk-mute-disable.png[The Manage alerts button lets you mute/unmute, enable/disable, and delete in bulk]

docs/management/alerting/alerts-and-actions-intro.asciidoc

@@ -0,0 +1,25 @@
+[role="xpack"]
+[[managing-alerts-and-actions]]
+== Alerts and Actions
+
+beta[]
+
+The *Alerts and Actions* UI lets you <<alert-management, see and control all the alerts>> in a space, and provides tools to <<connector-management, create and manage connectors>> so that alerts can trigger actions like notification, indexing, and ticketing. 
+
+To manage alerting and connectors, go to *Management > {kib} > Alerts and Actions*.
+
+[role="screenshot"]
+image:management/alerting/images/alerts-and-actions-ui.png[Example alert listing in the Alerts and Actions UI]
+
+[NOTE]
+============================================================================
+Similar to dashboards, alerts and connectors reside in a <<xpack-spaces, space>>.
+The *Alerts and Actions* UI only shows alerts and connectors for the current space.  
+============================================================================
+
+[NOTE]
+============================================================================
+{es} also offers alerting capabilities through Watcher, which
+can be managed through the <<watcher-ui, Watcher UI>>. See  
+<<alerting-concepts-differences>> for more information.
+============================================================================

docs/management/alerting/connector-management.asciidoc

@@ -0,0 +1,47 @@
+[role="xpack"]
+[[connector-management]]
+=== Managing Connectors
+
+beta[]
+
+Alerts use *Connectors* to route actions to different destinations like log files, ticketing systems, and messaging tools. While each {kib} app can offer their own types of alerts, they typically share connectors. The *Connectors* tab offers a central place to view and manage all the connectors in the current space.
+
+For more information on connectors and the types of actions available see <<action-types>>.
+
+[role="screenshot"]
+image::images/connector-listing.png[Example connector listing in the Alerts and Actions UI]
+
+
+[float]
+==== Connector list
+
+The *Connectors* tab lists all connectors in the current space. The *search bar* can be used to find specific connectors by name and/or type. 
+
+[role="screenshot"]
+image::images/connector-filter-by-search.png[Filtering the connector list using the search bar]
+
+
+The *type* dropdown also lets you filter to a subset of action types.
+
+[role="screenshot"]
+image::images/connector-filter-by-type.png[Filtering the connector list by types of actions]
+
+The *Actions* column indicates the number of actions that reference the connector. This count helps you confirm a connector is unused before you delete it, and tells you how many actions will be affected when a connector is modified. 
+
+[role="screenshot"]
+image::images/connector-action-count.png[Filtering the connector list by types of actions]
+
+You can delete individual connectors using the trash icon on the right of each row. Connectors can also be deleted in bulk by multi-selecting them and clicking the *Delete* button to the left of the search box. 
+
+[role="screenshot"]
+image::images/connector-delete.png[Deleting connectors individually or in bulk]
+
+[NOTE]
+============================================================================
+You can delete a connector even if there are still actions referencing it.
+When this happens the action will fail to execute, and appear as errors in the {kib} logs.
+============================================================================
+
+==== Creating a new connector
+
+New connectors can be created by clicking the *Create connector* button, which will guide you to select the type of connector and configure it's properties. Refer to <<action-types>> for the types of connectors available and how to configure them. Once you create a connector it will be made available to you anytime you set up an action in the current space.

docs/settings/alert-action-settings.asciidoc

@@ -27,7 +27,7 @@ If not set, {kib} will generate a random key on startup, but all alert and actio
 Although the key can be specified in clear text in `kibana.yml`, it's recommended to store this key securely in the <<secure-settings,{kib} Keystore>>.
 
 [float]
-[[alert-settings]]
+[[action-settings]]
 ==== Action settings
 
 `xpack.actions.whitelistedHosts`::
@@ -41,7 +41,7 @@ A list of action types that are enabled. It defaults to `[*]`, enabling all type
 Disabled action types will not appear as an option when creating new connectors, but existing connectors and actions of that type will remain in {kib} and will not function.  
 
 [float]
-[[action-settings]]
+[[alert-settings]]
 ==== Alert settings
 
 You do not need to configure any additional settings to use alerting in {kib}.

docs/user/alerting/action-types.asciidoc

@@ -0,0 +1,182 @@
+[role="xpack"]
+[[action-types]]
+== Action and connector types
+
+{kib} provides the following types of actions:
+
+* <<email-action-type, Email>>
+* <<index-action-type, Index>>
+* <<pagerduty-action-type, PagerDuty>>
+* <<server-log-action-type, ServerLog>>
+* <<slack-action-type, Slack>>
+* <<webhook-action-type, Webhook>>
+
+This section describes how to configure connectors and actions for each type. 
+
+[NOTE]
+==============================================
+Some action types are paid commercial features, while others are free. 
+For a comparison of the Elastic license levels, 
+see https://www.elastic.co/subscriptions[the subscription page]. 
+==============================================
+
+[float]
+[[email-action-type]]
+=== Email
+
+The email action type uses the SMTP protocol to send mail message, using an integration of https://nodemailer.com/[Nodemailer]. Email message text is sent as both plain text and html text.
+
+[float]
+[[email-connector-configuration]]
+==== Connector configuration
+
+Email connectors have the following configuration properties:
+
+Name::      The name of the connector. The name is used to identify a  connector in the management UI connector listing, or in the connector list when configuring an action. 
+Sender::    The from address for all emails sent with this connector, specified in `user@host-name` format.
+Host::      Host name of the service provider. If you are using the <<action-settings, `xpack.actions.whitelistedHosts`>> setting, make sure this hostname is whitelisted. 
+Port::      The port to connect to on the service provider.
+Secure::    If true the connection will use TLS when connecting to the service provider. See https://nodemailer.com/smtp/#tls-options[nodemailer TLS documentation] for more information.  
+Username::  username for 'login' type authentication.
+Password::  password for 'login' type authentication.
+
+[float]
+[[email-action-configuration]]
+==== Action configuration
+
+Email actions have the following configuration properties:
+
+To, CC, BCC::    Each is a list of addresses. Addresses can be specified in `user@host-name` format, or in `name <user@host-name>` format. One of To, CC, or BCC must contain an entry.
+Subject::       The subject line of the email.
+Message::       The message text of the email. Markdown format is supported.
+
+[float]
+[[index-action-type]]
+=== Index
+
+The index action type will index a document into {es}.
+
+[float]
+[[index-connector-configuration]]
+==== Connector configuration
+
+Index connectors have the following configuration properties:
+
+Name::      The name of the connector. The name is used to identify a  connector in the management UI connector listing, or in the connector list when configuring an action.
+Index::     The {es} index to be written to.
+Refresh::   Setting for the {ref}/docs-refresh.html[refresh] policy for the write request.
+Execution time field::  This field will be automatically set to the time the alert condition was detected.
+
+[float]
+[[index-action-configuration]]
+==== Action configuration
+
+Index actions have the following properties:
+
+Document::  The document to index in json format.
+
+[float]
+[[pagerduty-action-type]]
+=== PagerDuty
+
+The PagerDuty action type uses the https://v2.developer.pagerduty.com/docs/events-api-v2[v2 Events API] to trigger, acknowledge, and resolve PagerDuty alerts.
+
+[float]
+[[pagerduty-connector-configuration]]
+==== Connector configuration
+
+PagerDuty connectors have the following configuration properties:
+
+Name::      The name of the connector. The name is used to identify a  connector in the management UI connector listing, or in the connector list when configuring an action.
+API URL::   An optional PagerDuty event URL. Defaults to `https://events.pagerduty.com/v2/enqueue`. If you are using the <<action-settings, `xpack.actions.whitelistedHosts`>> setting, make sure the hostname is whitelisted. 
+Routing Key::   A 32 character PagerDuty Integration Key for an integration on a service or on a global ruleset.
+
+[float]
+[[pagerduty-action-configuration]]
+==== Action configuration
+
+PagerDuty actions have the following properties:
+
+Severity::      The perceived severity of on the affected system. This can be one of `Critical`, `Error`, `Warning` or `Info`(default).
+Event action::  One of `Trigger` (default), `Resolve`, or `Acknowledge`. See https://v2.developer.pagerduty.com/docs/events-api-v2#event-action[event action] for more details.
+Dedup Key::     All actions sharing this key will be associated with the same PagerDuty alert. This value is used to correlate trigger and resolution. This value is *optional*, and if unset defaults to `action:<action saved object id>`. The maximum length is *255* characters. See https://v2.developer.pagerduty.com/docs/events-api-v2#alert-de-duplication[alert deduplication] for details.
+Timestamp::     An *optional* https://v2.developer.pagerduty.com/v2/docs/types#datetime[ISO-8601 format date-time], indicating the time the event was detected or generated.
+Component::     An *optional* value indicating the component of the source machine that is responsible for the event, for example `mysql` or `eth0`.
+Group::         An *optional* value indicating the logical grouping of components of a service, for example `app-stack`.
+Source::        An *optional* value indicating the affected system, preferably a hostname or fully qualified domain name. Defaults to the {kib} saved object id of the action. 
+Summary::       An *optional* text summary of the event, defaults to `No summary provided`. The maximum length is 1024 characters.
+Class::         An *optional* value indicating the class/type of the event, for example `ping failure` or `cpu load`.
+
+For more details on these properties, see https://v2.developer.pagerduty.com/v2/docs/send-an-event-events-api-v2[PagerDuty v2 event parameters].
+
+[float]
+[[server-log-action-type]]
+=== Server log
+
+This action type writes and entry to the {kib} server log.
+
+[float]
+[[server-log-connector-configuration]]
+==== Connector configuration
+
+Server log connectors have the following configuration properties:
+
+Name::      The name of the connector. The name is used to identify a  connector in the management UI connector listing, or in the connector list when configuring an action.
+
+[float]
+[[server-log-action-configuration]]
+==== Action configuration
+
+Server log actions have the following properties:
+
+Message::   The message to log. 
+
+[float]
+[[slack-action-type]]
+=== Slack
+
+The Slack action type uses https://api.slack.com/incoming-webhooks[Slack Incoming Webhooks]. 
+
+[float]
+[[slack-connector-configuration]]
+==== Connector configuration
+
+Slack connectors have the following configuration properties:
+
+Name::      The name of the connector. The name is used to identify a  connector in the management UI connector listing, or in the connector list when configuring an action.
+Webhook URL::   The URL of the incoming webhook. See https://api.slack.com/messaging/webhooks#getting_started[Slack Incoming Webhooks] for instructions on generating this URL. If you are using the <<action-settings, `xpack.actions.whitelistedHosts`>> setting, make sure the hostname is whitelisted. 
+
+[float]
+[[slack-action-configuration]]
+==== Action configuration
+
+Slack actions have the following properties:
+
+Message::   The message	text, converted to the `text` field in the Webhook JSON payload. Currently only the text field is supported. Markdown, images, and other advanced formatting are not yet supported.
+
+[float]
+[[webhook-action-type]]
+=== Webhook
+
+The Webhook action type uses https://github.com/axios/axios[axios] to send a POST or PUT request to a web service.
+
+[float]
+[[webhook-connector-configuration]]
+==== Connector configuration
+
+Webhook connectors have the following configuration properties:
+
+Name::      The name of the connector. The name is used to identify a  connector in the management UI connector listing, or in the connector list when configuring an action.
+URL::       The request URL. If you are using the <<action-settings, `xpack.actions.whitelistedHosts`>> setting, make sure the hostname is whitelisted. 
+Method::    HTTP request method, either `post`(default) or `put`.
+Headers::   A set of key-value pairs sent as headers with the request
+User::      An optional username. If set, HTTP basic authentication is used. Currently only basic authentication is supported.
+Password::  An optional password. If set, HTTP basic authentication is used. Currently only basic authentication is supported.
+
+[float]
+[[webhook-action-configuration]]
+==== Action configuration
+
+Webhook actions have the following properties:
+
+Body::      A json payload sent to the request URL.