Add categorization fields usage docs (elastic#1242) (elastic#1257)

docs/field-values-usage.asciidoc

@@ -0,0 +1,175 @@
+[[ecs-using-the-categorization-fields]]
+=== Using the Categorization Fields
+
+The event categorization fields work together to identify and group similar events from multiple data sources.
+
+These general principles can help guide the categorization process:
+
+* Events from multiple data sources that are similar enough to be viewed or analyzed together, should fall into the same `event.category` field.
+* Both `event.category` and `event.type` are arrays and may be populated with multiple allowed values, if the event can be reasonably classified into more than one category and/or type.
+* `event.kind`, `event.category`, `event.type` and `event.outcome` all have allowed values. This is to normalize these fields. Values that aren't in the list of allowed values should not be used.
+* Values of `event.outcome` are a very limited set to indicate success or failure. Domain-specific actions, such as deny and allow, that could be considered outcomes are not
+  captured in the `event.outcome` field, but rather in the `event.type` and/or `event.action` fields.
+* Values of `event.category`, `event.type`, and `event.outcome` are consistent across all values of `event.kind`.
+* When a specific event doesn't fit into any of the defined allowed categorization values, the field should be left empty.
+
+The following examples detail populating the categorization fields and provides some context for the classification decisions.
+
+[float]
+==== Firewall blocking a network connection
+
+This event from a firewall describes a successfully blocked network connection:
+
+[source,json]
+----
+...
+  {
+    "source": {
+      "address": "10.42.42.42",
+      "ip": "10.42.42.42",
+      "port": 38842
+    },
+    "destination": {
+      "address": "10.42.42.1",
+      "ip": "10.42.42.1",
+      "port": 443
+    },
+    "rule": {
+      "name": "wan-lan",
+      "id": "default"
+    },
+    ...
+    "event": {
+      "kind": "event", <1>
+      "category": [ <2>
+        "network"
+      ],
+      "type": [ <3>
+        "connection",
+        "denied"
+      ],
+      "outcome": "success", <4>
+      "action": "dropped" <5>
+    }
+  }
+...
+----
+
+<1> Classifying as an `event`.
+<2> `event.category` categorizes this event as `network` activity.
+<3> The event was both an attempted network `connection` and was `denied`.
+<4> The blocking of this connection is expected. The outcome is a `success` from the perspective of the firewall emitting the event.
+<5> The firewall classifies this denied connection as `dropped`, and this value is captured in `event.action`.
+
+A "denied" network connection could fall under different action values: "blocked", "dropped", "quarantined", etc. The `event.action` field captures the action taken as described by the source, and populating `event.type:denied` provides an independent, normalized value.
+
+A single query will return all denied network connections which have been normalized with the same categorization values:
+
+[source,sh]
+----
+event.category:network AND event.type:denied
+----
+
+[float]
+==== Failed attempt to create a user account
+
+User `alice` attempts to add a user account, `bob`, into a directory service, but the action fails:
+
+[source,json]
+----
+{
+  "user": {
+    "name": "alice",
+    "target": {
+      "name": "bob"
+    }
+  },
+  "event": {
+    "kind": "event", <1>
+    "category": [ <2>
+      "iam"
+    ],
+    "type": [ <3>
+      "user",
+      "creation"
+    ],
+    "outcome": "failure" <4>
+  }
+}
+----
+
+<1> Again classifying as an `event`.
+<2> Categorized using `iam` for an event user account activity.
+<3> Both `user` and `creation`
+<4> The creation of a user account was attempted, but it was not successful.
+
+[float]
+==== Informational listing of a file
+
+A utility, such as a file integrity monitoring (FIM) application, takes inventory of a file but does not access or modify the file:
+
+[source,json]
+----
+{
+  "file": {
+    "name": "example.png",
+    "owner": "alice",
+    "path": "/home/alice/example.png",
+    "type": "file"
+  },
+  "event": {
+    "kind": "event", <1>
+    "category": [ <2>
+      "file"
+    ],
+    "type": [ <3>
+      "info"
+    ]
+  }
+}
+----
+
+<1> Classifying as `event`.
+<2> The event is reporting on a `file`.
+<3> The `info` type categorizes purely informational events. The target file here was not accessed or modified.
+
+The source data didn't include any context around the event's outcome, so `event.outcome` should not be populated.
+
+[float]
+=== Security application failed to block a network connection
+
+An intrusion detection system (IDS) attempts to block a connection but fails. The event emitted by the IDS is considered an alert:
+
+[source,json]
+----
+{
+  "source": {
+      "address": "10.42.42.42",
+      "ip": "10.42.42.42",
+      "port": 38842
+    },
+  "destination": {
+      "address": "10.42.42.1",
+      "ip": "10.42.42.1",
+      "port": 443
+  },
+  ...
+  "event": {
+    "kind": "alert", <1>
+    "category": [ <2>
+      "intrusion_detection",
+      "network"
+    ],
+    "type": [ <3>
+      "connection",
+      "denied"
+    ],
+    "outcome": "failure" <4>
+  }
+}
+----
+
+<1> The IDS emitted this event when a detection rule generated an alert. The `event.kind` is set to `alert`.
+<2> With the event emitted from a network IDS device, the event is categorized both as `network` and `intrusion_detection`.
+<3> The alert event is a `connection` that was `denied` by the IDS' configuration.
+<4> The IDS experienced an issue when attempting to deny the connection. Since the action taken by the IDS failed, the outcome is set as `failure`.

docs/field-values.asciidoc

@@ -22,6 +22,13 @@ NOTE: If your events don't match any of these categorization values, you should
 leave the fields empty. This will ensure you can start populating the fields
 once the appropriate categorization values are published, in a later release.
 
+[float]
+[[ecs-category-usage]]
+=== Categorization Usage
+
+<<ecs-using-the-categorization-fields,Using the categorization fields>> contains examples combining the categorization fields to classify different types of events.
+
+
 [[ecs-allowed-values-event-kind]]
 === ECS Categorization Field: event.kind
 
@@ -517,3 +524,6 @@ Indicates that this event describes a successful result. A common example is `ev
 Indicates that this event describes only an attempt for which the result is unknown from the perspective of the event producer. For example, if the event contains information only about the request side of a transaction that results in a response, populating `event.outcome:unknown` in the request event is appropriate. The unknown value should not be used when an outcome doesn't make logical sense for the event. In such cases `event.outcome` should not be populated.
 
 
+
+
+include::field-values-usage.asciidoc[]

scripts/templates/field_values.j2

@@ -21,6 +21,13 @@ ECS defines four categorization fields for this purpose, each of which falls und
 NOTE: If your events don't match any of these categorization values, you should
 leave the fields empty. This will ensure you can start populating the fields
 once the appropriate categorization values are published, in a later release.
+
+[float]
+[[ecs-category-usage]]
+=== Categorization Usage
+
+<<ecs-using-the-categorization-fields,Using the categorization fields>> contains examples combining the categorization fields to classify different types of events.
+
 {% for field in fields %}
 [[ecs-allowed-values-{{ field['dashed_name'] }}]]
 === ECS Categorization Field: {{ field['flat_name'] }}
@@ -45,3 +52,5 @@ once the appropriate categorization values are published, in a later release.
 {% endif %}
 {% endfor %}
 {%- endfor %}
+
+include::field-values-usage.asciidoc[]