Skip to content

Commit

Permalink
docs: move ecs table up to ecs_metadata section, other minor fixes
Browse files Browse the repository at this point in the history
Co-authored-by: Karol Bucek <karol.bucek@elastic.co>
Co-authored-by: Karen Metts <karen.metts@elastic.co>
  • Loading branch information
3 people committed Jun 27, 2021
1 parent 4b53619 commit fe8ec28
Show file tree
Hide file tree
Showing 2 changed files with 41 additions and 42 deletions.
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## 6.2.0
- Added ECS Compatibility Mode [#165](https://github.com/logstash-plugins/logstash-input-tcp/pull/165)
- When operating in an ECS Compatibility mode, metadata about the connection on which we are receiving data is nested in well-named fields under `[@metadata][tcp]` instead of at the root level.
- When operating in an ECS Compatibility mode, metadata about the connection on which we are receiving data is nested in well-named fields under `[@metadata][input][tcp]` instead of at the root level.
- Fix: source address is no longer missing when a proxy is present

## 6.1.1
Expand Down
81 changes: 40 additions & 41 deletions docs/index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -70,15 +70,52 @@ event timestamp
}
}

[id="plugins-{type}s-{plugin}-ecs_metadata"]
==== Event Metadata and the Elastic Common Schema (ECS)

In addition to decoding the events, this input will add metadata about the TCP connection itself to each event.
This can be helpful when applications are configured to send events directly to this input's TCP listener without including information about themselves.

Historically, this metadata was added to a variety of non-standard top-level fields, which had the potential to create confusion and schema conflicts downstream.
With ECS Compatibility Mode, we can ensure a pipeline still has access to this metadata throughout the event's lifecycle without polluting the top-level namespace.
With ECS compatibility mode, we can ensure a pipeline still has access to this metadata throughout the event's lifecycle without polluting the top-level namespace.

For a complete listing of metadata shape, see <<plugins-{type}s-{plugin}-ecs_compatibility>>.
.Source Metadata Location by `ecs_compatibility` value
[cols="3,7,5"]
|=======================================================================
| Metadata Group | ecs: `v1`, `v8` | ecs: `disabled`

.3+|Source Metadata from the TCP connection
on which events are being received, including
the sender's name, ip, and outbound port. l|[@metadata][input][tcp][source][name] l|[host]
l|[@metadata][input][tcp][source][ip] l|[@metadata][ip_address]
l|[@metadata][input][tcp][source][port] l|[port]

.2+|Proxy Metadata from a proxied TCP connection.
Available when receiving events by proxy and
`proxy_protocol => true` l|[@metadata][input][tcp][proxy][ip] l|[proxy_host]
l|[@metadata][input][tcp][proxy][port] l|[proxy_port]

.1+|SSL Subject Metadata from a secured TCP
connection. Available when `ssl_enable => true`
AND `ssl_verify => true` l|[@metadata][input][tcp][ssl][subject] l|[sslsubject]
|=======================================================================

For example, the Elastic Common Schema reserves the https://www.elastic.co/guide/en/ecs/current/ecs-host.html[top-level `host` field] for information about the host on which the event happened.
If an event is missing this metadata, it can be copied into place from the source TCP connection metadata that has been added to the event:

[source,txt]
-----
filter {
if [@metadata][input][tcp][source] and not [host] {
mutate {
copy {
"[@metadata][input][tcp][source][name]" => "[host][name]"
"[@metadata][input][tcp][source][ip]" => "[host][ip]"
}
}
}
}
-----

[id="plugins-{type}s-{plugin}-options"]
==== Tcp Input Configuration Options
Expand Down Expand Up @@ -131,45 +168,7 @@ at the TCP layer and IPs will not be resolved to hostnames.
** Otherwise, the default value is `disabled`.

Controls this plugin's compatibility with the https://www.elastic.co/guide/en/ecs/current/index.html[Elastic Common Schema (ECS)].
The value of this setting affects the placement of a TCP connection's metadata on events:

.Source Metadata Location by `ecs_compatibility` value
[cols="3,7,5"]
|=======================================================================
| Metadata Group | ecs: `v1`, `v8` | ecs: `disabled`

.3+|Source Metadata from the TCP connection
on which events are being received, including
the sender's name, ip, and outbound port. l|[@metadata][input][tcp][source][name] l|[host]
l|[@metadata][input][tcp][source][ip] l|[@metadata][ip_address]
l|[@metadata][input][tcp][source][port] l|[port]

.2+|Proxy Metadata from a proxied TCP connection.
Available when receiving events by proxy and
`proxy_protocol => true` l|[@metadata][input][tcp][proxy][ip] l|[proxy_host]
l|[@metadata][input][tcp][proxy][port] l|[proxy_port]

.1+|SSL Subject Metadata from a secured TCP
connection. Available when `ssl_enable => true`
AND `ssl_verify => true` l|[@metadata][input][tcp][ssl][subject] l|[sslsubject]
|=======================================================================

For example, the Elastic Common Schema reserves the https://www.elastic.co/guide/en/ecs/current/ecs-host.html[top-level `host` field] for information about the host on which the event happened.
If an event is missing this metadata, it can be copied into place from the source TCP connection metadata that has been added to the event:

[source]
-----
filter {
if [@metadata][input][tcp][source] and not [host] {
mutate {
copy {
"[@metadata][input][tcp][source][name]" => "[host][name]"
"[@metadata][input][tcp][source][ip]" => "[host][ip]"
}
}
}
}
-----
The value of this setting affects the <<plugins-{type}s-{plugin}-ecs_metadata,placement of a TCP connection's metadata>> on events.

[id="plugins-{type}s-{plugin}-host"]
===== `host`
Expand Down

0 comments on commit fe8ec28

Please sign in to comment.