Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[1.x] Uniformity across domain name breakdown fields (#981) #994

Merged
merged 2 commits into from
Oct 2, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ Thanks, you're awesome :-) -->

* Expanded field set definitions for `source.*` and `destination.*`. #967
* Provided better guidance for mapping network events. #969
* Added the field `.subdomain` under `client`, `destination`, `server`, `source` and `url`, to match its presence at `dns.question.subdomain`. #981

#### Deprecated

Expand Down
11 changes: 11 additions & 0 deletions code/go/ecs/client.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 11 additions & 0 deletions code/go/ecs/destination.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 11 additions & 0 deletions code/go/ecs/server.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 11 additions & 0 deletions code/go/ecs/source.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 11 additions & 0 deletions code/go/ecs/url.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

75 changes: 75 additions & 0 deletions docs/field-details.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -407,6 +407,21 @@ example: `example.com`

// ===============================================================

| client.subdomain
| The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of "www.east.mydomain.co.uk" is "east". If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period.

type: keyword



example: `east`

| extended

// ===============================================================

| client.top_level_domain
| The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com".

Expand Down Expand Up @@ -967,6 +982,21 @@ example: `example.com`

// ===============================================================

| destination.subdomain
| The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of "www.east.mydomain.co.uk" is "east". If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period.

type: keyword



example: `east`

| extended

// ===============================================================

| destination.top_level_domain
| The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com".

Expand Down Expand Up @@ -5058,6 +5088,21 @@ example: `example.com`

// ===============================================================

| server.subdomain
| The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of "www.east.mydomain.co.uk" is "east". If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period.

type: keyword



example: `east`

| extended

// ===============================================================

| server.top_level_domain
| The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com".

Expand Down Expand Up @@ -5397,6 +5442,21 @@ example: `example.com`

// ===============================================================

| source.subdomain
| The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of "www.east.mydomain.co.uk" is "east". If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period.

type: keyword



example: `east`

| extended

// ===============================================================

| source.top_level_domain
| The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com".

Expand Down Expand Up @@ -6321,6 +6381,21 @@ example: `https`

// ===============================================================

| url.subdomain
| The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of "www.east.mydomain.co.uk" is "east". If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period.

type: keyword



example: `east`

| extended

// ===============================================================

| url.top_level_domain
| The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com".

Expand Down
70 changes: 70 additions & 0 deletions generated/beats/fields.ecs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -302,6 +302,20 @@
list (http://publicsuffix.org). Trying to approximate this by simply taking
the last two labels will not work well for TLDs such as "co.uk".'
example: example.com
- name: subdomain
level: extended
type: keyword
ignore_above: 1024
description: 'The subdomain portion of a fully qualified domain name includes
all of the names except the host name under the registered_domain. In a partially
qualified domain, or if the the qualification level of the full name cannot
be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of "www.east.mydomain.co.uk" is "east".
If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com",
the subdomain field should contain "sub2.sub1", with no trailing period.'
example: east
default_field: false
- name: top_level_domain
level: extended
type: keyword
Expand Down Expand Up @@ -709,6 +723,20 @@
list (http://publicsuffix.org). Trying to approximate this by simply taking
the last two labels will not work well for TLDs such as "co.uk".'
example: example.com
- name: subdomain
level: extended
type: keyword
ignore_above: 1024
description: 'The subdomain portion of a fully qualified domain name includes
all of the names except the host name under the registered_domain. In a partially
qualified domain, or if the the qualification level of the full name cannot
be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of "www.east.mydomain.co.uk" is "east".
If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com",
the subdomain field should contain "sub2.sub1", with no trailing period.'
example: east
default_field: false
- name: top_level_domain
level: extended
type: keyword
Expand Down Expand Up @@ -4105,6 +4133,20 @@
list (http://publicsuffix.org). Trying to approximate this by simply taking
the last two labels will not work well for TLDs such as "co.uk".'
example: example.com
- name: subdomain
level: extended
type: keyword
ignore_above: 1024
description: 'The subdomain portion of a fully qualified domain name includes
all of the names except the host name under the registered_domain. In a partially
qualified domain, or if the the qualification level of the full name cannot
be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of "www.east.mydomain.co.uk" is "east".
If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com",
the subdomain field should contain "sub2.sub1", with no trailing period.'
example: east
default_field: false
- name: top_level_domain
level: extended
type: keyword
Expand Down Expand Up @@ -4427,6 +4469,20 @@
list (http://publicsuffix.org). Trying to approximate this by simply taking
the last two labels will not work well for TLDs such as "co.uk".'
example: example.com
- name: subdomain
level: extended
type: keyword
ignore_above: 1024
description: 'The subdomain portion of a fully qualified domain name includes
all of the names except the host name under the registered_domain. In a partially
qualified domain, or if the the qualification level of the full name cannot
be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of "www.east.mydomain.co.uk" is "east".
If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com",
the subdomain field should contain "sub2.sub1", with no trailing period.'
example: east
default_field: false
- name: top_level_domain
level: extended
type: keyword
Expand Down Expand Up @@ -5337,6 +5393,20 @@

Note: The `:` is not part of the scheme.'
example: https
- name: subdomain
level: extended
type: keyword
ignore_above: 1024
description: 'The subdomain portion of a fully qualified domain name includes
all of the names except the host name under the registered_domain. In a partially
qualified domain, or if the the qualification level of the full name cannot
be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of "www.east.mydomain.co.uk" is "east".
If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com",
the subdomain field should contain "sub2.sub1", with no trailing period.'
example: east
default_field: false
- name: top_level_domain
level: extended
type: keyword
Expand Down
5 changes: 5 additions & 0 deletions generated/csv/fields.csv
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description
1.7.0-dev,true,client,client.packets,long,core,,12,Packets sent from the client to the server.
1.7.0-dev,true,client,client.port,long,core,,,Port of the client.
1.7.0-dev,true,client,client.registered_domain,keyword,extended,,example.com,"The highest registered client domain, stripped of the subdomain."
1.7.0-dev,true,client,client.subdomain,keyword,extended,,east,The subdomain of the domain.
1.7.0-dev,true,client,client.top_level_domain,keyword,extended,,co.uk,"The effective top level domain (com, org, net, co.uk)."
1.7.0-dev,true,client,client.user.domain,keyword,extended,,,Name of the directory the user is a member of.
1.7.0-dev,true,client,client.user.email,keyword,extended,,,User email address.
Expand Down Expand Up @@ -80,6 +81,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description
1.7.0-dev,true,destination,destination.packets,long,core,,12,Packets sent from the destination to the source.
1.7.0-dev,true,destination,destination.port,long,core,,,Port of the destination.
1.7.0-dev,true,destination,destination.registered_domain,keyword,extended,,example.com,"The highest registered destination domain, stripped of the subdomain."
1.7.0-dev,true,destination,destination.subdomain,keyword,extended,,east,The subdomain of the domain.
1.7.0-dev,true,destination,destination.top_level_domain,keyword,extended,,co.uk,"The effective top level domain (com, org, net, co.uk)."
1.7.0-dev,true,destination,destination.user.domain,keyword,extended,,,Name of the directory the user is a member of.
1.7.0-dev,true,destination,destination.user.email,keyword,extended,,,User email address.
Expand Down Expand Up @@ -478,6 +480,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description
1.7.0-dev,true,server,server.packets,long,core,,12,Packets sent from the server to the client.
1.7.0-dev,true,server,server.port,long,core,,,Port of the server.
1.7.0-dev,true,server,server.registered_domain,keyword,extended,,example.com,"The highest registered server domain, stripped of the subdomain."
1.7.0-dev,true,server,server.subdomain,keyword,extended,,east,The subdomain of the domain.
1.7.0-dev,true,server,server.top_level_domain,keyword,extended,,co.uk,"The effective top level domain (com, org, net, co.uk)."
1.7.0-dev,true,server,server.user.domain,keyword,extended,,,Name of the directory the user is a member of.
1.7.0-dev,true,server,server.user.email,keyword,extended,,,User email address.
Expand Down Expand Up @@ -519,6 +522,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description
1.7.0-dev,true,source,source.packets,long,core,,12,Packets sent from the source to the destination.
1.7.0-dev,true,source,source.port,long,core,,,Port of the source.
1.7.0-dev,true,source,source.registered_domain,keyword,extended,,example.com,"The highest registered source domain, stripped of the subdomain."
1.7.0-dev,true,source,source.subdomain,keyword,extended,,east,The subdomain of the domain.
1.7.0-dev,true,source,source.top_level_domain,keyword,extended,,co.uk,"The effective top level domain (com, org, net, co.uk)."
1.7.0-dev,true,source,source.user.domain,keyword,extended,,,Name of the directory the user is a member of.
1.7.0-dev,true,source,source.user.email,keyword,extended,,,User email address.
Expand Down Expand Up @@ -637,6 +641,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description
1.7.0-dev,true,url,url.query,keyword,extended,,,Query string of the request.
1.7.0-dev,true,url,url.registered_domain,keyword,extended,,example.com,"The highest registered url domain, stripped of the subdomain."
1.7.0-dev,true,url,url.scheme,keyword,extended,,https,Scheme of the url.
1.7.0-dev,true,url,url.subdomain,keyword,extended,,east,The subdomain of the domain.
1.7.0-dev,true,url,url.top_level_domain,keyword,extended,,co.uk,"The effective top level domain (com, org, net, co.uk)."
1.7.0-dev,true,url,url.username,keyword,extended,,,Username of the request.
1.7.0-dev,true,user,user.domain,keyword,extended,,,Name of the directory the user is a member of.
Expand Down
Loading