From 90ca1ffcfbb4f043d10c137b75a35bca6e68a1ad Mon Sep 17 00:00:00 2001 From: jamiehynds <62879768+jamiehynds@users.noreply.github.com> Date: Thu, 27 Aug 2020 15:42:53 +0100 Subject: [PATCH 1/4] Create 0000-Data-Source-Categorization-Fields.md --- .../0000-Data-Source-Categorization-Fields.md | 73 +++++++++++++++++++ 1 file changed, 73 insertions(+) create mode 100644 rfcs/text/0000-Data-Source-Categorization-Fields.md diff --git a/rfcs/text/0000-Data-Source-Categorization-Fields.md b/rfcs/text/0000-Data-Source-Categorization-Fields.md new file mode 100644 index 0000000000..8f699baddc --- /dev/null +++ b/rfcs/text/0000-Data-Source-Categorization-Fields.md @@ -0,0 +1,73 @@ +# 0000: Data Source Categorization Fields + + +- Stage: **0 (strawperson)** +- Date: **August 26 2020** + + + + + +Elastic currently supports ingestion of data from 180+ sources, and growing. However, we do not have a coherent way to categorise these sources. This has resulted in a disconnect in how we categorize these sources from the Elastic website, in-product experiences and ECS. + +The fieldset we use to describe the data source is up for discussion, data_stream.category is a possibility. Here are proposed allowed values: + +- apm +- application +- audit +- CASB +- cloud +- collaboration +- Config Management +- containers +- CRM +- EDR +- email +- firewall +- Identity and access management +- IDS/IPS +- Operating System +- productivity +- proxy +- queue/message queue +- security +- storage +- threat intelligence +- ticketing +- VPN +- vulnerability scanner +- Web server + +## Usage +Categorization fields in ECS can govern how we categorize these data source, but only a limited set of event.category values are supported by the schema today. The event categorisation fields are catered to individual events, but don't categorise the data source. Expanding the values we support, allows us to align the user experience from ECS, Ingest Manager and the Elastic Website (elastic.co/integrations). Some additional context here: #845 (comment). + +These categories could also be used to categorise detection rules, to map data sources to corresponding rules. This would improve our onboarding experience by suggesting detection rules to users based on the sources they are ingesting data from. + + +## People + +The following are the people that consulted on the contents of this RFC. + +* @jamiehynds | author +* @exekias | sponsor + +## References + +https://github.com/elastic/ecs/issues/901 +https://github.com/elastic/ecs/pull/845 + +### RFC Pull Requests + + + +* Stage 0: https://github.com/elastic/ecs/pull/954 + + From 2155e4e503ba4ac087ea777eb6fecd52d483c7f5 Mon Sep 17 00:00:00 2001 From: Eric Beahan Date: Thu, 27 Aug 2020 13:02:29 -0500 Subject: [PATCH 2/4] update stage 0 PR reference --- rfcs/text/0000-Data-Source-Categorization-Fields.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/rfcs/text/0000-Data-Source-Categorization-Fields.md b/rfcs/text/0000-Data-Source-Categorization-Fields.md index 8f699baddc..defe7e8209 100644 --- a/rfcs/text/0000-Data-Source-Categorization-Fields.md +++ b/rfcs/text/0000-Data-Source-Categorization-Fields.md @@ -13,7 +13,7 @@ Feel free to remove these comments as you go along. Stage 0: Provide a high level summary of the premise of these changes. Briefly describe the nature, purpose, and impact of the changes. ~2-5 sentences. --> -Elastic currently supports ingestion of data from 180+ sources, and growing. However, we do not have a coherent way to categorise these sources. This has resulted in a disconnect in how we categorize these sources from the Elastic website, in-product experiences and ECS. +Elastic currently supports ingestion of data from 180+ sources, and growing. However, we do not have a coherent way to categorise these sources. This has resulted in a disconnect in how we categorize these sources from the Elastic website, in-product experiences and ECS. The fieldset we use to describe the data source is up for discussion, data_stream.category is a possibility. Here are proposed allowed values: @@ -58,14 +58,14 @@ The following are the people that consulted on the contents of this RFC. ## References -https://github.com/elastic/ecs/issues/901 -https://github.com/elastic/ecs/pull/845 +* https://github.com/elastic/ecs/issues/901 +* https://github.com/elastic/ecs/pull/845 ### RFC Pull Requests -* Stage 0: https://github.com/elastic/ecs/pull/954 +* Stage 0: https://github.com/elastic/ecs/pull/958