harm: remove type defacement

merged into information-content-security > unauthorised-information-modification adapt bots depending on the name add changelog and news entries, including SQL update statements
certtools · Jun 17, 2021 · 2d47b2f · 2d47b2f
1 parent 28abba8
commit 2d47b2f
Show file tree

Hide file tree

Showing 21 changed files with 30 additions and 26 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -49,6 +49,7 @@ Update allowed classification fields to 2020-01-28 version (#1409, #1476).
   - The type `compromised` has been renamed to `system-compromise`.
   - The type `unauthorized-command` has been merged into `system-compromise`.
   - The type `unauthorized-login` has been merged into `system-compromise`.
+  - The type `defacement` has been merged into taxonomy `information-content-security`, type `unauthorised-information-modification` (PR#1994 by Sebastian Wagner, addresses #1409).
 - The taxonomy `information gathering` has been rename to `information-gathering`.
 - The taxonomy `malicious code` has been renamed to `malicious-code`.
   - The type `c2server` has been renamed to `c2-server`.

diff --git a/NEWS.md b/NEWS.md
@@ -39,7 +39,7 @@ New features:
 The [sieve bot documentation](https://intelmq.readthedocs.io/en/maintenance/user/bots.html#intelmq-bots-experts-sieve-expert) has been updated to reflect on these new changes.
 
 ### Harmonization
-The classification scheme has been updated to better match the [Reference Security Incident Taxonomy (RSIT)](https://github.com/enisaeu/Reference-Security-Incident-Taxonomy-Task-Force/). The following labels were renamed:
+The classification scheme has been updated to better match the [Reference Security Incident Taxonomy (RSIT)](https://github.com/enisaeu/Reference-Security-Incident-Taxonomy-Task-Force/). The following labels were renamed, deleted or merged into others:
 
 | old taxonomy name | old type name | new taxonomy name | new type name |
 |-|-|-|-|
@@ -49,6 +49,7 @@ The classification scheme has been updated to better match the [Reference Securi
 | intrusion attempts           |                    | intrusion-attempts           |                                        |
 | information gathering        |                    | information-gathering        |                                        |
 | intrusions                   | compromised        | intrusions                   | system-compromise                      |
+| intrusions                   | defacement         | information-content-security | unauthorised-information-modification  |
 | intrusions                   | unauthorized-login | intrusions                   | system-compromise                      |
 | intrusions                   | unauthorized-command | intrusions                 | system-compromise                      |
 | malicious code               |                    | malicious-code               |                                        |
@@ -107,6 +108,9 @@ UPDATE events
 UPDATE events
    SET "classification.type" = 'system-compromise'
    WHERE "classification.type" IN ('compromised', 'unauthorized-login', 'unauthorized-command');
+UPDATE events
+   SET "classification.taxonomy" = 'information-content-security', "classification.type" = 'unauthorised-information-modification'
+   WHERE "classification.taxonomy" = 'intrusions', "classification.type" = 'defacement'
 UPDATE events
    SET "classification.taxonomy" = 'malicious-code'
    WHERE "classification.taxonomy" = 'malicious code';

diff --git a/docs/dev/data-format.rst b/docs/dev/data-format.rst
@@ -130,7 +130,6 @@ The taxonomy can be automatically added by the taxonomy expert bot based on the
    intrusions                    application-compromise                      Compromise of an application by exploiting (un)known software vulnerabilities, e.g. SQL injection.
    intrusions                    backdoor                                    This refers to hosts, which have been compromised and backdoored with a remote administration software or Trojan in the traditional sense. Not in ENISA eCSIRT-II taxonomy.
    intrusions                    burglary                                    Physical intrusion, e.g. into corporate building or data center.
-   intrusions                    defacement                                  This IOC refers to hacktivism related activity. Not in ENISA eCSIRT-II taxonomy.
    intrusions                    privileged-account-compromise               Compromise of a system where the attacker gained administrative privileges.
    intrusions                    system-compromise                           Compromise of a system, e.g. unauthorised logins or commands. This includes compromising attempts on honeypot systems.
    intrusions                    unprivileged-account-compromise             Compromise of a system using an unprivileged (user/service) account.
@@ -166,7 +165,6 @@ Meaning of source, destination and local values for each classification type and
  brute-force              *attacker*                                        target
  c2-server                *(sinkholed) c&c server*                                                      zeus, palevo, feodo
  ddos                     *attacker*                                        target
- defacement               *defaced website*
  dga-domain               *infected device*
  dropzone                 *server hosting stolen data*
  exploit                  *hosting server*

diff --git a/intelmq/bots/experts/idea/expert.py b/intelmq/bots/experts/idea/expert.py
@@ -42,7 +42,6 @@ class IdeaExpertBot(Bot):
         "exploit": "Attempt.Exploit",
         "brute-force": "Attempt.Login",
         "ids-alert": "Attempt.Exploit",
-        "defacement": "Intrusion.AppCompromise",
         "system-compromise": "Intrusion.AdminCompromise",
         "backdoor": "Intrusion.AdminCompromise",
         "blacklist": "Other",

diff --git a/intelmq/bots/experts/taxonomy/expert.py b/intelmq/bots/experts/taxonomy/expert.py
@@ -43,7 +43,6 @@
     "application-compromise": "intrusions",
     "backdoor": "intrusions",  # not in ENISA eCSIRT-II taxonomy
     "burglary": "intrusions",
-    "defacement": "intrusions",  # not in ENISA eCSIRT-II taxonomy
     "privileged-account-compromise": "intrusions",
     "system-compromise": "intrusions",
     "unprivileged-account-compromise": "intrusions",

diff --git a/intelmq/bots/parsers/cert_eu/parser_csv.py b/intelmq/bots/parsers/cert_eu/parser_csv.py
@@ -30,7 +30,7 @@ class CertEUCSVParserBot(ParserBot):
         "compromised server": "system-compromise",
         "ddos infrastructure": "ddos",
         "ddos target": "ddos",
-        "defacement": "defacement",
+        "defacement": "unauthorised-information-modification",
         "dropzone": "dropzone",
         "exploit url": "exploit",
         "ids alert": "ids-alert",

diff --git a/intelmq/bots/parsers/cymru/parser_cap_program.py b/intelmq/bots/parsers/cymru/parser_cap_program.py
@@ -143,7 +143,7 @@ def parse_line_old(self, line, report):
             raise NotImplementedError('Report %r not implemented, format is unknown.' % report_type)
         elif report_type == 'defacement':  # TODO: verify
             # defacement|192.0.2.1|ASN|YYYY-MM-DD HH:MM:SS|<URL> [<ZONE-H ID>]|ASNAME
-            event['classification.type'] = 'system-compromise'
+            event['classification.type'] = 'unauthorised-information-modification'
             event['classification.identifier'] = report_type
             event['source.url'] = comment_split[0]
             event['extra.zoneh.id'] = comment_split[1]

diff --git a/intelmq/bots/parsers/misp/parser.py b/intelmq/bots/parsers/misp/parser.py
@@ -30,7 +30,7 @@ class MISPParserBot(Bot):
         'ecsirt:intrusion-attempts="exploit"': 'exploit',
         'ecsirt:intrusion-attempts="brute-force"': 'brute-force',
         'ecsirt:intrusion-attempts="ids-alert"': 'ids-alert',
-        'ecsirt:intrusions="defacement"': 'defacement',
+        'ecsirt:intrusions="defacement"': 'unauthorised-information-modification',
         'ecsirt:intrusions="compromised"': 'system-compromise',
         'ecsirt:intrusions="backdoor"': 'backdoor',
         'ecsirt:vulnerable="vulnerable-service"': 'vulnerable-system',

diff --git a/intelmq/bots/parsers/zoneh/parser.py b/intelmq/bots/parsers/zoneh/parser.py
@@ -21,7 +21,7 @@ def parse_line(self, row, report):
         parsed_url = urlparse(row["domain"])
 
         event.add('classification.identifier', "compromised-website")
-        event.add('classification.type', 'system-compromise')
+        event.add('classification.type', 'unauthorised-information-modification')
         event.add('event_description.text', 'defacement')
         event.add('time.source', row["add_date"] + ' UTC')
         event.add('raw', self.recover_line(self.current_line))

diff --git a/intelmq/lib/harmonization.py b/intelmq/lib/harmonization.py
@@ -280,7 +280,6 @@ class ClassificationType(String):
                       'data-loss',
                       'ddos',
                       'ddos-amplifier',
-                      'defacement',
                       'dga-domain',
                       'dos',
                       'dropzone',
@@ -378,6 +377,8 @@ def sanitize(value: str) -> Optional[str]:
             value = 'system-compromise'
         elif value == 'compromised':
             value = 'system-compromise'
+        elif value == 'defacement':
+            value = 'unauthorised-information-modification'
         return GenericType().sanitize(value)
 
 

diff --git a/intelmq/tests/bots/experts/field_reducer/test_expert.py b/intelmq/tests/bots/experts/field_reducer/test_expert.py
@@ -10,15 +10,15 @@
 from intelmq.bots.experts.field_reducer.expert import FieldReducerExpertBot
 
 EXAMPLE_INPUT = {"__type": "Event",
-                 "classification.type": "defacement",
+                 "classification.type": "unauthorised-information-modification",
                  "time.observation": "2015-09-12T00:00:00+00:00",
                  "feed.name": "test-feed",
                  }
 WHITELIST_OUTPUT = {"__type": "Event",
                     "time.observation": "2015-09-12T00:00:00+00:00",
                     }
 BLACKLIST_OUTPUT = {"__type": "Event",
-                    "classification.type": "defacement",
+                    "classification.type": "unauthorised-information-modification",
                     "feed.name": "test-feed",
                     }
 

diff --git a/intelmq/tests/bots/experts/filter/test_expert_absolute_after.py b/intelmq/tests/bots/experts/filter/test_expert_absolute_after.py
@@ -10,7 +10,7 @@
 from intelmq.bots.experts.filter.expert import FilterExpertBot
 
 EXAMPLE_INPUT = {"__type": "Event",
-                 "classification.type": "defacement",
+                 "classification.type": "unauthorised-information-modification",
                  "time.source": "2005-01-01T00:00:00+00:00",
                  "time.observation": "2015-09-12T00:00:00+00:00",
                  "feed.name": "test-feed",

diff --git a/intelmq/tests/bots/experts/filter/test_expert_absolute_before.py b/intelmq/tests/bots/experts/filter/test_expert_absolute_before.py
@@ -10,7 +10,7 @@
 from intelmq.bots.experts.filter.expert import FilterExpertBot
 
 EXAMPLE_INPUT = {"__type": "Event",
-                 "classification.type": "defacement",
+                 "classification.type": "unauthorised-information-modification",
                  "time.source": "2005-01-01T00:00:00+00:00",
                  "time.observation": "2015-09-12T00:00:00+00:00",
                  "feed.name": "test-feed",

diff --git a/intelmq/tests/bots/experts/filter/test_expert_regex_search.py b/intelmq/tests/bots/experts/filter/test_expert_regex_search.py
@@ -10,7 +10,7 @@
 from intelmq.bots.experts.filter.expert import FilterExpertBot
 
 EXAMPLE_INPUT = {"__type": "Event",
-                 "classification.type": "defacement",
+                 "classification.type": "unauthorised-information-modification",
                  "time.source": "2005-01-01T00:00:00+00:00",
                  "time.observation": "2015-09-12T00:00:00+00:00",
                  "feed.name": "test-feed",

diff --git a/intelmq/tests/bots/experts/filter/test_expert_relative_after.py b/intelmq/tests/bots/experts/filter/test_expert_relative_after.py
@@ -10,7 +10,7 @@
 from intelmq.bots.experts.filter.expert import FilterExpertBot
 
 EXAMPLE_INPUT = {"__type": "Event",
-                 "classification.type": "defacement",
+                 "classification.type": "unauthorised-information-modification",
                  "time.source": "2005-01-01T00:00:00+00:00",
                  "time.observation": "2015-09-12T00:00:00+00:00",
                  "feed.name": "test-feed",

diff --git a/intelmq/tests/bots/experts/filter/test_expert_relative_before.py b/intelmq/tests/bots/experts/filter/test_expert_relative_before.py
@@ -10,7 +10,7 @@
 from intelmq.bots.experts.filter.expert import FilterExpertBot
 
 EXAMPLE_INPUT = {"__type": "Event",
-                 "classification.type": "defacement",
+                 "classification.type": "unauthorised-information-modification",
                  "time.source": "2005-01-01T00:00:00+00:00",
                  "time.observation": "2015-09-12T00:00:00+00:00",
                  "feed.name": "test-feed",

diff --git a/intelmq/tests/bots/experts/filter/test_extra.py b/intelmq/tests/bots/experts/filter/test_extra.py
@@ -10,7 +10,7 @@
 from intelmq.bots.experts.filter.expert import FilterExpertBot
 
 EXAMPLE_INPUT = {"__type": "Event",
-                 "classification.type": "defacement",
+                 "classification.type": "unauthorised-information-modification",
                  "time.source": "2005-01-01T00:00:00+00:00",
                  "source.asn": 123,
                  "extra.test1": True,

diff --git a/intelmq/tests/bots/experts/filter/test_paths.py b/intelmq/tests/bots/experts/filter/test_paths.py
@@ -10,7 +10,7 @@
 from intelmq.bots.experts.filter.expert import FilterExpertBot
 
 EXAMPLE_INPUT = {"__type": "Event",
-                 "classification.type": "defacement",
+                 "classification.type": "unauthorised-information-modification",
                  "time.source": "2005-01-01T00:00:00+00:00",
                  "source.asn": 123,
                  "extra.test1": True,
@@ -29,7 +29,7 @@ def set_bot(cls):
         cls.bot_reference = FilterExpertBot
         cls.input_message = EXAMPLE_INPUT
         cls.sysconfig = {'filter_key': 'classification.type',
-                         'filter_value': "defacement",
+                         'filter_value': "unauthorised-information-modification",
                          'filter_action': 'drop'}
 
     def test_extra_filter_drop(self):

diff --git a/intelmq/tests/bots/experts/taxonomy/test_expert.py b/intelmq/tests/bots/experts/taxonomy/test_expert.py
@@ -12,12 +12,12 @@
 
 
 EXAMPLE_INPUT1 = {"__type": "Event",
-                 "classification.type": "defacement",
+                 "classification.type": "unauthorised-information-modification",
                  "time.observation": "2015-01-01T00:00:00+00:00",
                  }
 EXAMPLE_OUTPUT1 = {"__type": "Event",
-                  "classification.type": "defacement",
-                  "classification.taxonomy": "intrusions",
+                  "classification.type": "unauthorised-information-modification",
+                  "classification.taxonomy": "information-content-security",
                   "time.observation": "2015-01-01T00:00:00+00:00",
                   }
 EXAMPLE_INPUT2 = {"__type": "Event",

diff --git a/intelmq/tests/bots/parsers/zoneh/test_zoneh.py b/intelmq/tests/bots/parsers/zoneh/test_zoneh.py
@@ -28,7 +28,7 @@
 ACCEPTED_EVENT00 = {
     '__type': 'Event',
     'feed.name': 'ZoneH Defacements',
-    'classification.type': 'system-compromise',
+    'classification.type': 'unauthorised-information-modification',
     'classification.identifier': 'compromised-website',
     'extra.accepted_date': "2016-06-01 13:20:21",
     "extra.actor": "L33tz",
@@ -50,7 +50,7 @@
 ACCEPTED_EVENT01 = {
     '__type': 'Event',
     'feed.name': 'ZoneH Defacements',
-    'classification.type': 'system-compromise',
+    'classification.type': 'unauthorised-information-modification',
     'classification.identifier': 'compromised-website',
     'extra.accepted_date': "2017-06-11 10:00:00",
     "extra.actor": "mayhab",
@@ -86,7 +86,7 @@
 PENDING_EVENT00 = {
     '__type': 'Event',
     'feed.name': 'ZoneH Defacements',
-    'classification.type': 'system-compromise',
+    'classification.type': 'unauthorised-information-modification',
     'classification.identifier': 'compromised-website',
     'protocol.application': 'https',
     'extra.actor': "xyz crew",

diff --git a/intelmq/tests/lib/test_harmonization.py b/intelmq/tests/lib/test_harmonization.py
@@ -522,6 +522,8 @@ def test_classification_type_sanitize(self):
                          'system-compromise')
         self.assertEqual(harmonization.ClassificationType.sanitize('compromised'),
                          'system-compromise')
+        self.assertEqual(harmonization.ClassificationType.sanitize('defacement'),
+                         'unauthorised-information-modification')
 
     def test_classification_type_sanitize_invalid(self):
         """ Test ClassificationType.is_valid with invalid arguments. """