[Azure Logs] Replace foreach+set combo with a script processor (#34478)

* Replace foreach+set combo with a script processor

The `set` processor expands the dots contained in the field name into
subfield.

Sometimes attributes contained in `authentication_processing_details`
have dots, for example:

    # source
    {"key": "a.b.c", "value": true}

In such cases, the `set` processor would turn it into:

    # this is a side-effect
    {
      "a": {
        "b": {
          "c": true
        }
      }
    }

Instead of:

    # this is the expected result
    {"a.b.c": True}

* Update changelog

(cherry picked from commit a0b2db2)

CHANGELOG.next.asciidoc

@@ -30,6 +30,63 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
 
 *Affecting all Beats*
 
+- Fix Windows service install/uninstall when Win32_Service returns error, add logic to wait until the Windows Service is stopped before proceeding. {pull}33322[33322]
+- Support for multiline zookeeper logs {issue}2496[2496]
+- Allow `clock_nanosleep` in the default seccomp profiles for amd64 and 386. Newer versions of glibc (e.g. 2.31) require it. {issue}33792[33792]
+- Disable lockfile when running under elastic-agent. {pull}33988[33988]
+- Fix lockfile logic, retry locking {pull}34194[34194]
+- Add checks to ensure reloading of units if the configuration actually changed. {pull}34346[34346]
+- Fix namespacing on self-monitoring {pull}32336[32336]
+- Fix race condition when stopping runners {pull}32433[32433]
+- Fix concurrent map writes when system/process code called from reporter code {pull}32491[32491]
+- Log errors from the Elastic Agent V2 client errors channel. Avoids blocking when error occurs communicating with the Elastic Agent. {pull}34392[34392]
+- Only log publish event messages in trace log level under elastic-agent. {pull}34391[34391]
+
+*Auditbeat*
+
+
+*Filebeat*
+- [Auditbeat System Package] Added support for Apple Silicon chips. {pull}34433[34433] 
+- [Azure blob storage] Changed logger field name from `container` to `container_name` so that it does not clash
+   with the ecs field name `container`. {pull}34403[34403]
+- [GCS] Added support for more mime types & introduced offset tracking via cursor state. Also added support for
+   automatic splitting at root level, if root level element is an array. {pull}34155[34155]
+- [httpsjon] Improved error handling during pagination with chaining & split processor {pull}34127[34127]
+- [Azure blob storage] Added support for more mime types & introduced offset tracking via cursor state. {pull}33981[33981]
+- Fix EOF on single line not producing any event. {issue}30436[30436] {pull}33568[33568]
+- Fix handling of error in states in direct aws-s3 listing input {issue}33513[33513] {pull}33722[33722]
+- Fix `httpjson` input page number initialization and documentation. {pull}33400[33400]
+- Add handling of AAA operations for Cisco ASA module. {issue}32257[32257] {pull}32789[32789]
+- Fix gc.log always shipped even if gc fileset is disabled {issue}30995[30995]
+- Fix handling of empty array in httpjson input. {pull}32001[32001]
+- Fix reporting of `filebeat.events.active` in log events such that the current value is always reported instead of the difference from the last value. {pull}33597[33597]
+- Fix splitting array of strings/arrays in httpjson input {issue}30345[30345] {pull}33609[33609]
+- Fix Google workspace pagination and document ID generation. {pull}33666[33666]
+- Fix PANW handling of messages with event.original already set. {issue}33829[33829] {pull}33830[33830]
+- Rename identity as identity_name when the value is a string in Azure Platform Logs. {pull}33654[33654]
+- Fix 'requires pointer' error while getting cursor metadata. {pull}33956[33956]
+- Fix input cancellation handling when HTTP client does not support contexts. {issue}33962[33962] {pull}33968[33968]
+- Update mito CEL extension library to v0.0.0-20221207004749-2f0f2875e464 {pull}33974[33974]
+- Fix CEL result deserialisation when evaluation fails. {issue}33992[33992] {pull}33996[33996]
+- Fix handling of non-200/non-429 status codes. {issue}33999[33999] {pull}34002[34002]
+- [azure-eventhub input] Switch the run EPH run mode to non-blocking {pull}34075[34075]
+- [google_workspace] Fix pagination and cursor value update. {pull}34274[34274]
+- Fix handling of quoted values in auditd module. {issue}22587[22587] {pull}34069[34069]
+- Fixing system tests not returning expected content encoding for azure blob storage input. {pull}34412[34412]
+- [Azure Logs] Fix authentication_processing_details parsing in sign-in logs. {issue}34330[34330] {pull}34478[34478]
+
+*Heartbeat*
+
+- Fix broken zip URL monitors. NOTE: Zip URL Monitors will be removed in version 8.7 and replaced with project monitors. {pull}33723[33723]
+- Fix bug where states.duration_ms was incorrect type. {pull}33563[33563]
+- Fix handling of long UDP messages in UDP input. {issue}33836[33836] {pull}33837[33837]
+- Fix browser monitor summary reporting as up when monitor is down. {issue}33374[33374] {pull}33819[33819]
+- Fix beat capabilities on Docker image. {pull}33584[33584]
+- Fix serialization of state duration to avoid scientific notation. {pull}34280[34280]
+
+
+*Heartbeat*
+
 
 *Auditbeat*
 

x-pack/filebeat/module/azure/signinlogs/ingest/pipeline.yml

@@ -174,20 +174,16 @@ processors:
       field:
         - azure.signinlogs.properties.location
       ignore_missing: true
-  - foreach:
-      field: azure.signinlogs.properties.authentication_processing_details
-      ignore_missing: true
-      processor:
-        set:
-          field: '_tmp.{{{_ingest._value.key}}}'
-          copy_from: _ingest._value.value
-  - set:
-      if: ctx?._tmp != null
-      field: azure.signinlogs.properties.authentication_processing_details
-      copy_from: _tmp
-  - remove:
-      field: _tmp
-      ignore_missing: true
+  - script:
+      description: "Turns the authentication_processing_details array elements into key/value pairs. For example, the array element ``{key: 'key1', value: 'value1'}`` becomes ``{key1: 'value1'}``."
+      lang: painless
+      source: |
+        def tmp = [:];
+        for (item in ctx.azure.signinlogs.properties.authentication_processing_details) {
+            tmp[item.key] = item.value;
+        }
+        ctx.azure.signinlogs.properties.authentication_processing_details = tmp;
+      if: ctx.azure?.signinlogs?.properties?.authentication_processing_details != null && ctx.azure.signinlogs.properties.authentication_processing_details instanceof List
   - set:
       field: event.kind
       value: event