[Metricbeat] Support processors defined for light modules

[mtojek]

This PR adds processor support for light modules. Processors can be enabled using definitions in manifest.yml.

Issue: #14740

Let me start with draft and if we're fine with design and implementation, I will cover it with tests.

How to test this PR

1. Rollback changes in the IBM MQ module manifest to use script processors:

  module: prometheus
  metricset: collector
  defaults:
    metrics_path: /metrics

# The custom processor is responsible for filtering Prometheus metrics
# not stricly related to the IBM MQ domain, e.g. system load, process,
# metrics HTTP server.
processors:
  - script:
      lang: javascript
      source: >
        function process(event) {
          var metrics = event.Get("prometheus.metrics");
          if (metrics == null) {
            event.Cancel();
            return;
          }
          Object.keys(metrics).forEach(function(key) {
            if (!(key.match(/^ibmmq_.*$/))) {
              event.Delete("prometheus.metrics." + key);
            }
          });
          metrics = event.Get("prometheus.metrics");
          if (Object.keys(metrics).length == 0) {
            event.Cancel();
          }
        }

(Remove metrics_filters)

2. Use docker-compose up to boot up IBM MQ module. The port localhost:9157 should be exposed.
3. Enable ibmmq metricbeat (x-pack) module.
4. Start metricbeat with -e -d processors and observe events flow.

Filtering should be applied. To make sure that the processor is used, make a mistake in the configuration. It should appear in the processor's debug logs.

[jsoriano]

Thanks @mtojek for working on this!

I have added some comments, the main issues I see:

• Processors defined in the manifest should be executed before processors defined in the configuration. This will be important to know what to expect from the metricset itself, and what fields are available for processors in the configuration.
• The link between the metricsets and the connectors, based on a map with the metricset name, and the content of a field in the event, looks a bit weak to me. It'd be nice if we could find a more explicit way of connecting the metricsets with the runners.

Have you considered to create one wrapper per metricset when needed? Not sure about the implications. But this would allow to create also a runner per metricset without modifying the current runner. For these cases Factory.Create could return a runner group struct that implements the Runner interface and contains all the runners generated for a configuration.

Or maybe we need to have multiple outputs in each wrapper, and attach them to each one of its specific clients.

[mtojek]

I fixed all your concerns and got rid of map of clients. A runner group combining all wrappers has been applied.
When we agree on implementation, I'll proceed with testing.

[jsoriano]

I think we can continue with this approach, good job!

[mtojek]

Thanks for adding the assertion to the system test!

... and it passed. I will proceed with merging the PR.

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)