Add host inventory metrics to system module

[kaiyan-sheng]

What does this PR do?

This PR is to add proposed host common fields into system module:

• host.id
• host.name
• host.cpu.pct
• host.network.in.bytes
• host.network.in.packets
• host.network.out.bytes
• host.network.out.packets
• host.disk.read.bytes
• host.disk.write.bytes

For network and disk metrics, instead of reporting one event per interface, host.network.* and host.disk.* aggregates values from all interfaces.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

Change system module config system.yml with:

- module: system
  period: 10s
  metricsets:
    - cpu
    - load
    - network
  cpu.metrics: [normalized_percentages]

Then start Metricbeat with ./metricbeat -e and you should see host common fields listed above in events.

Event Example

CPU metric:

{
    "event": {
      "duration": 88523,
      "dataset": "system.cpu",
      "module": "system"
    },
    "metricset": {
      "name": "cpu",
      "period": 10000
    },
    "service": {
      "type": "system"
    },
    "system": {
      "cpu": {
        "user": {
          "pct": 0.7653,
          "norm": {
            "pct": 0.0638
          }
        },
        "steal": {
          "pct": 0,
          "norm": {
            "pct": 0
          }
        },
        "cores": 12,
        "system": {
          "pct": 0.2961,
          "norm": {
            "pct": 0.0247
          }
        },
        "iowait": {
          "pct": 0,
          "norm": {
            "pct": 0
          }
        },
        "nice": {
          "pct": 0,
          "norm": {
            "pct": 0
          }
        },
        "total": {
          "norm": {
            "pct": 0.0884
          },
          "pct": 1.0614
        },
        "idle": {
          "pct": 10.9386,
          "norm": {
            "pct": 0.9116
          }
        },
        "irq": {
          "norm": {
            "pct": 0
          },
          "pct": 0
        },
        "softirq": {
          "pct": 0,
          "norm": {
            "pct": 0
          }
        }
      }
    },
    "host": {
      "cpu": {
        "pct": 0.0884
      },
      "name": "KaiyanMacBookPro",
      "hostname": "KaiyanMacBookPro",
      "architecture": "x86_64"
    }
  }
}

Network metrics:

{
  "_source": {
    "metricset": {
      "name": "network",
      "period": 60000
    },
    "service": {
      "type": "system"
    },
    "host.network.out.bytes": 26123693254,
    "host.network.out.packets": 134348248,
    "event": {
      "dataset": "system.network",
      "module": "system",
      "duration": 6234971
    },
    "host.network.in.bytes": 74833783286,
    "host.network.in.packets": 139823676,
    "host": {
      "name": "KaiyanMacBookPro",
      "hostname": "KaiyanMacBookPro",
      "architecture": "x86_64",
      "id": "9C7FAB7B-29D1-5926-8E84-158A9CA3E25D"
    }
}

Disk metrics:

{
  "_source": {
    "host.disk.read.bytes": 535049839616,
    "host.disk.write.bytes": 782890594304,
    "event": {
      "dataset": "system.diskio",
      "module": "system",
      "duration": 606341
    },
    "metricset": {
      "name": "diskio",
      "period": 10000
    },
    "service": {
      "type": "system"
    },
    "host": {
      "name": "KaiyanMacBookPro",
      "architecture": "x86_64",
      "hostname": "KaiyanMacBookPro"
    }
  }
}

Related issues

• Relates to Common Fields for Host Inventory Schema #19757

[elasticmachine]

Pinging @elastic/integrations-platforms (Team:Platforms)

[elasticmachine]

💚 Build Succeeded

Expand to view the summary

Build stats

• Build Cause: [Pull request #20415 updated]

• Start Time: 2020-09-21T16:18:54.458+0000

• Duration: 64 min 55 sec

Test stats 🧪

Test	Results
Failed	0
Passed	3212
Skipped	706
Total	3918

[jsoriano]

Could you update the data.json files?

[fearful-symmetry]

The only thing I'm kinda worried about here is how conceptually similar metrics (for example, network.in) are turned into gauges from counters. That seems like a potential point of confusion, but if we're treating the host.* fields as basically artifacts for other components of the stack, I don't think it'll be a huge issue.

[fearful-symmetry]

All the other metricsets this PR adds host data to just constructs a MapStr in Event(). I wonder if we want to do that instead for the sake of consistency?

[kaiyan-sheng]

@fearful-symmetry Yes, these metrics will be common across all hosts/VMs, such as AWS EC2, Azure compute VM. These components don't report counters so in order to make them all consistent, we decide to use gauges instead. We will definitely keep all the counters in system module because they are the raw data and more accurate when data loss occurred. But for the sake of consistency and for Metrics UI display, we will use gauges instead.

[fearful-symmetry]

Looks Good. Also +1 on updating the data.json files.

[kaiyan-sheng]

@fearful-symmetry Thank you! Will do! (maybe early next week). I will ping you and Jaime for review again after I updated the data.json files.

[kaiyan-sheng]

data.json files updated. For disk and network host metrics, it’s not showing up in data.json. That’s because the host disk and host network metrics are calculated gauges instead of counters. So it needs more than one metric to calculate gauge.

[fearful-symmetry]

LGTM.