Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(SUP-2054) Add VMware metrics collection #68

Merged
merged 4 commits into from
Sep 30, 2020

Conversation

Sharpie
Copy link
Member

@Sharpie Sharpie commented Sep 18, 2020

This changeset updates the puppet_metrics_collector::system class to include collection of VMware metrics if facter virtual reports "vmware". Collection is done via vmware-toolbox-cmd stat and includes:

  • session: The VM session ID along with uptime since being booted or migrated along with ESXi version info.
  • host: Information about the hypervisor, CPU counts and frequencies.
  • resources: CPU and memory resources allocated to the VM. Shows CPU time that has been lost to hypervisor overprovisioning and overhead along with memory usage, memory swapping, memory lost to ESXi ballooning, and NUMA distribution.
  • vscsi: I/O performance for each virtual disk attached to the VM.
  • vnet: I/O performance for each virtual NIC attached to the VM.

This commit adds a Ruby script, `files/vmware_metrics`, that uses
`vmware-toolbox-cmd stat` to collect VM performance metrics from
an ESXi hypervisor.
This commit updates the `puppet_metrics_collector::sar_metric` type to
refer to `puppet_metrics_collector::sytem::scripts_dir` when configuring
metrics cleanup jobs.

This change allows the `puppet_metrics_collector::system` class to be
used without including the `puppet_metrics_collector` class --- which
enables spec testing of the system class.
This commit updates the `puppet_metrics_collector::system` class to
include collection of VMware metrics if the virtualization platform
is detected to be VMware.
@puppet-community-rangefinder
Copy link

puppet_metrics_collector::sar_metric is a type

that may have no external impact to Forge modules.

puppet_metrics_collector::system is a class

that may have no external impact to Forge modules.

puppet_metrics_collector::system::vmware is a class

that may have no external impact to Forge modules.

This module is declared in 0 of 575 indexed public Puppetfiles.


These results were generated with Rangefinder, a tool that helps predict the downstream impact of breaking changes to elements used in Puppet modules. You can run this on the command line to get a full report.

Exact matches are those that we can positively identify via namespace and the declaring modules' metadata. Non-namespaced items, such as Puppet 3.x functions, will always be reported as near matches only.

@Sharpie
Copy link
Member Author

Sharpie commented Sep 18, 2020

Example output:

# cat /opt/puppetlabs/puppet-metrics-collector/vmware/busier-becoming/20200918T194503Z.json |python -m json.tool
{
    "servers": {
        "busier-becoming": {
            "vmware": {
                "host": {
                    "host.cpu.coresPerPkg": {
                        "type": "static",
                        "units": "count",
                        "value": 20
                    },
                    "host.cpu.processorMHz": {
                        "type": "instant",
                        "units": "MHz",
                        "value": 1995
                    }
                },
                "resources": {
                    "guest.mem.ballooned": {
                        "type": "instant",
                        "units": "KB",
                        "value": 0
                    },
                    "guest.mem.consumed": {
                        "type": "instant",
                        "units": "KB",
                        "value": 8155716
                    },
                    "guest.mem.limit": {
                        "type": "static",
                        "units": "KB",
                        "value": -1
                    },
                    "guest.mem.mapped": {
                        "type": "instant",
                        "units": "KB",
                        "value": 8255488
                    },
                    "guest.mem.reserved": {
                        "type": "static",
                        "units": "KB",
                        "value": 0
                    },
                    "guest.mem.swapIn": {
                        "type": "accumulate",
                        "units": "KB",
                        "value": 0
                    },
                    "guest.mem.swapOut": {
                        "type": "accumulate",
                        "units": "KB",
                        "value": 0
                    },
                    "guest.mem.swapped": {
                        "type": "instant",
                        "units": "KB",
                        "value": 0
                    },
                    "ovhd.mem.swapIn": {
                        "type": "accumulate",
                        "units": "KB",
                        "value": 0
                    },
                    "ovhd.mem.swapOut": {
                        "type": "accumulate",
                        "units": "KB",
                        "value": 0
                    },
                    "ovhd.mem.swapped": {
                        "type": "instant",
                        "units": "KB",
                        "value": 0
                    },
                    "vm.cpu.contention.cpu": {
                        "type": "accumulate",
                        "units": "us",
                        "value": 53064863
                    },
                    "vm.cpu.contention.mem": {
                        "type": "accumulate",
                        "units": "us",
                        "value": 0
                    },
                    "vm.cpu.limit": {
                        "type": "static",
                        "units": "MHz",
                        "value": -1
                    },
                    "vm.cpu.reserved": {
                        "type": "static",
                        "units": "MHz",
                        "value": 0
                    },
                    "vm.cpu.used": {
                        "type": "accumulate",
                        "units": "us",
                        "value": 8929863954
                    },
                    "vm.numa.local": {
                        "type": "instant",
                        "units": "KB",
                        "value": 8255488
                    },
                    "vm.numa.remote": {
                        "type": "instant",
                        "units": "KB",
                        "value": 0
                    }
                },
                "session": {
                    "provider": "",
                    "session": {
                        "type": "static",
                        "units": "int",
                        "value": 7350989680269531893
                    },
                    "uptime": {
                        "type": "static",
                        "units": "us",
                        "value": 91006153434
                    },
                    "uuid.bios": "42 1a 53 a5 db e4 61 a5-0a c9 97 c3 4c 31 c3 f1",
                    "version": "VMware ESX 6.7.0 build-16713306"
                },
                "vnet": {
                    "00:50:56:9a:74:69": {
                        "limit": {
                            "type": "static",
                            "units": "MBps",
                            "value": -1
                        },
                        "num.rx": {
                            "type": "accumulate",
                            "units": "packets",
                            "value": 945092
                        },
                        "num.tx": {
                            "type": "accumulate",
                            "units": "packets",
                            "value": 70857
                        },
                        "reservation": {
                            "type": "static",
                            "units": "MBps",
                            "value": 0
                        },
                        "size.rx": {
                            "type": "accumulate",
                            "units": "bytes",
                            "value": 904013131
                        },
                        "size.tx": {
                            "type": "accumulate",
                            "units": "bytes",
                            "value": 7399832
                        }
                    }
                },
                "vscsi": {
                    "scsi0:0": {
                        "latency.reads": {
                            "type": "accumulate",
                            "units": "us",
                            "value": 7645046
                        },
                        "latency.writes": {
                            "type": "accumulate",
                            "units": "us",
                            "value": 228170729
                        },
                        "num.reads": {
                            "type": "accumulate",
                            "units": "int",
                            "value": 7634
                        },
                        "num.writes": {
                            "type": "accumulate",
                            "units": "int",
                            "value": 166168
                        },
                        "size.reads": {
                            "type": "accumulate",
                            "units": "bytes",
                            "value": 268690944
                        },
                        "size.writes": {
                            "type": "accumulate",
                            "units": "bytes",
                            "value": 5444434432
                        }
                    }
                }
            }
        }
    },
    "timestamp": "2020-09-18T19:45:03Z"
}

The `Style/BracesAroundHashParameters` check gives bad advice that will
result in deprecation warnings with Ruby 2.7. This check has been
removed upstream starting with RuboCop 0.80.0.
Copy link
Contributor

@jarretlavallee jarretlavallee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. It was enabled automatically on my vSphere based master with system metrics as expected.

@Sharpie Sharpie merged commit 73c53ac into puppetlabs:master Sep 30, 2020
@Sharpie Sharpie deleted the SUP-2054-gather-vmware-metrics branch October 30, 2020 21:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants