Added an example that uses exported resources to set up a master-age…

…nt infrastructure

examples/example3/README.md

@@ -0,0 +1,75 @@
+## Example 3 – Using virtual resources and collection in a master-agent set-up.
+
+This is an (almost) complete example for a master-agent set-up using virtual resources in Puppet.
+
+### Manifests
+
+All nodes that should be monitored inherit from the monitorednode role, thus applying the `profile::icinga::agent` class.
+
+Each monitored node exports itself as an endpoint and a zone. This information is then automatically collected on the master to generate the necessary configuration files.
+
+Agents also export a `Host` object (`@@icinga2::object::host`) and use hiera_hash() to get and assemble host properties from the respective hiera files throughout the hiera hierarchy.
+
+#### A word of caution on services and apply rules
+
+This set-up does not use any "manually" created `Service` objects, but _applies_ services to hosts based on their vars exclusively. The result is a much simplier Icinga2 configuration, among other things.
+
+There are two things to note here about `Apply Rules`:
+
+1. We don't use `icinga2::object::service` to define the apply rules, since the current version (0.7.1) of this Puppet module does not yet support the entire range of available functions and macros. Instead, we use `file` resources from a custom, dedicated module, together with the _icinga2::config::file_ tag. The tag makes sure the files will be put in place at the correct point in time and the icinga2 service restarted afterwards (see the module's [README](https://github.com/Icinga/puppet-icinga2-rewrite#custom-configuration) for more information).
+
+An example for one such apply rule file would be:
+
+```
+apply Service "nginx-status" to Host {
+    import "generic-service"
+
+    vars += host.vars.checks["nginx-status"]
+    check_command = "nginx_status"
+    command_endpoint = host.vars.client_endpoint
+
+    assign where host.vars.os == "Linux" && host.vars.checks["nginx-status"]
+    ignore where !host.address || !host.vars.client_endpoint || !host.vars.checks
+}
+```
+
+2. Most apply rules are defined on the master and not on the individual nodes. Although it would be preferable to define apply rules as exported resources on the individual nodes – so that they are created only as additional services (profiles) are added to a node, this is not possible when you have more than one node with the same services in your infrastructure since it will lead to duplicate resource definitions.
+
+Compare the `profile::backuppc::server` manifest to the `profile::nginx` profile. In the first, you will find the apply rule definition is exported as a file resource to be collected in `profile::icinga::applyrules` at the very bottom, whereas the second does not contain any `Service` object at all.
+Apply rules for the _nginx_ profile have been defined in `profile::icinga::applyrules` since there are several nodes using this same profile.
+
+The nginx profile, however, additionally installs a check script that does not come with any of the _monitoring-plugins-*_ packages on Debian.
+
+### Hiera
+
+With a hiera hierarchy as the following (simplified), all nodes will consume both common.yaml and their dedicated yaml file, if they have one:
+
+```yaml
+---
+:backends:
+  - yaml
+:hierarchy:
+  - "nodes/%{::fqdn}"
+  - common
+```
+
+In common.yaml, we define host vars valid for all hosts throughout the infrastructure, while in the respective nodes' yaml files, we define further checks and vars that apply to that host only.
+
+The icingamaster.yaml file contains most of the config necessary for the master configuration.
+
+It's important that we set empty endpoints and zones in this file, so they will not be automatically generated by the icinga2 module, which uses defaults if the values are not set.
+
+```yaml
+icinga2::feature::api::endpoints: {}
+icinga2::feature::api::zones: {}
+```
+
+We use hiera_array() and hiera_hash() lookup functions from the manifests in order to merge arrays and hashes from the various levels in our hierarchy, something that Puppet does not support with automatically looked up values (cf. https://tickets.puppetlabs.com/browse/HI-233)
+
+### Notes
+
+The example is not 100% complete, some of the profile classes that are not relevant to illustrate a master-agent set-up using virtual resources in Puppet are not included and are left as an exercise to the reader.
+
+### Disclaimer
+
+This example does not claim to be a perfect setup. You might have improvements to suggest and those are of course welcome.

examples/example3/hieradata/common.yaml

@@ -0,0 +1,12 @@
+---
+classes:
+ - 'profile::base'
+
+icinga_vars:
+  os: Linux
+  cores: "%{::processorcount}"
+  virtual_machine: "%{::is_virtual}"
+  distro: "%{::operatingsystem}"
+  disks:
+    'disk /':
+      disk_partitions: '/'

examples/example3/hieradata/nodes/icingamaster.yaml

@@ -0,0 +1,31 @@
+---
+icinga2::manage_repo: true
+icinga2::confd: true
+icinga2::constants:
+  NodeName: "%{::fqdn}"
+  ZoneName: 'master'
+icinga2::features:
+  - 'api'
+  - 'checker'
+  - 'mainlog'
+  - 'notification'
+  - 'statusdata'
+  - 'compatlog'
+  - 'command'
+  - 'idomysql'
+  - 'graphite'
+  # - 'debuglog'
+
+icinga2::feature::api::accept_commands: true
+icinga2::feature::api::endpoints: {}
+icinga2::feature::api::zones: {}
+
+icinga2::feature::graphite::host: '192.168.1.8'
+icinga2::feature::graphite::port: '2003'
+icinga2::feature::graphite::enable_send_tresholds: true
+icinga2::feature::graphite::enable_send_metadata: true
+
+icinga2::feature::idomysql::database: icinga2
+icinga2::feature::idomysql::user: icinga2
+icinga2::feature::idomysql::password: *************
+icinga2::feature::idomysql::import_schema: true

examples/example3/hieradata/nodes/webserver.yaml

@@ -0,0 +1,17 @@
+---
+classes:
+  - 'role::webserver'
+
+icinga_vars:
+  client_endpoint: "%{::fqdn}"
+  role: http-server
+  vhosts:
+    sub.domain.tld:
+      uri: '/ping'
+  checks:
+    'nginx-status':
+      nginx_status_host_address: '127.0.0.1'
+      nginx_status_port: '8433'
+      nginx_status_url: '/nginx-status'
+      nginx_status_warn: '100,50,100'
+      nginx_status_critical: '200,100,200'

examples/example3/site.pp

@@ -0,0 +1 @@
+hiera_include('classes')

examples/example3/site/profile/manifests/backuppc/server.pp

@@ -0,0 +1,54 @@
+class profile::backuppc::server {
+    class { '::backuppc::server': }
+    create_resources('backuppc::server::user', hiera('backuppc_users', []))
+
+    # Icinga CheckCommand and Apply Rules
+    @@icinga2::object::checkcommand { 'backuppc':
+        import      => [
+            'plugin-check-command',
+        ],
+        command     => [
+            'sudo', '-u', 'backuppc',
+            'PluginContribDir + /check_backuppc',
+        ],
+        arguments   => {
+            '-w' => '$backuppc_wtime$',
+            '-c' => '$backuppc_ctime$',
+            '-H' => {
+                'value' => '$backuppc_desired$',
+                'set_if' => '$backuppc_desired$',
+            },
+            '-x' => {
+                'value' => '$backuppc_exclude$',
+                'set_if' => '$backuppc_exclude$',
+            },
+            '-V' => {
+                'set_if' => '$backuppc_version$'
+            },
+            '-a' => {
+                'set_if' => '$backuppc_archiveonly$',
+            },
+            '-b' => {
+                'set_if' => '$backuppc_backuponly$',
+            },
+            '-s' => {
+                'set_if' => '$backuppc_statusonly$',
+            },
+        },
+        vars        => {
+            'backuppc_wtime' => '2',
+            'backuppc_ctime' => '4',
+        },
+        target      => '/etc/icinga2/zones.d/global-templates/backuppc-command.conf',
+    }
+
+    @@file { '/etc/icinga2/conf.d/services/backuppc.conf':
+        ensure  => file,
+        owner   => 'nagios',
+        group   => 'nagios',
+        tag     => 'icinga2::config::exported',
+        source => [
+            "puppet:///modules/1024/icinga/services/backuppc.conf",
+        ],
+    }
+}

examples/example3/site/profile/manifests/icinga/agent.pp

@@ -0,0 +1,57 @@
+class profile::icinga::agent {
+
+    # By default, the icinga module only installs monitoring-plugins-base
+    ensure_packages([
+        'monitoring-plugins-standard',
+        'nagios-plugins-contrib',
+        'libmonitoring-plugin-perl',
+    ], {
+        install_options => ['--no-install-recommends'],
+    })
+
+    # Options valid for all agents, thus defined inside the manifest
+    class { '::icinga2':
+        manage_repo => true,
+        confd       => false,
+        features    => [ 'checker','mainlog' ],
+    }
+
+    # Leave this here or put it in a yaml file common
+    # to icinga agent nodes only.
+    class { '::icinga2::feature::api':
+        pki             => 'puppet',
+        accept_config   => true,
+        accept_commands => true,
+        endpoints       => {},
+        zones           => {},
+    }
+
+    icinga2::object::zone { 'global-templates':
+        global => true,
+    }
+
+    # All nodes export resources for icinga monitoring
+    # The vars (set in the various nodes hiera files) are used to Apply Services
+    # to these hosts. (See profile::icinga::server)
+    @@::icinga2::object::host { $::fqdn:
+        display_name            => $::fqdn,
+        address                 => $::ipaddress_eth0,
+        check_command           => 'hostalive',
+        vars                    => hiera_hash('icinga_vars', {}),
+        target                  => "/etc/icinga2/zones.d/master/${::fqdn}.conf"
+    }
+
+    # Create virtual resources for this agent node
+    @@::icinga2::object::endpoint { "$::fqdn":
+        host => "$::ipaddress_eth0",
+    }
+
+    @@::icinga2::object::zone { "$::fqdn":
+        endpoints => [ "$::fqdn", ],
+        parent    => 'master',
+    }
+
+    # Collect and realize info about self and master, but no other nodes.
+    Icinga2::Object::Endpoint <<| title == $::fqdn or title == 'master.sub.domain.tld' |>> { }
+    Icinga2::Object::Zone <<| title == $::fqdn or title == 'master' |>> { }
+}

examples/example3/site/profile/manifests/icinga/applyrules.pp

@@ -0,0 +1,53 @@
+class profile::icinga::applyrules {
+
+    # Global apply rules
+    # We attempt to export them with the respective services where possible.
+    # However, that only works if the service is unique on the infrastructure and would
+    # not lead to duplicate resources.
+    #
+    # All multi-use (apply) services are defined here.
+    #
+    # We do not use "icinga2::object::service" but files with the "icinga2::config::file" tag. See the
+    # example's README on why this is the case.
+
+    file { '/etc/icinga2/conf.d/services/nginx.conf':
+        ensure  => file,
+        owner   => 'nagios',
+        group   => 'nagios',
+        tag     => 'icinga2::config::file',
+        source => [
+            "puppet:///modules/1024/icinga/services/nginx.conf",
+        ],
+    }
+
+    file { '/etc/icinga2/conf.d/services/postgres.conf':
+        ensure  => file,
+        owner   => 'nagios',
+        group   => 'nagios',
+        tag     => 'icinga2::config::file',
+        source => [
+            "puppet:///modules/1024/icinga/services/postgres.conf",
+        ],
+    }
+
+    file { '/etc/icinga2/conf.d/services/elasticsearch.conf':
+        ensure  => file,
+        owner   => 'nagios',
+        group   => 'nagios',
+        tag     => 'icinga2::config::file',
+        source => [
+            "puppet:///modules/1024/icinga/services/elasticsearch.conf",
+        ],
+    }
+
+    # Collect any files exported and tagged elsewhere (can be created inside
+    # services or master zone)
+    # We need to use a different tag then icinga itself (icinga2::config::file)
+    # or the agent will try to collect any resources tagged so on himself.
+    File <<| ensure != 'directory' and tag == 'icinga2::config::exported' |>> {
+        require => [
+            File['icinga2_masterzone'],
+            File['icinga2_services'],
+        ],
+    }
+}

examples/example3/site/profile/manifests/icinga/server.pp

@@ -0,0 +1,81 @@
+class profile::icinga::server {
+
+    class { '::icinga2': }
+
+    icinga2::object::zone { 'global-templates':
+        global => true,
+    }
+
+    file { 'icinga2_global_templates':
+        path    => '/etc/icinga2/zones.d/global-templates',
+        ensure  => directory,
+        purge   => true,
+        recurse => true,
+    }->
+    File <<| ensure != 'directory' and tag == 'icinga2::scripts::file' |>> { }
+
+    # Collect all hosts into their respective directories.
+    file { 'icinga2_masterzone':
+        path    => '/etc/icinga2/zones.d/master',
+        ensure  => directory,
+        purge   => true,
+        recurse => true,
+    }->
+    file { 'icinga2_hosts':
+        path    => '/etc/icinga2/conf.d/hosts',
+        ensure  => directory,
+        purge   => true,
+        recurse => true,
+    }->
+    Icinga2::Object::Host <<| |>> { }
+
+    # Export master zone and endpoint for all agents to collect
+    @@icinga2::object::zone { 'master':
+        endpoints => [ "$::fqdn", ],
+    }
+    @@icinga2::object::endpoint { "$::fqdn":
+        host => "$::ipaddress_eth0",
+    }
+
+    # Collect and realize all agent zones and endpoints
+    Icinga2::Object::Endpoint <<| |>> { }
+    Icinga2::Object::Zone <<| |>> { }
+
+    # Collect services and notifications exported on agent nodes
+    # (and not created by the Apply Rules included below)
+    file { 'icinga2_services':
+        path    => '/etc/icinga2/conf.d/services',
+        ensure  => directory,
+        purge   => true,
+        recurse => true,
+    }->
+    Icinga2::Object::Service <<| |>> { }
+
+    file { 'icinga2_notifications':
+        path    => '/etc/icinga2/conf.d/notifications',
+        ensure  => directory,
+        purge   => true,
+        recurse => true,
+    }->
+    Icinga2::Object::Notification <<| |>> { }
+
+    # Collect check and notification commands that are not created by Apply
+    file { 'icinga2_commands':
+        path    => '/etc/icinga2/conf.d/commands',
+        ensure  => directory,
+        purge   => true,
+        recurse => true,
+    }->
+    Icinga2::Object::Checkcommand <<| |>> { }->
+    Icinga2::Object::NotificationCommand <<| |>> { }
+
+    # Define apply rules that
+    contain profile::icinga::applyrules
+
+    # Note: these manifests are not included in this example
+    contain profile::icinga::hostgroups
+    contain profile::icinga::users
+    contain profile::icinga::timeperiods
+    contain profile::icinga::notifications
+    contain profile::icinga::checkcommands
+}