Add module for cockroachdb (#12467)

Module implemented as a lightweight module, it
collects metrics from cochroachdb, from its Prometheus endpoint.
All metrics are collected as they are and stored under `prometheus.metrics`.

An overview dashboard is included.

CHANGELOG.next.asciidoc

@@ -239,6 +239,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
 - Always report Pod UID in the `pod` metricset. {pull}12345[12345]
 - Add Vsphere Virtual Machine operating system to `os` field in Vsphere virtualmachine module. {pull}12391[12391]
 - Add validation for elasticsearch and kibana modules' metricsets when xpack.enabled is set to true. {pull}12386[12386]
+- Add CockroachDB module. {pull}12467[12467]
 - Add support for metricbeat modules based on existing modules (a.k.a. light modules) {issue}12270[12270] {pull}12465[12465]
 - Add a system/entropy metricset {pull}12450[12450]
 - Add kubernetes metricset `controllermanager` {pull}12409[12409]

metricbeat/docs/fields.asciidoc

@@ -19,6 +19,7 @@ grouped in the following categories:
 * <<exported-fields-beat>>
 * <<exported-fields-ceph>>
 * <<exported-fields-cloud>>
+* <<exported-fields-cockroachdb>>
 * <<exported-fields-common>>
 * <<exported-fields-consul>>
 * <<exported-fields-coredns>>
@@ -3277,6 +3278,14 @@ alias to: cloud.region
 
 --
 
+[[exported-fields-cockroachdb]]
+== CockroachDB fields
+
+CockroachDB module
+
+
+
+
 [[exported-fields-common]]
 == Common fields
 

metricbeat/docs/modules/cockroachdb.asciidoc

@@ -0,0 +1,58 @@
+////
+This file is generated! See scripts/docs_collector.py
+////
+
+[[metricbeat-module-cockroachdb]]
+== CockroachDB module
+
+beta[]
+
+This module periodically fetches metrics from CockroachDB.
+
+[float]
+=== Compatibility
+
+The CockroachDB `status` metricset is compatible with any CockroachDB version
+exposing metrics in Prometheus format.
+
+
+[float]
+=== Dashboard
+
+The CockroachDB module includes a predefined dashboard with overview information
+of the monitored servers.
+
+image::./images/metricbeat-cockroachdb-overview.png[]
+
+
+[float]
+=== Example configuration
+
+The CockroachDB module supports the standard configuration options that are described
+in <<configuration-metricbeat>>. Here is an example configuration:
+
+[source,yaml]
+----
+metricbeat.modules:
+- module: cockroachdb
+  metricsets: ['status']
+  period: 10s
+  hosts: ['localhost:8080']
+
+  # This module uses the Prometheus collector metricset, all
+  # the options for this metricset are also available here.
+  #metrics_path: /_status/vars
+----
+
+This module supports TLS connections when using `ssl` config field, as described in <<configuration-ssl>>.
+It also supports the options described in <<module-http-config-options>>.
+
+[float]
+=== Metricsets
+
+The following metricsets are available:
+
+* <<metricbeat-metricset-cockroachdb-status,status>>
+
+include::cockroachdb/status.asciidoc[]
+

metricbeat/docs/modules/cockroachdb/status.asciidoc

@@ -0,0 +1,17 @@
+////
+This file is generated! See scripts/docs_collector.py
+////
+
+[[metricbeat-metricset-cockroachdb-status]]
+=== CockroachDB status metricset
+
+beta[]
+
+include::../../../../x-pack/metricbeat/module/cockroachdb/status/_meta/docs.asciidoc[]
+
+
+==== Fields
+
+For a description of each field in the metricset, see the
+<<exported-fields-cockroachdb,exported fields>> section.
+

metricbeat/docs/modules_list.asciidoc

@@ -24,6 +24,8 @@ This file is generated! See scripts/docs_collector.py
 |<<metricbeat-metricset-ceph-osd_df,osd_df>>   
 |<<metricbeat-metricset-ceph-osd_tree,osd_tree>>   
 |<<metricbeat-metricset-ceph-pool_disk,pool_disk>>   
+|<<metricbeat-module-cockroachdb,CockroachDB>>  beta[]   |image:./images/icon-yes.png[Prebuilt dashboards are available]    |  
+.1+| .1+|  |<<metricbeat-metricset-cockroachdb-status,status>> beta[]  
 |<<metricbeat-module-consul,consul>>  beta[]   |image:./images/icon-yes.png[Prebuilt dashboards are available]    |  
 .1+| .1+|  |<<metricbeat-metricset-consul-agent,agent>> beta[]  
 |<<metricbeat-module-coredns,coredns>>  beta[]   |image:./images/icon-yes.png[Prebuilt dashboards are available]    |  
@@ -185,6 +187,7 @@ include::modules/aerospike.asciidoc[]
 include::modules/apache.asciidoc[]
 include::modules/aws.asciidoc[]
 include::modules/ceph.asciidoc[]
+include::modules/cockroachdb.asciidoc[]
 include::modules/consul.asciidoc[]
 include::modules/coredns.asciidoc[]
 include::modules/couchbase.asciidoc[]

x-pack/metricbeat/docker-compose.yml

@@ -10,10 +10,16 @@ services:
       - /var/run/docker.sock:/var/run/docker.sock
     command: make
     env_file:
+      - ./module/cockroachdb/_meta/env
       - ./module/coredns/_meta/env
       - ./module/mssql/_meta/env
 
   # Modules
+  cockroachdb:
+    build: ./module/cockroachdb/_meta
+    ports:
+      - 8080
+
   coredns:
     build: ./module/coredns/_meta
     ports:

x-pack/metricbeat/magefile.go

@@ -84,7 +84,7 @@ func TestPackages() error {
 		devtools.WithModules(),
 
 		// To be increased or removed when more light modules are added
-		devtools.MinModules(0))
+		devtools.MinModules(1))
 }
 
 // Fields generates a fields.yml and fields.go for each module.

x-pack/metricbeat/metricbeat.reference.yml

@@ -235,6 +235,16 @@ metricbeat.modules:
   hosts: ["localhost:5000"]
   enabled: true
 
+#----------------------------- CockroachDB Module -----------------------------
+- module: cockroachdb
+  metricsets: ['status']
+  period: 10s
+  hosts: ['localhost:8080']
+
+  # This module uses the Prometheus collector metricset, all
+  # the options for this metricset are also available here.
+  #metrics_path: /_status/vars
+
 #-------------------------------- Consul Module --------------------------------
 - module: consul
   metricsets:

x-pack/metricbeat/module/cockroachdb/README.md

@@ -0,0 +1,29 @@
+For manual testing and development of this module, docker can be used.
+
+The following docker compose starts a two node cluster, dbprod-02 service
+can be copied and renamed many times to have more servers:
+```
+services:
+  dbprod-01:
+    image: cockroachdb/cockroach:v19.1.1
+    command: start --insecure --advertise-addr dbprod-01
+
+  dbprod-02:
+    image: cockroachdb/cockroach:v19.1.1
+    command: start --insecure --advertise-addr dbprod-02 --join dbprod-01
+```
+
+And this configuration can be used for metricbeat:
+```
+metricbeat.autodiscover.providers:
+  - type: docker
+    templates:
+      - condition:
+          contains:
+            docker.container.image: cockroachdb
+        config:
+          - module: cockroachdb
+            hosts: ['${data.host}:8080']
+```
+
+To generate data and load, these examples are pretty handy: https://github.com/cockroachdb/examples-go

x-pack/metricbeat/module/cockroachdb/_meta/Dockerfile

@@ -0,0 +1,7 @@
+FROM cockroachdb/cockroach:v19.1.1
+
+RUN apt-get update && apt-get install -y curl
+
+HEALTHCHECK --interval=1s --retries=90 CMD curl -q http://localhost:8080/_stats/vars
+
+CMD ["start", "--insecure"]

x-pack/metricbeat/module/cockroachdb/_meta/config.yml

@@ -0,0 +1,8 @@
+- module: cockroachdb
+  metricsets: ['status']
+  period: 10s
+  hosts: ['localhost:8080']
+
+  # This module uses the Prometheus collector metricset, all
+  # the options for this metricset are also available here.
+  #metrics_path: /_status/vars

x-pack/metricbeat/module/cockroachdb/_meta/docs.asciidoc

@@ -0,0 +1,16 @@
+This module periodically fetches metrics from CockroachDB.
+
+[float]
+=== Compatibility
+
+The CockroachDB `status` metricset is compatible with any CockroachDB version
+exposing metrics in Prometheus format.
+
+
+[float]
+=== Dashboard
+
+The CockroachDB module includes a predefined dashboard with overview information
+of the monitored servers.
+
+image::./images/metricbeat-cockroachdb-overview.png[]

x-pack/metricbeat/module/cockroachdb/_meta/env

@@ -0,0 +1,2 @@
+COCKROACHDB_HOST=cockroachdb
+COCKROACHDB_PORT=8080

x-pack/metricbeat/module/cockroachdb/_meta/fields.yml

@@ -0,0 +1,10 @@
+- key: cockroachdb
+  title: 'CockroachDB'
+  release: beta
+  description: >
+    CockroachDB module
+  settings: ["ssl", "http"]
+  fields:
+    - name: cockroachdb
+      type: group
+      fields: