Skip to content

Commit

Permalink
Add flowmetrics doc
Browse files Browse the repository at this point in the history
  • Loading branch information
jotak committed May 30, 2024
1 parent 03d8540 commit 513f4a2
Show file tree
Hide file tree
Showing 3 changed files with 339 additions and 28 deletions.
299 changes: 299 additions & 0 deletions docs/flowmetric-flows-netobserv-io-v1alpha1.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,299 @@
// Automatically generated by 'openshift-apidocs-gen'. Do not edit.
:_mod-docs-content-type: REFERENCE
[id="flowmetric-flows-netobserv-io-v1alpha1"]
= FlowMetric [flows.netobserv.io/v1alpha1]



Description::
+
--
FlowMetric is the Schema for the flowmetrics API
--

Type::
`object`




[cols="1,1,1",options="header"]
|===
| Property | Type | Description

| `apiVersion`
| `string`
| APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and might reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources

| `kind`
| `string`
| Kind is a string value representing the REST resource this object represents. Servers might infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds

| `metadata`
| `object`
| Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata

| `spec`
| `object`
| FlowMetricSpec defines the desired state of FlowMetric
The provided API allows you to customize these metrics according to your needs. +

When adding new metrics or modifying existing labels, you must carefully monitor the memory
usage of Prometheus workloads as this could potentially have a high impact. Cf https://rhobs-handbook.netlify.app/products/openshiftmonitoring/telemetry.md/#what-is-the-cardinality-of-a-metric +

To check the cardinality of all Network Observability metrics, run as `promql`: `count({__name__=~"netobserv.*"}) by (__name__)`.

|===
== .metadata
Description::
+
--
Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata
--

Type::
`object`




== .spec
Description::
+
--
FlowMetricSpec defines the desired state of FlowMetric
The provided API allows you to customize these metrics according to your needs. +

When adding new metrics or modifying existing labels, you must carefully monitor the memory
usage of Prometheus workloads as this could potentially have a high impact. Cf https://rhobs-handbook.netlify.app/products/openshiftmonitoring/telemetry.md/#what-is-the-cardinality-of-a-metric +

To check the cardinality of all Network Observability metrics, run as `promql`: `count({__name__=~"netobserv.*"}) by (__name__)`.
--

Type::
`object`

Required::
- `metricName`
- `type`



[cols="1,1,1",options="header"]
|===
| Property | Type | Description

| `buckets`
| `array (string)`
| A list of buckets to use when `type` is "Histogram". The list must be parseable as floats. When not set, Prometheus default buckets are used.

| `charts`
| `array`
| Charts configuration, for the {product-title} Console in the administrator view, Dashboards menu.

| `direction`
| `string`
| Filter for ingress, egress or any direction flows.
When set to `Ingress`, it is equivalent to adding the regex filter on `FlowDirection`: `0\|2`.
When set to `Egress`, it is equivalent to adding the regex filter on `FlowDirection`: `1\|2`.

| `divider`
| `string`
| When non-zero, scale factor (divider) of the value. Metric value = Flow value / Divider.

| `filters`
| `array`
| `filters` is a list of fields and values used to restrict which flows are taken into account. Oftentimes, these filters must
be used to eliminate duplicates: `Duplicate != "true"` and `FlowDirection = "0"`.
Refer to the documentation for the list of available fields: https://docs.openshift.com/container-platform/latest/observability/network_observability/json-flows-format-reference.html.

| `labels`
| `array (string)`
| `labels` is a list of fields that should be used as Prometheus labels, also known as dimensions.
From choosing labels results the level of granularity of this metric, as well as the available aggregations at query time.
It must be done carefully as it impacts the metric cardinality (cf https://rhobs-handbook.netlify.app/products/openshiftmonitoring/telemetry.md/#what-is-the-cardinality-of-a-metric).
In general, avoid setting very high cardinality labels such as IP or MAC addresses.
"SrcK8S_OwnerName" or "DstK8S_OwnerName" should be preferred over "SrcK8S_Name" or "DstK8S_Name" as much as possible.
Refer to the documentation for the list of available fields: https://docs.openshift.com/container-platform/latest/observability/network_observability/json-flows-format-reference.html.

| `metricName`
| `string`
| Name of the metric. In Prometheus, it is automatically prefixed with "netobserv_".

| `type`
| `string`
| Metric type: "Counter" or "Histogram".
Use "Counter" for any value that increases over time and on which you can compute a rate, such as Bytes or Packets.
Use "Histogram" for any value that must be sampled independently, such as latencies.

| `valueField`
| `string`
| `valueField` is the flow field that must be used as a value for this metric. This field must hold numeric values.
Leave empty to count flows rather than a specific value per flow.
Refer to the documentation for the list of available fields: https://docs.openshift.com/container-platform/latest/observability/network_observability/json-flows-format-reference.html.

|===
== .spec.charts
Description::
+
--
Charts configuration, for the {product-title} Console in the administrator view, Dashboards menu.
--

Type::
`array`




== .spec.charts[]
Description::
+
--
Configures charts / dashboard generation associated to a metric
--

Type::
`object`

Required::
- `dashboardName`
- `queries`
- `title`
- `type`



[cols="1,1,1",options="header"]
|===
| Property | Type | Description

| `dashboardName`
| `string`
| Name of the containing dashboard. If this name does not refer to an existing dashboard, a new dashboard is created.

| `queries`
| `array`
| List of queries to be displayed on this chart. If `type` is `SingleStat` and multiple queries are provided,
this chart is automatically expanded in several panels (one per query).

| `sectionName`
| `string`
| Name of the containing dashboard section. If this name does not refer to an existing section, a new section is created.
If `sectionName` is omitted or empty, the chart is placed in the global top section.

| `title`
| `string`
| Title of the chart.

| `type`
| `string`
| Type of the chart.

| `unit`
| `string`
| Unit of this chart. Only a few units are currently supported. Leave empty to use generic number.

|===
== .spec.charts[].queries
Description::
+
--
List of queries to be displayed on this chart. If `type` is `SingleStat` and multiple queries are provided,
this chart is automatically expanded in several panels (one per query).
--

Type::
`array`




== .spec.charts[].queries[]
Description::
+
--
Configures PromQL queries
--

Type::
`object`

Required::
- `legend`
- `promQL`
- `top`



[cols="1,1,1",options="header"]
|===
| Property | Type | Description

| `legend`
| `string`
| The query legend that applies to each timeseries represented in this chart. When multiple timeseries are displayed, you should set a legend
that distinguishes each of them. It can be done with the following format: `{{ Label }}`. For example, if the `promQL` groups timeseries per
label such as: `sum(rate($METRIC[2m])) by (Label1, Label2)`, you might write as the legend: `Label1={{ Label1 }}, Label2={{ Label2 }}`.

| `promQL`
| `string`
| The `promQL` query to be run against Prometheus. If the chart `type` is `SingleStat`, this query should only return
a single timeseries. For other types, a top 7 is displayed.
You can use `$METRIC` to refer to the metric defined in this resource. For example: `sum(rate($METRIC[2m]))`.
To learn more about `promQL`, refer to the Prometheus documentation: https://prometheus.io/docs/prometheus/latest/querying/basics/

| `top`
| `integer`
| Top N series to display per timestamp. Does not apply to `SingleStat` chart type.

|===
== .spec.filters
Description::
+
--
`filters` is a list of fields and values used to restrict which flows are taken into account. Oftentimes, these filters must
be used to eliminate duplicates: `Duplicate != "true"` and `FlowDirection = "0"`.
Refer to the documentation for the list of available fields: https://docs.openshift.com/container-platform/latest/observability/network_observability/json-flows-format-reference.html.
--

Type::
`array`




== .spec.filters[]
Description::
+
--

--

Type::
`object`

Required::
- `field`
- `matchType`



[cols="1,1,1",options="header"]
|===
| Property | Type | Description

| `field`
| `string`
| Name of the field to filter on

| `matchType`
| `string`
| Type of matching to apply

| `value`
| `string`
| Value to filter on. When `matchType` is `Equal` or `NotEqual`, you can use field injection with `$(SomeField)` to refer to any other field of the flow.

|===

3 changes: 3 additions & 0 deletions hack/asciidoc-gen-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,6 @@ apiMap:
- kind: FlowCollector
group: flows.netobserv.io
version: v1beta2
- kind: FlowMetric
group: flows.netobserv.io
version: v1alpha1
65 changes: 37 additions & 28 deletions hack/asciidoc-gen.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
set -e

mkdir -p _tmp
oc get --raw /openapi/v2 | jq . > _tmp/openapi.json
oc get --raw /openapi/v2 | jq . > _tmp/openapi.1.json

jq '.definitions |= ({"io.netobserv.flows.v1beta2.FlowCollector"})
jq '.definitions |= ({"io.netobserv.flows.v1beta2.FlowCollector", "io.netobserv.flows.v1alpha1.FlowMetric"})
| del(.definitions."io.netobserv.flows.v1beta2.FlowCollector".properties.status)
| del(.definitions."io.netobserv.flows.v1beta2.FlowCollector".properties.metadata."$ref")
| .definitions."io.netobserv.flows.v1beta2.FlowCollector".properties.metadata += {type:"object"}
Expand All @@ -22,29 +22,38 @@ jq '.definitions |= ({"io.netobserv.flows.v1beta2.FlowCollector"})
| del(.definitions."io.netobserv.flows.v1beta2.FlowCollector".properties.spec.properties.consolePlugin.properties.advanced.properties.scheduling.properties.tolerations.items)
| del(.definitions."io.netobserv.flows.v1beta2.FlowCollector".properties.spec.properties.processor.properties.kafkaConsumerAutoscaler.properties)
| .definitions."io.netobserv.flows.v1beta2.FlowCollector".properties.spec.properties.consolePlugin.properties.autoscaler.description |= . + " Refer to HorizontalPodAutoscaler documentation (autoscaling/v2)."
| .definitions."io.netobserv.flows.v1beta2.FlowCollector".properties.spec.properties.processor.properties.kafkaConsumerAutoscaler.description |= . + " Refer to HorizontalPodAutoscaler documentation (autoscaling/v2)."' \
_tmp/openapi.json > _tmp/openapi-amended.json

openshift-apidocs-gen build -c hack/asciidoc-gen-config.yaml _tmp/openapi-amended.json

ADOC=docs/flowcollector-flows-netobserv-io-v1beta2.adoc

mv _tmp/flows_netobserv_io/flowcollector-flows-netobserv-io-v1beta2.adoc $ADOC

sed -i -r 's/^:_content-type: ASSEMBLY$/:_mod-docs-content-type: REFERENCE/' $ADOC
sed -i -r 's/^\[id="flowcollector-flows-netobserv-io-v.+"\]$/[id="network-observability-flowcollector-api-specifications_{context}"]/' $ADOC
sed -i -r 's/= FlowCollector \[flows.netobserv.io.*/= FlowCollector API specifications/' $ADOC
sed -i -r '/^:toc: macro$/d ' $ADOC
sed -i -r '/^:toc-title:$/d ' $ADOC
sed -i -r '/^toc::\[\]$/d ' $ADOC
sed -i -r '/^== Specification$/d ' $ADOC
sed -i -r 's/^==/=/g' $ADOC
sed -i -r '/^= API endpoints/Q' $ADOC
sed -i -r 's/OpenShift/{product-title}/g' $ADOC
sed -i -r 's/\<NetObserv\>/Network Observability/g' $ADOC
sed -i -r 's/<br>/ +\n/g' $ADOC
sed -i -r 's/<i>/_/g' $ADOC
sed -i -r 's/<\/i>/_/g' $ADOC
sed -i -r 's/ may / might /g' $ADOC
# Our asciidoc gen doesn't handle arrays very well, producing duplicate fields... so remove one of them
sed -i -r '/^\| `.+\[\]`$/,+3d' $ADOC
| .definitions."io.netobserv.flows.v1beta2.FlowCollector".properties.spec.properties.processor.properties.kafkaConsumerAutoscaler.description |= . + " Refer to HorizontalPodAutoscaler documentation (autoscaling/v2)."
| del(.definitions."io.netobserv.flows.v1alpha1.FlowMetric".properties.status)
| del(.definitions."io.netobserv.flows.v1alpha1.FlowMetric".properties.metadata."$ref")
| .definitions."io.netobserv.flows.v1alpha1.FlowMetric".properties.metadata += {type:"object"}' \
_tmp/openapi.1.json > _tmp/openapi.2.json

openshift-apidocs-gen build -c hack/asciidoc-gen-config.yaml _tmp/openapi.2.json


amend_doc() {
local filename=$1

mv _tmp/flows_netobserv_io/$filename docs/$filename

sed -i -r 's/^:_content-type: ASSEMBLY$/:_mod-docs-content-type: REFERENCE/' docs/$filename
sed -i -r 's/^\[id="flowcollector-flows-netobserv-io-v.+"\]$/[id="network-observability-flowcollector-api-specifications_{context}"]/' docs/$filename
sed -i -r 's/= FlowCollector \[flows.netobserv.io.*/= FlowCollector API specifications/' docs/$filename
sed -i -r '/^:toc: macro$/d ' docs/$filename
sed -i -r '/^:toc-title:$/d ' docs/$filename
sed -i -r '/^toc::\[\]$/d ' docs/$filename
sed -i -r '/^== Specification$/d ' docs/$filename
sed -i -r 's/^==/=/g' docs/$filename
sed -i -r '/^= API endpoints/Q' docs/$filename
sed -i -r 's/OpenShift/{product-title}/g' docs/$filename
sed -i -r 's/\<NetObserv\>/Network Observability/g' docs/$filename
sed -i -r 's/<br>/ +\n/g' docs/$filename
sed -i -r 's/<i>/_/g' docs/$filename
sed -i -r 's/<\/i>/_/g' docs/$filename
sed -i -r 's/ may / might /g' docs/$filename
# Our asciidoc gen doesn't handle arrays very well, producing duplicate fields... so remove one of them
sed -i -r '/^\| `.+\[\]`$/,+3d' docs/$filename
}

amend_doc "flowcollector-flows-netobserv-io-v1beta2.adoc"
amend_doc "flowmetric-flows-netobserv-io-v1alpha1.adoc"

0 comments on commit 513f4a2

Please sign in to comment.