Span the annotation.message in alerts as YAML multiline strings.

[aallawala]

What this PR does:
This PR attempts to fix the issue where the annotation message is split across two lines instead of one. This causes an issue for our alertmanager config template to not render correctly since it splits up the printf template across the two lines.

Before:

groups:
- name: cortex_alerts
  rules:
  - alert: CortexIngesterUnhealthy
    annotations:
      message: Cortex cluster {{ $labels.cluster }}/{{ $labels.namespace }} has {{
        printf "%f" $value }} unhealthy ingester(s).
    expr: |
      min by (cluster, namespace) (cortex_ring_members{state="Unhealthy", name="ingester"}) > 0
    for: 15m
    labels:
      severity: critical

After

groups:
- name: cortex_alerts
  rules:
  - alert: CortexIngesterUnhealthy
    annotations:
      message: |
        Cortex cluster {{ $labels.cluster }}/{{ $labels.namespace }} has {{ printf "%f" $value }} unhealthy ingester(s).
    expr: |
      min by (cluster, namespace) (cortex_ring_members{state="Unhealthy", name="ingester"}) > 0
    for: 15m
    labels:
      severity: critical

Which issue(s) this PR fixes:
Fixes #

Checklist

• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

[pracucci]

Changes LGTM, but why does this even fix your issue? Could you clarify, please?

[aallawala]

Changes LGTM, but why does this even fix your issue? Could you clarify, please?

Thank you @pracucci for taking a look.

Spanning the template across multiple lines affects us since we utilize a separate Go templating layer on top of the Go templating layer expected by the alertmanager. Our templates in code without this change look something like this:

groups:
- name: cortex_alerts
  rules:
  - alert: CortexIngesterUnhealthy
    annotations:
      message: Cortex cluster {{ `{{ "{{ $labels.cluster }}" }}` }}/{{ `{{ "{{ $labels.namespace }}" }}` }} has {{ `{{ "{{
        printf \"%.*f\" $value }}" }}` }} unhealthy ingester(s).
    expr: |
      min by (cluster, namespace) (cortex_ring_members{state="Unhealthy", name="ingester"}) > 0
    for: 15m
    labels:
      severity: critical

The line delimit prior to the printf falls between a {{ }} that causes the Go templating to fail.

[pracucci]

I'm still not sure why this change outputs the message on a single line instead only cutting it. That was my question. Could you elaborate on it, please?

[aallawala]

I'm still not sure why this change outputs the message on a single line instead only cutting it. That was my question. Could you elaborate on it, please?

From what I understand from the jsonnet spec, wrapping the message with a ||| in the jsonnet template preserves the line endings whereas the single-quoted ' and double-quoted " strings are allowed to span multiple lines in the output of the jsonnet.

[pracucci]

I'm still not sure why this change outputs the message on a single line instead only cutting it. That was my question. Could you elaborate on it, please?

From what I understand from the jsonnet spec, wrapping the message with a ||| in the jsonnet template preserves the line endings whereas the single-quoted ' and double-quoted " strings are allowed to span multiple lines in the output of the jsonnet.

I didn't know it, but now it makes perfect sense.