Support enableRemoteWriteReceiver flag in prometheus{,K8s} config

[jirislav]

I need to deliver metrics from the k6-operator (testing framework from Grafana, runs in jobs) to Prometheus.

At the time of writing this issue, K6 officially supports only the Prometheus remote write as the only way to get the metrics there.

Since the CMO uses CoreOS Prometheus operator under the hood, it should be possible to extend the configuration in such a way so that the following is possible:

apiVersion: monitoring.coreos.com/v1
kind: Prometheus
spec:
  enableRemoteWriteReceiver: true

By default, Prometheus has this feature disabled, resulting in 404 Not Found when trying to use the API:

{"error":"got status code: 404 instead expected a 2xx successful status code","level":"error","msg":"Failed to send the time series data to the endpoint","output":"Prometheus remote write"}

I would expect this to be configured in the Cluster Monitoring Operator configuration in a way similar to these drafts:

kind: ConfigMap
apiVersion: v1
data:
  config.yaml: |
    prometheusK8s:
      enableRemoteWriteReceiver: true

kind: ConfigMap
apiVersion: v1
data:
  config.yaml: |
    prometheus:
      enableRemoteWriteReceiver: true

[jakovrebic]

@jirislav Any news on this? I'm having exact same issue only trying to connect Grafana Tempo as remote write source to Prometheus

[jirislav]

@jirislav Any news on this? I'm having exact same issue only trying to connect Grafana Tempo as remote write source to Prometheus

No, I have no news, we ended up not using the integration, sadly.

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[jirislav]

/remove-lifecycle stale

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[jirislav]

/remove-lifecycle stale