ECK and Beats

[pebrc]

Beats seems like a logical addition when running ECK in production scenarios. We are currently tracking a few cosmetic improvements around auto-discovery in #1978 and ECS in #2002 but maybe it would be appropriate to go one step further and also deploy Beats via ECK and a new CRD?

Current status of our work related to Beats:

• We are implementing a new generic CRD for Beats in [META] Beat CRD #3040
• We will build a stack monitoring CRD on top of that generic Beat support Stack monitoring with Metricbeat #2415
• We are monitoring the developments around agent and will transition to that in time

[roncohen]

hey @pebrc. Do you see this as a way to monitor ES/Kibana/APM Server deployed in ECK using beats - aka stack monitoring? or are you thinking of this as a way to have the operator manage beats installed across the users kubernetes cluster?

[exekias]

Pinging @elastic/beats for awareness

[ph]

ping @ycombinator since you might be interested in that :)

[ycombinator]

Pinging @elastic/stack-monitoring for awareness.

Also, while I was on the Stack Monitoring team I worked on a POC of running 1 Metricbeat and 1 Filebeat per pod to monitor an Elasticsearch node running in the same pod: https://github.com/elastic/stack-monitoring-dev/pull/54. It's just a POC so definitely a very naive implementation but just thought I'd mention it as it seemed relevant to this discussion.

[anyasabo]

Do you see this as a way to monitor ES/Kibana/APM Server deployed in ECK using beats - aka stack monitoring? or are you thinking of this as a way to have the operator manage beats installed across the users kubernetes cluster?

I think both. I would really like to get to a point where we can have something like kube-prometheus. We have a lot of the pieces but even I have trouble putting them together.

[pebrc]

@roncohen what @anyasabo said. I think both aspects are relevant in general but this issue was meant to be mainly about deploying beats in the k8s cluster. We can split this up into two issues if we are conflating things in the discussion too much.

[pebrc]

I created a separate issues for

• stack monitoring Stack monitoring with Metricbeat #2415 as it seems to be a distinct problem that could be solved independently from running Beats on k8s
• and another issue Running Beats for k8s monitoring #2417 for solving how to run Beats in general on k8s in combination with ECK e.g. for system/Kubernetes monitoring.

[anyasabo]

Right now I'm not totally convinced they should be separate problems -- ES emits prometheus metrics so it seems like we could treat them all the same. The stack is just another application with metrics that are consumed by a beat. I'm good with approaching them separately though and seeing if they end up aligning

[GTB3NW]

Hey, I'm sampling the beats config at the moment. I'm going to be using prometheus for my metrics and wouldn't want to be forced to store metricbeats as well. Could you please take this into consideration and allow users to select which parts of beats they want? It would be greatly appreciated.

Cheers!

[pebrc]

I am closing this as the we have an implementation for Beats now. There might be need/interest to support Beats for Stack monitoring in non-privileged mode, but let's open a new issue for that.