Couldn't get all the processes data running in Metricbeats Windows Module

[prakash1243]

Hi @narph -
I took the snapshot build and tested the metricbeats, I encountered an issue with the Windows Perfmon Module, metricbeats couldn't able to pull all the processes data running on the server, lets say if I restart a process, metricbeats is not able to capture the data for that particular restarted process unless I restart the metricbeats, Please let me know if you need any additional information. Thanks !

[narph]

@prakash1243 , I have briefly tested this scenario and could not reproduce the issue.
What is the exact operating system you are running metricbeat on? (will try to recreate a simialr env)
Is there a specific process you are noticing this or any?
The metricbeat configuration configuration will be helpful here as well.
Are you noticing any messages in the metricbeat logs when the process is restarted?

When metricbeat starts it will calculate all possible counter paths based on the configuration set and add them to a list, it will then fetch the counter values based on the interval set and that counter list. If a process was stopped when metricbeat has started there will be a chance it is not included in the list of counter paths.
Is this the case?

[prakash1243]

@narph - Am using Windows 2012R2 64 bit server.
I just have a question:

• How does it handle a new process being started while metric beat Is already running?
  Please let me know.

[prakash1243]

@narph - any response to my above question. Thanks !

[narph]

hi @prakash1243,

Regarding:

How does it handle a new process being started while metric beat Is already running?
Please let me know.

I have briefly described the scenario above:

When metricbeat starts it will calculate all possible counter paths based on the configuration set and add them to a list, it will then fetch the counter values based on the interval set and that counter list. If a process was stopped when metricbeat has started there will be a chance it is not included in the list of counter paths.

The list of processes/instances we retrieve the performance counters on is calculated when metricbeat starts. At each fetch, we retrieve the counters values but we do not reload the list of processes/instances so it could be that you encounter the situation when a new process is started (for example) while metricbeat is running and is not part of the list. No performance counters are collected in this case.

To avoid this situation, the list should be reloaded/refreshed at each fetch in order to pick up any new processes/instances that started during the run.

In your case, you are describing the situation where the process has been restarted instead of just started.

lets say if I restart a process, metricbeats is not able to capture the data for that particular restarted process unless I restart the metricbeats,

Just to understand the behavior you are describing:
Was the process already running and you were seeing performance counters being collected in metricbeat until you restarted the process? Or, the process was stopped when metricbeat was running and you started the process?

[prakash1243]

@narph - Yes, we had few processes had been set to delayed start, by then metricbeats service has been started. So that was the reason we don't see the data. Is there any way that we can expect metricbeats to discover the new processes and start collecting the data? Please let me know. Thanks !

[prakash1243]

To avoid this situation, the list should be reloaded/refreshed at each fetch in order to pick up any new processes/instances that started during the run.

@narph - Exactly, how do we handle this?

[prakash1243]

Hi @narph -
Please let me know if this should be a new PR or this defect to be closed? Thanks !

[narph]

hi @prakash1243, a new PR will be necessary in order to add this refresh list option

[prakash1243]

@narph - There seems to be high in CPU Usage after adding the refresh list in Perf counters in every fetch and some unrealsitic data found in for couple of counters. Do you want me to open a new defect for this ?

[narph]

@prakash1243 , yes, can you create a new issue for this and also provide the following for testing:

• current configuration
• os version
• details regarding unrealsitic data found in for couple of counters, maybe some data values etc