Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve stats api #27963

Merged
merged 30 commits into from
Oct 20, 2021
Merged

Improve stats api #27963

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
47e0935
singleton sysinfo host to avoid frequently collecting host info
newly12 Sep 16, 2021
165b149
Merge branch 'elastic:master' into improve_stats_api
newly12 Sep 16, 2021
9100f05
add Host object to Stats object
newly12 Sep 24, 2021
e0f8dae
Merge branch 'elastic:master' into improve_stats_api
newly12 Sep 27, 2021
1694024
update changelog
newly12 Sep 27, 2021
30e78c2
set procStats.host to nil if any error calling sysinfo.Host()
newly12 Oct 6, 2021
88e81a6
Merge branch 'master' into improve_stats_api
newly12 Oct 7, 2021
902d579
Update aws-lambda-go library version to 1.13.3 (#28236)
kaiyan-sheng Oct 7, 2021
80b9e28
[cloud][docker] use the private docker namespace (#28286)
v1v Oct 7, 2021
358842b
[7.x] [DOCS] Update api_key example on elasticsearch output (#28288)
ugosan Oct 7, 2021
f80d1ac
packetbeat/protos/dns: don't render missing A and AAAA addresses from…
efd6 Oct 10, 2021
eccac0e
seccomp: allow clone3 syscall for x86 (#28117)
BlackYoup Oct 11, 2021
59a5238
Osquerybeat: Improve handling of osquery.autoload file, allow customi…
aleksmaus Oct 11, 2021
7491d08
Osquerybeat: Runner and Fetcher unit tests (#28290)
aleksmaus Oct 11, 2021
8a1ff9e
Update go release version 1.17.1 (#27543)
apmmachine Oct 11, 2021
3e5fe14
Move labels and annotations under kubernetes.namespace. (#27917)
MichaelKatsoulis Oct 11, 2021
056c4d7
Remove GCP support from Functionbeat (#28253)
kvch Oct 11, 2021
5b26c25
Fix build tags for Go 1.17 (#28338)
aleksmaus Oct 11, 2021
b191b59
[Elastic Agent] Add ability to communicate with Kibana through servic…
blakerouse Oct 11, 2021
d416312
Add username to ASA Security negotiation log (#26975)
LaZyDK Oct 11, 2021
c77da38
x-pack/filebeat/module/cisco: loosen time parsing and add group and s…
efd6 Oct 11, 2021
a8c1ecd
Redis: remove deprecated fields (#28246)
Oct 12, 2021
b278380
Disable generator tests temporarily (#28362)
kvch Oct 12, 2021
e6780a5
Windows/perfmon metricset - remove deprecated perfmon.counters confi…
narph Oct 12, 2021
f23aff7
[Filebeat] - S3 Input - Add support for only iterating/accessing only…
Oct 12, 2021
69f2780
Breaking change for 8.0, namespace_annotations replaced by namespace.…
MichaelKatsoulis Oct 12, 2021
3fe0fed
[Heartbeat] Setuid to regular user / lower capabilities when possible…
andrewvc Oct 13, 2021
172c6ba
Merge remote-tracking branch 'upstream/master' into improve_stats_api
newly12 Oct 13, 2021
6ecb286
Merge remote-tracking branch 'upstream/master' into improve_stats_api
newly12 Oct 14, 2021
a2c536c
mage fmt
newly12 Oct 19, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Remove deprecated/undocumented IncludeCreatorMetadata setting from kubernetes metadata config options {pull}28006[28006]
- Remove deprecated fields from kubernetes module {pull}28046[28046]
- Remove deprecated config option aws_partition. {pull}28120[28120]
- Improve stats API {pull}27963[27963]
- Enable IMDSv2 support for `add_cloud_metadata` processor on AWS. {issue}22101[22101] {pull}28285[28285]
- Update kubernetes.namespace from keyword to group field and add name, labels, annotations, uuid as its fields {pull}27917[27917]

Expand Down
21 changes: 14 additions & 7 deletions libbeat/metric/system/process/process.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ import (

"github.com/pkg/errors"

"github.com/elastic/go-sysinfo/types"

"github.com/elastic/beats/v7/libbeat/common"
"github.com/elastic/beats/v7/libbeat/common/match"
"github.com/elastic/beats/v7/libbeat/logp"
Expand Down Expand Up @@ -61,7 +63,7 @@ type Process struct {
FD sigar.ProcFDUsage
Env common.MapStr

//cpu stats
// cpu stats
cpuSinceStart float64
cpuTotalPct float64
cpuTotalPctNorm float64
Expand Down Expand Up @@ -95,6 +97,7 @@ type Stats struct {
envRegexps []match.Matcher // List of regular expressions used to whitelist env vars.
cgroups *cgroup.Reader
logger *logp.Logger
host types.Host
}

// Ticks of CPU for a process
Expand Down Expand Up @@ -293,15 +296,11 @@ func GetOwnResourceUsageTimeInMillis() (int64, int64, error) {
}

func (procStats *Stats) getProcessEvent(process *Process) common.MapStr {

// This is a holdover until we migrate this library to metricbeat/internal
// At which point we'll use the memory code there.
var totalPhyMem uint64
host, err := sysinfo.Host()
if err != nil {
procStats.logger.Warnf("Getting host details: %v", err)
} else {
memStats, err := host.Memory()
if procStats.host != nil {
memStats, err := procStats.host.Memory()
if err != nil {
procStats.logger.Warnf("Getting memory details: %v", err)
} else {
Expand Down Expand Up @@ -427,6 +426,14 @@ func (procStats *Stats) matchProcess(name string) bool {
// cannot be compiled.
func (procStats *Stats) Init() error {
procStats.logger = logp.NewLogger("processes")

var err error
procStats.host, err = sysinfo.Host()
if err != nil {
procStats.host = nil
procStats.logger.Warnf("Getting host details: %v", err)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I 100% trust the behavior of go-sysinfo here. If we get an error, we may want to just set procStats.host to nil ourselves.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure. updated.


procStats.ProcsMap = make(ProcsMap)

if len(procStats.Procs) == 0 {
Expand Down