[Metricbeat] Kibana/stats set `exclude_usage=true` in the `init` phase #29911

afharo · 2022-01-19T11:50:05Z

What does this PR do?

It moves the full URL generation to the init phase for the Kibana-stats collection module.

Why is it important?

Since m.isUsageExcludable is calculated in the init phase, I think that it's not necessary to overwrite it on every fetch.

Also, because of the previous defer statement, when building the event later on, it showed the original URL instead of the one actually performed (including &exclude_usage=true).

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
~~- [x] I have made corresponding changes to the documentation~~
~~- [x] I have made corresponding change to the default configuration files~~
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

Q to the beats maintainers (as a Go NOOB): do we call the init method multiple times? Can this be an issue when calling the GetVersion method?

How to test this PR locally

Related issues

Bad requests from spaces saved object client during kibana startup kibana#120422

Use cases

Screenshots

Logs

mergify · 2022-01-19T11:50:15Z

This pull request does not have a backport label. Could you fix it @afharo? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

elasticmachine · 2022-01-19T13:12:27Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-01-20T09:00:21.579+0000
Duration: 82 min 53 sec
Commit: 4e54607

Test stats 🧪

Test	Results
Failed	0
Passed	9412
Skipped	2423
Total	11835

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

…ana/stats/exclude_usage

elasticmachine · 2022-01-20T09:02:48Z

Pinging @elastic/stack-monitoring (Stack monitoring)

elasticmachine · 2022-01-20T09:02:49Z

Pinging @elastic/integrations-services (Team:Services)

matschaffer

Makes sense to me! Maybe good to get @sayden to weigh-in

sayden

Please, double check the thing I commented. I also suggest to test with both metricsets at the same time, again, just to double check. Every metricset uses a different http client based on the metricset but I'd prefer to confirm this before the storm of SDH's appear.

sayden · 2022-01-24T14:25:01Z

metricbeat/module/kibana/stats/stats.go

+	// Add exclude_usage=true if the Kibana Version supports it
+	if kibana.IsUsageExcludable(kibanaVersion) {
+		origURI := statsHTTP.GetURI()
+		statsHTTP.SetURI(origURI + "&exclude_usage=true")


I'll double check this change. The code in this part is definitely obscure but if the defer statement is deleted here, it might be the case that the uri will be &exclude_usage=true&exclude_usage=true&exclude_usage=true after 3 periods.

it makes perfect sense! Is there any documentation I can use to run this PR? Or any artifacts generated during CI that I can use to test?

Nope sorry, you'll have to do a go build and setup everything 😅

oh wait you can use https://github.com/elastic/elastic-package to spin up the entire environment and then use go build with your code.

@sayden I spent a while trying to test it locally, but I'm on an M1 Mac now and there's a library that is no-longer compatible. I'd appreciate some help when you have some time 😇

I have just seen this. I'll do a try asap

mergify · 2022-02-01T15:53:06Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b metricbeat/kibana/stats/exclude_usage upstream/metricbeat/kibana/stats/exclude_usage
git merge upstream/master
git push upstream metricbeat/kibana/stats/exclude_usage

jlind23 · 2022-03-31T16:31:54Z

@sayden @matschaffer what should we do with this PR?

sayden · 2022-04-01T08:22:53Z

@klacabane you should be aware of this

matschaffer · 2022-04-04T04:36:05Z

I opened #31118 to have this reviewed and possibly merged.

afharo · 2022-04-06T13:43:45Z

Closing in favour of the new approach discussed in #31118.

[Metricbeat] Kibana/stats set exclude_usage=true in the init phase

a1a8f6d

afharo added the cleanup label Jan 19, 2022

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jan 19, 2022

mergify bot added the backport-skip Skip notification from the automated backport with mergify label Jan 19, 2022

afharo added Metricbeat Metricbeat Feature:Stack Monitoring Team:Services (Deprecated) Label for the former Integrations-Services team labels Jan 19, 2022

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jan 19, 2022

afharo added backport-v8.1.0 Automated backport with mergify backport-v8.2.0 Automated backport with mergify backport-v8.3.0 Automated backport with mergify labels Jan 19, 2022

mergify bot removed the backport-skip Skip notification from the automated backport with mergify label Jan 19, 2022

afharo added 2 commits January 20, 2022 09:57

Merge branch 'master' of github.com:elastic/beats into metricbeat/kib…

bd1c93f

…ana/stats/exclude_usage

Changelog

4e54607

afharo marked this pull request as ready for review January 20, 2022 09:02

afharo mentioned this pull request Jan 20, 2022

Bad requests from spaces saved object client during kibana startup elastic/kibana#120422

Closed

afharo requested review from a team January 20, 2022 09:36

matschaffer approved these changes Jan 24, 2022

View reviewed changes

sayden suggested changes Jan 24, 2022

View reviewed changes

mergify bot assigned afharo Feb 8, 2022

matschaffer mentioned this pull request Apr 4, 2022

[Stack Monitoring] [Kibana] Avoid querying /api/status on every cycle #31118

Open

matschaffer self-requested a review April 4, 2022 04:36

matschaffer unassigned afharo Apr 4, 2022

mergify bot assigned afharo Apr 4, 2022

matschaffer requested review from matschaffer and removed request for matschaffer April 5, 2022 00:44

afharo closed this Apr 6, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Metricbeat] Kibana/stats set `exclude_usage=true` in the `init` phase #29911

[Metricbeat] Kibana/stats set `exclude_usage=true` in the `init` phase #29911

afharo commented Jan 19, 2022 •

edited

Loading

mergify bot commented Jan 19, 2022

elasticmachine commented Jan 19, 2022 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

elasticmachine commented Jan 20, 2022

elasticmachine commented Jan 20, 2022

matschaffer left a comment

sayden left a comment

sayden Jan 24, 2022

afharo Jan 24, 2022

sayden Jan 28, 2022

sayden Jan 28, 2022

afharo Feb 2, 2022 •

edited

Loading

sayden Feb 8, 2022

mergify bot commented Feb 1, 2022

jlind23 commented Mar 31, 2022

sayden commented Apr 1, 2022

matschaffer commented Apr 4, 2022

afharo commented Apr 6, 2022

[Metricbeat] Kibana/stats set exclude_usage=true in the init phase #29911

[Metricbeat] Kibana/stats set exclude_usage=true in the init phase #29911

Conversation

afharo commented Jan 19, 2022 • edited Loading

What does this PR do?

Why is it important?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

mergify bot commented Jan 19, 2022

elasticmachine commented Jan 19, 2022 • edited by jenkins-beats-ci bot Loading

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

elasticmachine commented Jan 20, 2022

elasticmachine commented Jan 20, 2022

matschaffer left a comment

Choose a reason for hiding this comment

sayden left a comment

Choose a reason for hiding this comment

sayden Jan 24, 2022

Choose a reason for hiding this comment

afharo Jan 24, 2022

Choose a reason for hiding this comment

sayden Jan 28, 2022

Choose a reason for hiding this comment

sayden Jan 28, 2022

Choose a reason for hiding this comment

afharo Feb 2, 2022 • edited Loading

Choose a reason for hiding this comment

sayden Feb 8, 2022

Choose a reason for hiding this comment

mergify bot commented Feb 1, 2022

jlind23 commented Mar 31, 2022

sayden commented Apr 1, 2022

matschaffer commented Apr 4, 2022

afharo commented Apr 6, 2022

[Metricbeat] Kibana/stats set `exclude_usage=true` in the `init` phase #29911

[Metricbeat] Kibana/stats set `exclude_usage=true` in the `init` phase #29911

afharo commented Jan 19, 2022 •

edited

Loading

elasticmachine commented Jan 19, 2022 •

edited by jenkins-beats-ci bot

Loading

afharo Feb 2, 2022 •

edited

Loading