Cherry-pick #14143 to 7.5: [Metricbeat] Add FailOnRequired when applying schema #14202
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Add FailOnRequired when applying schema (cherry picked from commit 09bc35a)
jsoriano
approved these changes
Oct 23, 2019
| @@ -68,7 +68,7 @@ func (m *MetricSet) Fetch(reporter mb.ReporterV2) error { | |||
| return errors.Wrap(err, "failed to retrieve serverStatus") | |||
| } | |||
|
|
|||
| data, err := schema.Apply(result) | |||
| data, err := schemaMetrics.Apply(result, schema.FailOnRequired) | |||
There was a problem hiding this comment.
Take into account that with this what we are doing is to ignore all keys errors, because there are no fields marked as required.
There was a problem hiding this comment.
Yes, I tried at the beginning to add c.DictOptional and s.Optional to the fields that reported in error message: 0f247c4#diff-381ff88259685023d10591351a678e8dR57
But then I realized every time in the error message, fields are different so I ended up adding a lot of c.DictOptional and s.Optional to almost all fields 😂
Do you know if there are fields that are required from metrics metricset? If so, we should add the required label in data.go.
leweafan
pushed a commit
to leweafan/beats
that referenced
this pull request
Apr 28, 2023
…n applying schema (elastic#14202) * [Metricbeat] Add FailOnRequired when applying schema (elastic#14143) * Add FailOnRequired when applying schema (cherry picked from commit 972900f)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cherry-pick of PR #14143 to 7.5 branch. Original message:
Original problem reported is when running metricbeat mongodb module,
metricsmetricset failled with:I was able to reproduce it and turned out this error message changes with different keys not found when I reproduced it.
This error should be fixed by adding
schema.FailOnRequiredwhen applying the schema.Second problem is after adding
schema.FailOnRequired, a warning showed up when trying to runmetricsmetricset:I believe some metrics in
metricsmetricset are missing.countsuffix, likemongodb.metrics.query_executor.scanned_indexes.How to test this:
Please start mongodb locally and start Metricbeat with mongodb module enabled. Make sure metrics are flowing with no error above. You might see some error/warning message at INFO level indicating permission error but that's not related to this.