fix(js/plugins/google-cloud): Make metric export time intervals mutually exclusive #1869
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The short of it:
Modify the startTime of the datapoints so that we ensure there are no overlapping export intervals.
Add a shutdown hook to our metrics wrapper ensures we are awaiting the callback function from the wrapped cloud exporter which delegates to an async function with a callback.
The long of it:
We were seeing occasions where exported metrics are being overwritten. This is because Cloud does not currently support the delta metric kind for custom metrics. Instead, it will convert delta metrics into "pseudo delta" metrics by calling them cumulative for the time interval provided. Where we run into trouble is that the PeriodicMetricExporterReader creates a time interval based on the last export, setting the start time of the subsequent export to the end time of the previous export. The overlap in time period means that the last in wins and is counted as the cumulative amount, resulting in what appear to be dropped metrics or sometimes, even more confusingly, metrics that increase one minute and decrease the next.
Manually tested locally using
forceDevExport = true
to make sure metrics are still being exported.Checklist (if applicable):