Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

receiver/prometheus: add ToMetricPdata method #3695

Conversation

odeke-em
Copy link
Member

This change wires together the logic to convert
the pdata converters into a method ToMetricPdata
that appends converted metrics. The next change or two,
will then hook everything up directly and will
allow us to delete all the prior code!!

Updates #3137
Updates PR #3674
Requires PR #3694
Updates #3691

@odeke-em odeke-em force-pushed the receiver-prometheus-addPdata.ToMetric branch 2 times, most recently from fbf3759 to 6addfa5 Compare July 23, 2021 21:59
@odeke-em odeke-em force-pushed the receiver-prometheus-addPdata.ToMetric branch 2 times, most recently from b06d71d to e8c0b21 Compare July 29, 2021 22:49
odeke-em added a commit to orijtech/opentelemetry-collector that referenced this pull request Jul 30, 2021
Wire up and use the direct Prometheus->Pdata conversion end to end.
With this change the receiver will no longer need OpenCensus.

This change will involve more follow-ups that just migrate over the tests,
because we don't want a super bloated/massive PR.

Fixes open-telemetry#3691
Depends on PR open-telemetry#3694
Depends on PR open-telemetry#3695
@odeke-em odeke-em force-pushed the receiver-prometheus-addPdata.ToMetric branch from e8c0b21 to 0b8fd4a Compare July 30, 2021 01:41
odeke-em added a commit to orijtech/opentelemetry-collector that referenced this pull request Jul 30, 2021
Wire up and use the direct Prometheus->Pdata conversion end to end.
With this change the receiver will no longer need OpenCensus.

This change will involve more follow-ups that just migrate over the tests,
because we don't want a super bloated/massive PR.

Fixes open-telemetry#3691
Depends on PR open-telemetry#3694
Depends on PR open-telemetry#3695
@odeke-em odeke-em marked this pull request as ready for review July 30, 2021 01:51
@odeke-em odeke-em requested a review from a team July 30, 2021 01:51
@odeke-em odeke-em force-pushed the receiver-prometheus-addPdata.ToMetric branch from 0b8fd4a to 9e1cef7 Compare July 30, 2021 01:55
@tigrannajaryan
Copy link
Member

@Aneurysm9 @rakyll please review.

odeke-em added a commit to orijtech/opentelemetry-collector that referenced this pull request Jul 30, 2021
Wire up and use the direct Prometheus->Pdata conversion end to end.
With this change the receiver will no longer need OpenCensus.

This change will involve more follow-ups that just migrate over the tests,
because we don't want a super bloated/massive PR.

Fixes open-telemetry#3691
Depends on PR open-telemetry#3694
Depends on PR open-telemetry#3695
}
pointCount = sdpL.Len()

default:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is missing a case for counters, but that seems to be covered in #3741.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, I added that case in the other PR but I've brought it in, please take a look again. Thank you @Aneurysm9!

odeke-em added a commit to orijtech/opentelemetry-collector that referenced this pull request Jul 31, 2021
Wire up and use the direct Prometheus->Pdata conversion end to end.
With this change the receiver will no longer need OpenCensus.

This change will involve more follow-ups that just migrate over the tests,
because we don't want a super bloated/massive PR.

Fixes open-telemetry#3691
Depends on PR open-telemetry#3694
Depends on PR open-telemetry#3695
@bogdandrutu
Copy link
Member

Please rebase

@odeke-em odeke-em force-pushed the receiver-prometheus-addPdata.ToMetric branch from 9e1cef7 to d8b80e8 Compare August 2, 2021 18:42
This change wires together the logic to convert
the pdata converters into a method ToMetricPdata
that appends converted metrics. The next change or two,
will then hook everything up directly and will allow us
to delete all the prior code!!

Updates #3137
Updates PR open-telemetry#3674
Requires PR open-telemetry#3694
Updates open-telemetry#3691
@odeke-em odeke-em force-pushed the receiver-prometheus-addPdata.ToMetric branch from d8b80e8 to 8e005a1 Compare August 2, 2021 18:44
@odeke-em
Copy link
Member Author

odeke-em commented Aug 2, 2021

Please rebase

Rebase completed, thank you and please take another look @bogdandrutu @tigrannajaryan.

@bogdandrutu
Copy link
Member

Per @Aneurysm9 approval, will merge this :) Rules of the reviewer: you are now the maintainer :)

@bogdandrutu bogdandrutu merged commit 814f4f3 into open-telemetry:main Aug 3, 2021
@odeke-em odeke-em deleted the receiver-prometheus-addPdata.ToMetric branch August 4, 2021 04:03
odeke-em added a commit to orijtech/opentelemetry-collector that referenced this pull request Aug 4, 2021
Wire up and use the direct Prometheus->Pdata conversion end to end.
With this change the receiver will no longer need OpenCensus.

This change will involve more follow-ups that just migrate over the tests,
because we don't want a super bloated/massive PR.

Fixes open-telemetry#3691
Depends on PR open-telemetry#3694
Depends on PR open-telemetry#3695
odeke-em added a commit to orijtech/opentelemetry-collector that referenced this pull request Aug 4, 2021
Wire up and use the direct Prometheus->Pdata conversion end to end.
With this change the receiver will no longer need OpenCensus.

This change will involve more follow-ups that just migrate over the tests,
because we don't want a super bloated/massive PR.

Fixes open-telemetry#3691
Depends on PR open-telemetry#3694
Depends on PR open-telemetry#3695
odeke-em added a commit to orijtech/opentelemetry-collector that referenced this pull request Aug 12, 2021
Wire up and use the direct Prometheus->Pdata conversion end to end.
With this change the receiver will no longer need OpenCensus.

This change will involve more follow-ups that just migrate over the tests,
because we don't want a super bloated/massive PR.

Fixes open-telemetry#3691
Depends on PR open-telemetry#3694
Depends on PR open-telemetry#3695
odeke-em added a commit to orijtech/opentelemetry-collector that referenced this pull request Aug 17, 2021
Wire up and use the direct Prometheus->Pdata conversion end to end.
With this change the receiver will no longer need OpenCensus.

This change will involve more follow-ups that just migrate over the tests,
because we don't want a super bloated/massive PR.

Fixes open-telemetry#3691
Depends on PR open-telemetry#3694
Depends on PR open-telemetry#3695
odeke-em added a commit to orijtech/opentelemetry-collector-contrib that referenced this pull request Sep 21, 2021
…thout OpenCensus

Wire up and use the direct Prometheus->Pdata conversion end to end.
With this change the receiver will no longer need OpenCensus.

This change will involve more follow-ups that just migrate over the tests,
because we don't want a super bloated/massive PR.

Fixes #4892
Depends on PR open-telemetry/opentelemetry-collector#3694
Depends on PR open-telemetry/opentelemetry-collector#3695
bogdandrutu pushed a commit to open-telemetry/opentelemetry-collector-contrib that referenced this pull request Dec 8, 2021
* receiver/prometheus: roundtrip Prometheus->Pdata direct conversion without OpenCensus

Wire up and use the direct Prometheus->Pdata conversion end to end.
With this change the receiver will no longer need OpenCensus.

This change will involve more follow-ups that just migrate over the tests,
because we don't want a super bloated/massive PR.

Fixes #4892
Depends on PR open-telemetry/opentelemetry-collector#3694
Depends on PR open-telemetry/opentelemetry-collector#3695

* Retrofit top level package for pdata comparisons

* Reuse for pdata metrics creators from PR #5231

* Shed all in-code direct references to OpenCensus

* receiver/prometheus: fix tests

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* receiver/prometheus: buffer metrics in builder to avoid issues with out-of-order exposition

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* receiver/prometheus: restore prom->OC->pdata pipeline

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* receiver/prometheus: add feature gate to control whether to use OpenCensus or pdata directly

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* receiver/prometheus: localize metrics test helpers

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* receiver/prometheus: store metrics in a map in metricbuilder to avoid issues with out-of-order exposition data

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* receiver/prometheus: run component tests against both OC and pdata export paths

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* receiver/prometheus: ensure OC metrics builder tests are deterministic

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* receiver/prometheus: restore replace directive to ensure use of local copy of opencensus translator during development

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* receiver/prometheus: fix lint issues

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* receiver/prometheus: fix lint issues

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* make porto

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* receiver/prometheus: run e2e tests in parallel to avoid doubling test suite runtime

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* Revert "receiver/prometheus: run e2e tests in parallel to avoid doubling test suite runtime"

This reverts commit 7e04b44.

* receiver/prometheus: address PR feedback

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* receiver/prometheus: ensure test helper can extract timestamp from metrics of any type

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* receiver/prometheus: tests should not be expecting empty label values

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* make porto

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* receiver/prometheus: remove gauge histogram pdata adjustment tests

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* receiver/prometheus: remove unused gauge histogram data generator helper

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* Fix lint errors

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* receiver/prometheus: Add comments detailing use of locks in metrics adjuster.

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

Co-authored-by: Emmanuel T Odeke <emmanuel@orijtech.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants