Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Distributor: Refactor OTLP handler #6719

Merged
merged 11 commits into from
Nov 29, 2023
Merged

Distributor: Refactor OTLP handler #6719

merged 11 commits into from
Nov 29, 2023

Conversation

aknuds1
Copy link
Contributor

@aknuds1 aknuds1 commented Nov 23, 2023 •
edited
Loading

What this PR does

Refactor distributor OTLP handler to use util.ParseProtoReader, to consolidate with the normal push handler.

There's also some optimization of the OTLP handler in the bargain:

✗ benchstat otlp-proto.txt otlp-proto-refactor.txt
goos: linux
goarch: amd64
pkg: github.com/grafana/mimir/pkg/distributor
cpu: AMD Ryzen 9 3950X 16-Core Processor            
                        │ otlp-proto.txt │      otlp-proto-refactor.txt      │
                        │     sec/op     │   sec/op     vs base              │
OTLPHandler/protobuf-32     1039.2µ ± 2%   960.3µ ± 2%  -7.59% (p=0.002 n=6)

                        │ otlp-proto.txt │       otlp-proto-refactor.txt       │
                        │      B/op      │     B/op      vs base               │
OTLPHandler/protobuf-32     532.3Ki ± 0%   471.6Ki ± 0%  -11.41% (p=0.002 n=6)

                        │ otlp-proto.txt │      otlp-proto-refactor.txt      │
                        │   allocs/op    │  allocs/op   vs base              │
OTLPHandler/protobuf-32      7.141k ± 0%   7.135k ± 0%  -0.08% (p=0.002 n=6)

TODO:

  • Test in dev

Which issue(s) this PR fixes or relates to

Checklist

  • Tests updated.
  • Documentation added.
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX].
  • about-versioning.md updated with experimental features.

@aknuds1 aknuds1 force-pushed the arve/otlp-benchmark branch 3 times, most recently from 9413fcd to 9affe4f Compare November 23, 2023 15:43
@aknuds1 aknuds1 changed the title WIP: Refactor OTLP handler + add benchmark WIP: Refactor OTLP handler Nov 23, 2023
@aknuds1
distributor.OTLPHandler: Use util.ParseProtoReader
b09612a 
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
@aknuds1 aknuds1 changed the title WIP: Refactor OTLP handler Refactor OTLP handler Nov 23, 2023
@aknuds1 aknuds1 marked this pull request as ready for review November 23, 2023 16:08
@aknuds1 aknuds1 requested a review from a team as a code owner November 23, 2023 16:08
@aknuds1 aknuds1 added the chore label Nov 23, 2023
@aknuds1 aknuds1 changed the title Refactor OTLP handler Distributor: Refactor OTLP handler Nov 23, 2023
@aknuds1
Merge remote-tracking branch 'origin/main' into arve/otlp-benchmark
895f720 
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
fayzal-g
fayzal-g reviewed Nov 24, 2023
View reviewed changes
pkg/distributor/otel.go Outdated Show resolved Hide resolved
pkg/util/http.go Outdated Show resolved Hide resolved
@aknuds1
Merge remote-tracking branch 'origin/main' into arve/otlp-benchmark
f81974f 
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
@aknuds1 aknuds1 force-pushed the arve/otlp-benchmark branch 2 times, most recently from 554c641 to 0a166f2 Compare November 29, 2023 12:35
@aknuds1
Tweak util.ParseProtoReader based on pstibrany input
ca7f811 
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
pstibrany
pstibrany reviewed Nov 29, 2023
View reviewed changes
pkg/util/http.go Outdated Show resolved Hide resolved
pkg/util/http.go Show resolved Hide resolved
aknuds1 and others added 3 commits November 29, 2023 15:31
@aknuds1
Add util.ParseProtoReader test cases
38c24bc 
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
@aknuds1 @pstibrany
Update pkg/util/http.go
b4c618e 
Co-authored-by: Peter Štibraný <pstibrany@gmail.com>
@aknuds1
Implement review feedback
f74ec06 
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
pstibrany
pstibrany approved these changes Nov 29, 2023
View reviewed changes
Copy link
Member

@pstibrany pstibrany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for addressing my feedback.

@aknuds1 aknuds1 merged commit 62ab3bf into main Nov 29, 2023
28 checks passed
@aknuds1 aknuds1 deleted the arve/otlp-benchmark branch November 29, 2023 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pstibrany pstibrany pstibrany approved these changes

@fayzal-g fayzal-g Awaiting requested review from fayzal-g

Assignees
No one assigned
Labels
area/opentelemetry chore component/distributor
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@aknuds1 @pstibrany @fayzal-g