Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add protobuf via gRPC exporting support for Jaeger #1471

Merged
merged 46 commits into from
Jan 4, 2021
Merged
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
0e9976c
Add grpc and proto dependencies
srikanthccv Dec 11, 2020
b6b25bd
Add grpc client and move thrift convert logic to separate file
srikanthccv Dec 11, 2020
f431dbe
Rename test file
srikanthccv Dec 11, 2020
dc9b59a
Add jaeger proto files
srikanthccv Dec 11, 2020
4da0209
Add generated files
srikanthccv Dec 11, 2020
e8673fa
Create separate package for thrift and protobuf translation
srikanthccv Dec 11, 2020
c1a5e61
Move AgentClient and Collector to package
srikanthccv Dec 11, 2020
e34f5d9
Add tests for jaeger and protobuf
srikanthccv Dec 11, 2020
84533a6
WIP
srikanthccv Dec 12, 2020
47570ff
Fix translation bugs
srikanthccv Dec 12, 2020
3dc0ede
Add function for trace_id and span_id to bytes
srikanthccv Dec 12, 2020
ff506a8
Fix tests
srikanthccv Dec 12, 2020
cfef392
Update documentation
srikanthccv Dec 13, 2020
b435a12
Remove unused imports
srikanthccv Dec 13, 2020
58497bf
Update translations
srikanthccv Dec 13, 2020
07771eb
Add gRPC example
srikanthccv Dec 13, 2020
ec310ac
Fix tests
srikanthccv Dec 13, 2020
8e98f5a
Fix <= p36 tests
srikanthccv Dec 13, 2020
b091816
Remove fields as order fails the test
srikanthccv Dec 13, 2020
cc9cf7f
Remove exporter example
srikanthccv Dec 13, 2020
a732ff3
Lint fixes
srikanthccv Dec 13, 2020
c1907f4
Fix tests; update doc
srikanthccv Dec 13, 2020
b730cd2
Fix py35 test
srikanthccv Dec 13, 2020
a4f655f
Fix lint
srikanthccv Dec 13, 2020
76a9a42
Bump sphinx version
srikanthccv Dec 13, 2020
a7c78fe
Bump typehints version
srikanthccv Dec 13, 2020
b78a09b
Resolve merge conflicts
srikanthccv Dec 13, 2020
5e51115
Fix docs build issue
srikanthccv Dec 13, 2020
647a211
Final changes
srikanthccv Dec 13, 2020
1aba1f9
Exclude jaeger from pypy
srikanthccv Dec 13, 2020
44635a9
Add grpc example
srikanthccv Dec 13, 2020
54aace1
Add changelog entry
srikanthccv Dec 13, 2020
cf1b278
Merge branch 'master' into jaeger-protobuf
srikanthccv Dec 15, 2020
7a13b69
Add more explanation to jaeger protobuf example
srikanthccv Dec 18, 2020
504b676
Review suggestions
srikanthccv Dec 18, 2020
a7b0f1c
Merge branch 'master' into jaeger-protobuf
srikanthccv Dec 18, 2020
add23d2
Merge branch 'master' into jaeger-protobuf
srikanthccv Dec 19, 2020
9070d73
Make attributes private
srikanthccv Dec 23, 2020
68c477c
Use .lower for format name
srikanthccv Dec 23, 2020
6aaf2a0
Merge branch 'master' into jaeger-protobuf
srikanthccv Dec 25, 2020
fab1ed1
grpc endpoint changes
srikanthccv Dec 25, 2020
87f6274
review suggestions
srikanthccv Dec 26, 2020
cf8c72c
Fix lint
srikanthccv Dec 26, 2020
f105915
Fix docs
srikanthccv Dec 26, 2020
206622f
Fix docs
srikanthccv Dec 26, 2020
070b28f
Add intersphinx mapping for grpc
srikanthccv Dec 26, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased](https://github.com/open-telemetry/opentelemetry-python/compare/v0.16b1...HEAD)

- Add protobuf via gRPC exporting support for Jaeger
([#1471](https://github.com/open-telemetry/opentelemetry-python/pull/1471))

- Add support for Python 3.9
([#1441](https://github.com/open-telemetry/opentelemetry-python/pull/1441))

Expand Down
10 changes: 10 additions & 0 deletions docs/exporter/jaeger/jaeger.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,13 @@ Submodules
:members:
:undoc-members:
:show-inheritance:

.. automodule:: opentelemetry.exporter.jaeger.send.thrift
:members:
:undoc-members:
:show-inheritance:

.. automodule:: opentelemetry.exporter.jaeger.gen.collector_pb2_grpc
:members:
:undoc-members:
:show-inheritance:
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,14 @@
# password=xxxx, # optional
)

# gRPC example
srikanthccv marked this conversation as resolved.
Show resolved Hide resolved
# jaeger_exporter = jaeger.JaegerSpanExporter(
# service_name="my-helloworld-service",
# collector_endpoint="localhost:14250",
# insecure=True,
# transport_format="protobuf",
# )

# create a BatchExportSpanProcessor and add the exporter to it
span_processor = BatchExportSpanProcessor(jaeger_exporter)

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
// Copyright (c) 2019 The Jaeger Authors.
// Copyright (c) 2018 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

syntax="proto3";

package jaeger.api_v2;

import "model.proto";
import "gogoproto/gogo.proto";
import "google/api/annotations.proto";
import "protoc-gen-swagger/options/annotations.proto";

option go_package = "api_v2";
option java_package = "io.jaegertracing.api_v2";

// Enable gogoprotobuf extensions (https://github.com/gogo/protobuf/blob/master/extensions.md).
// Enable custom Marshal method.
option (gogoproto.marshaler_all) = true;
// Enable custom Unmarshal method.
option (gogoproto.unmarshaler_all) = true;
// Enable custom Size method (Required by Marshal and Unmarshal).
option (gogoproto.sizer_all) = true;
// Enable registration with golang/protobuf for the grpc-gateway.
option (gogoproto.goproto_registration) = true;

option (grpc.gateway.protoc_gen_swagger.options.openapiv2_swagger) = {
info: {
version: "1.0";
};
external_docs: {
url: "https://github.com/jaegertracing/jaeger";
description: "Jaeger API";
}
schemes: HTTP;
schemes: HTTPS;
};

message PostSpansRequest {
Batch batch = 1 [
(gogoproto.nullable) = false
];
}

message PostSpansResponse {
}

service CollectorService {
rpc PostSpans(PostSpansRequest) returns (PostSpansResponse) {
option (google.api.http) = {
post: "/api/v2/spans"
body: "*"
};
}
}
166 changes: 166 additions & 0 deletions exporter/opentelemetry-exporter-jaeger/proto/api_v2/model.proto
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
// Copyright (c) 2018 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

syntax="proto3";

package jaeger.api_v2;

import "gogoproto/gogo.proto";
import "google/protobuf/timestamp.proto";
import "google/protobuf/duration.proto";

// TODO: document all types and fields

// TODO: once this moves to jaeger-idl repo, we may want to change Go pkg to api_v2
// and rewrite it to model only in this repo. That should make it easier to generate
// classes in other languages.
option go_package = "model";
option java_package = "io.jaegertracing.api_v2";

// Enable gogoprotobuf extensions (https://github.com/gogo/protobuf/blob/master/extensions.md).
// Enable custom Marshal method.
option (gogoproto.marshaler_all) = true;
// Enable custom Unmarshal method.
option (gogoproto.unmarshaler_all) = true;
// Enable custom Size method (Required by Marshal and Unmarshal).
option (gogoproto.sizer_all) = true;
// Enable registration with golang/protobuf for the grpc-gateway.
option (gogoproto.goproto_registration) = true;

enum ValueType {
STRING = 0;
BOOL = 1;
INT64 = 2;
FLOAT64 = 3;
BINARY = 4;
};

message KeyValue {
option (gogoproto.equal) = true;
option (gogoproto.compare) = true;

string key = 1;
ValueType v_type = 2;
string v_str = 3;
bool v_bool = 4;
int64 v_int64 = 5;
double v_float64 = 6;
bytes v_binary = 7;
}

message Log {
google.protobuf.Timestamp timestamp = 1 [
(gogoproto.stdtime) = true,
(gogoproto.nullable) = false
];
repeated KeyValue fields = 2 [
(gogoproto.nullable) = false
];
}

enum SpanRefType {
CHILD_OF = 0;
FOLLOWS_FROM = 1;
};

message SpanRef {
bytes trace_id = 1 [
(gogoproto.nullable) = false,
(gogoproto.customtype) = "TraceID",
(gogoproto.customname) = "TraceID"
];
bytes span_id = 2 [
(gogoproto.nullable) = false,
(gogoproto.customtype) = "SpanID",
(gogoproto.customname) = "SpanID"
];
SpanRefType ref_type = 3;
}

message Process {
string service_name = 1;
repeated KeyValue tags = 2 [
(gogoproto.nullable) = false
];
}

message Span {
bytes trace_id = 1 [
(gogoproto.nullable) = false,
(gogoproto.customtype) = "TraceID",
(gogoproto.customname) = "TraceID"
];
bytes span_id = 2 [
(gogoproto.nullable) = false,
(gogoproto.customtype) = "SpanID",
(gogoproto.customname) = "SpanID"
];
string operation_name = 3;
repeated SpanRef references = 4 [
(gogoproto.nullable) = false
];
uint32 flags = 5 [
(gogoproto.nullable) = false,
(gogoproto.customtype) = "Flags"
];
google.protobuf.Timestamp start_time = 6 [
(gogoproto.stdtime) = true,
(gogoproto.nullable) = false
];
google.protobuf.Duration duration = 7 [
(gogoproto.stdduration) = true,
(gogoproto.nullable) = false
];
repeated KeyValue tags = 8 [
(gogoproto.nullable) = false
];
repeated Log logs = 9 [
(gogoproto.nullable) = false
];
Process process = 10;
string process_id = 11 [
(gogoproto.customname) = "ProcessID"
];
repeated string warnings = 12;
}

message Trace {
message ProcessMapping {
string process_id = 1 [
(gogoproto.customname) = "ProcessID"
];
Process process = 2 [
(gogoproto.nullable) = false
];
}
repeated Span spans = 1;
repeated ProcessMapping process_map = 2 [
(gogoproto.nullable) = false
];
repeated string warnings = 3;
}

message Batch {
repeated Span spans = 1;
Process process = 2 [
(gogoproto.nullable) = true
];
}

message DependencyLink {
string parent = 1;
string child = 2;
uint64 call_count = 3;
string source = 4;
}
2 changes: 2 additions & 0 deletions exporter/opentelemetry-exporter-jaeger/setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ package_dir=
=src
packages=find_namespace:
install_requires =
grpcio >= 1.0.0, < 2.0.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since this isn't required for all exporter configurations, should this be an extras_require?

I personally don't care about bloating package dependencies and including unneeded ones, but I've found that to be a prickly subject for others.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is needed as we have import for from grpc and typehints in certain places.

from grpc import (
    ChannelCredentials,
    insecure_channel,
    secure_channel,
    ssl_channel_credentials,
)

googleapis-common-protos ~= 1.52.0
thrift >= 0.10.0
opentelemetry-api == 0.17.dev0
opentelemetry-sdk == 0.17.dev0
Expand Down
Loading