Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding OTLP Exporter #699

Merged
merged 13 commits into from
Aug 17, 2020
Merged

Adding OTLP Exporter #699

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
98103b7
adding otlp exporter spec
Jul 13, 2020
43c2fbb
apply review feedback
Jul 14, 2020
ac663b1
add protocol, link to otel-proto and otel-collector
Jul 20, 2020
b9a356e
adding collector link
Jul 20, 2020
595e102
adding transient error definition
Jul 22, 2020
97cf514
remove link to grpc endpoint example as it can be used for http as well
Jul 22, 2020
4db5961
Update specification/trace/sdk_exporters/otlp.md
Jul 23, 2020
6bcd0b1
update link to OTLP
Jul 23, 2020
0f32de5
disable compression by default
Jul 27, 2020
48a5d54
add links to otep-35 & otep-99
Jul 29, 2020
527adea
updated endpoint to show path example
Jul 29, 2020
e0652b0
remove references to the collector
Jul 30, 2020
95a19f6
fix broken link
Aug 7, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion specification/correlationcontext/api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ OPTIONAL parameters:

### Serialization

Until the [W3C Correlation Context](https://w3c.github.io/correlation-context/) specification is recommended for use, OpenTelemetry `CorrelationContext` implementations MUST be serialized according to the [editor's draft of W3C Correlation Context as of March 27, 2020](https://github.com/w3c/correlation-context/blob/c974664b9ab4d33af6355f1f7f03a2d52c89a99e/correlation_context/HTTP_HEADER_FORMAT.md) using a vendor-specific header name to avoid collisions with the W3C Correlation Context specification should it change in the future.
Until the [W3C Correlation Context](https://w3c.github.io/baggage/) specification is recommended for use, OpenTelemetry `CorrelationContext` implementations MUST be serialized according to the [editor's draft of W3C Correlation Context as of March 27, 2020](https://github.com/w3c/correlation-context/blob/c974664b9ab4d33af6355f1f7f03a2d52c89a99e/correlation_context/HTTP_HEADER_FORMAT.md) using a vendor-specific header name to avoid collisions with the W3C Correlation Context specification should it change in the future.

#### Header Name

Expand Down
43 changes: 43 additions & 0 deletions specification/trace/sdk_exporters/otlp.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# OpenTelemetry Protocol Exporter

This document specifies the configuration options available to the OpenTelemetry Protocol ([OTLP](https://github.com/open-telemetry/oteps/blob/master/text/0035-opentelemetry-protocol.md)) `SpanExporter` and `MetricsExporter` as well as the retry behavior.

## Configuration Options

The configuration options are configurable separately for the `SpanExporter` and the `MetricsExporter`.
tigrannajaryan marked this conversation as resolved.
Show resolved Hide resolved

| Configuration Option | Description | Default | Env variable |
| -------------------- | ------------------------------------------------------------ | ----------------- | ------------------------------------------------------------ |
| Endpoint | Target to which the exporter is going to send spans or metrics. This MAY be configured to include a path (e.g. `example.com/v1/traces`). | `localhost:55680` | `OTEL_EXPORTER_OTLP_SPAN_ENDPOINT` `OTEL_EXPORTER_OTLP_METRIC_ENDPOINT` |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not completely clear what happens if the Endpoint does not include the path. Is the default path automatically appended? So if I specify Endpoint=example.com does it mean the actual endpoint is example.con/v1/trace? What if I want the path to be the root and don't want the default path to be appended?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My expectation here is that the value configured is taken verbatim and passed to the OTLP exporter in the different languages. I would not expect a path to be appended. This does raise a good question of whether a default value for http and another for grpc makes sense here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure I like this approach. We are basically not providing any easy way to use the default port and path (for http transport).

The default path for OTLP/HTTP is defined in the spec: https://github.com/open-telemetry/oteps/blob/master/text/0099-otlp-http.md#request
I think we need to honor it. It is also important to honor the default port if the user does not specify it.

I believe we need to design the env variables such that configuration is easy in the most common case and for the uncommon cases it is OK if it is a bit more difficult.

The most common case is when I need to specify only the ip address or the hostname of the server and choose the transport. Everything else I expect to be set to the correct defaults. The defaults include the port number and when using otlp/http also the path.

For example I would expect the following to result in the destination of https://example.com:55680/v1/traces:

OTEL_EXPORTER_OTLP_SPAN_ENDPOINT=example.com
OTEL_EXPORTER_OTLP_SPAN_TRANSPORT=http

If I wanted to override the path I would probably be OK if I need to specify the full URL using a different env variable and in that case I don't need to specify the ENDPOINT and TRANSPORT:

OTEL_EXPORTER_OTLP_SPAN_ENDPOINT_URL=http://example.com:1234/mytraces

There may be other ways to structure this but ultimately I want to make sure the most common case is easy and with the current PR I think we are not achieving that goal.

Perhaps it is not possible to achieve what I ask for, so feel free to ignore this comment.

| Protocol | The protocol used to transmit the data. One of `grpc`,`http/json`,`http/proto`. | `grpc` | `OTEL_EXPORTER_OTLP_SPAN_PROTOCOL` `OTEL_EXPORTER_OTLP_METRIC_PROTOCOL` |
| Insecure | Whether to enable client transport security for the exporter's `grpc` or `http` connection. | `false` | `OTEL_EXPORTER_OTLP_SPAN_INSECURE` `OTEL_EXPORTER_OTLP_METRIC_INSECURE` |
| Certificate File | Certificate file for TLS credentials of gRPC client. Should only be used if `insecure` is set to `false`. | n/a | `OTEL_EXPORTER_OTLP_SPAN_CERTIFICATE` `OTEL_EXPORTER_OTLP_METRIC_CERTIFICATE` |
| Headers | The headers associated with gRPC or HTTP requests. | n/a | `OTEL_EXPORTER_OTLP_SPAN_HEADERS` `OTEL_EXPORTER_OTLP_METRIC_HEADERS` |
| Compression | Compression key for supported compression types. Supported compression: `gzip`| no compression | `OTEL_EXPORTER_OTLP_SPAN_COMPRESSION` `OTEL_EXPORTER_OTLP_METRIC_COMPRESSION` |
| Timeout | Max waiting time for the backend to process each spans or metrics batch. | 60s | `OTEL_EXPORTER_OTLP_SPAN_TIMEOUT` `OTEL_EXPORTER_OTLP_METRIC_TIMEOUT` |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some other configuration options are exposed by some exporter implementations. I'm not sure if these are generalizable to implementation in all SDKs, but they're worth considering (if they were previously considered and rejected in this PR, please ignore this comment):

Go also allows configuration of gRPC, which includes the retry limit (amongst other things).


## Retry

[Transient errors](#transient-errors) MUST be handled with a retry strategy. This retry strategy MUST implement an exponential back-off with jitter to avoid overwhelming the destination until the network is restored or the destination has recovered.

## Transient Errors
Transient errors are errors which expect the backend to recover. The following status codes are defined as transient errors:

| HTTP Status Code | Description |
| ---------------- | ----------- |
| 408 | Request Timeout |
| 5xx | Server Errors |

| gRPC Status Code | Description |
| ---------------- | ----------- |
| 1 | Cancelled |
| 4 | Deadline Exceeded |
| 7 | Permission Denied |
| 8 | Resource Exhausted |
| 10 | Aborted |
| 10 | Out of Range |
| 14 | Unavailable |
| 15 | Data Loss |
| 16 | Unauthenticated |

Additional details on transient errors can be found in [otep-35](https://github.com/open-telemetry/oteps/blob/master/text/0035-opentelemetry-protocol.md#export-response) for gRPC and [otep-99](https://github.com/open-telemetry/oteps/blob/master/text/0099-otlp-http.md#failures) for HTTP