Implement a mechanism to configure exporters with autoinstrumentation command

[mauriciovasquezbernal]

The goal of the opentelemetry-auto-instrumentation command is to allow the user to instrument an application without having to write code. That command is able to instrument the different supported libraries by invoking the instrumentor entrypoint on them, but the user still has to modify the program code to setup the span processor/exporters.

A mechanism that allows the user to configure opentelemetry (processors/exporters, propagation formats, specific library settings) without touching the application code is needed.

It's needed to understand how this could couple with the current configuration mechanism (opentelemetry-api/src/opentelemetry/configuration) and the format to be used #564, also how flexible it should be, for instance, will the user be able to pass advance features like #656?

Another consideration is if this command is only intended to be used with the official SDK or it should be generic.

[majorgreys]

I think that work could be done in an "instrumentor" for a vendor exporter. E.g., the _instrument method would add the vendor exporter to a span processor as well as set up the vendor propagator. This would help in the case of what I've been doing for the Datadog exporter.

Though this would broaden term "instrumentor" beyond enabling tracing for third-party library.

[owais]

As a first step, I think auto-instrumentation should automatically enable the OTLP exporter by default. All exporters should be configurable with environment variables so the CLI interface doesn't get too polluted. It could look something like:

export ZIPKIN_EXPORTER_HOST=localhost
export ZIPKIN_EXPORTER_PORT=9412
export OPENTELEMETRY_EXPORTER=opentelemetry.ext.zipkin.ZipkinSpanExporter
opentelemetry-instrument flask run 

or

export ZIPKIN_EXPORTER_HOST=localhost
export ZIPKIN_EXPORTER_PORT=9412
opentelemetry-instrument --exporter=opentelemetry.ext.zipkin.ZipkinSpanExporter flask run 

[owais]

In any case, I think as a first step, we should enable OTLP by default when running opentelemetry-instrument. One open question with this is how do we detect that the user has enabled a different exporter in code and then disable the default exporter. We can detect the existence of another exporter but how do we detect the intent? The simple answer is to always disable auto-enabled exporter if any other exporters are configured by the user but would like to hear more thoughts on this.

[owais]

Can someone assign this to me please?

[codeboten]

Done!

[codeboten]

@owais If the OTLP exporter is configured by default, would you expect the opentelemetry-ext-otlp package to become a dependency of the opentelemetry-instrumentation package? Or would it still be installed separately and report an error if it wasn't present? An alternative could be to move the otlp exporter into the SDK, but then that makes grpc a requirement of the SDK.

I wonder if there's a way we could use entrypoints to make the configuration easier for our users here.

[owais]

As an end user, I'd expect it to work out of box when running opentelemetry-instrument <app> command. I think perhaps the bootstrap script can help? May be we can add an --exporter flag to bootstrap that installs the specified exporter but defaults to otlp. Then the instrument command can have some mechanism to detect the installed and preferred exporter. Thoughts?

[lzchen]

I'd prefer to not have any default behavior regarding things that are supposed to be configurable (exporters). What about a configuration file that can be passed into the opentelemetry-instrument? Using env variables doesn't seem as scalable, especially with the ever growing list of configurable "things" that users can do.

[codeboten]

I like the idea of a configuration file , makes it easy to check in and re-use as well. Would be nice if config validation take care of reporting any things that are configured but unavailable as well.

Re. environment variables it's worth noting this issue: open-telemetry/opentelemetry-specification#572

[codeboten]

This doc was put together by @mauriciovasquezbernal around what a configuration file could look like: https://docs.google.com/document/d/10GW_yfi0yHozZIpUdi3t7Vg3XLftID1brnnzPwWK-Is/edit

[owais]

I think there are two issues to discuss here.

1. Default exporter (and other components) when using auto-instrumentation.
2. Way to override the defaults from outside the code with configuration.

I don't think we should mix the two. IMO OpenTelemetry Python library should definitely default to the OpenTelemetry exporter, at least when using the auto-instrumentation mechanism to start a service. As an end user, I think having sane (and expected) defaults in invaluable. We should reduce as much friction as we can when people are trying to get started.

I think an experience like the following is worth having:

pip install opentelemtry-instrumentation
opentelemtry-instrumentation -a=install
opentelemetry-bootstrap python main.py

This should be all that is required to instrument a Python service and start exporting telemetry to OpenTelemetry collector. In reality users would need to install SDK/API/Exporter Python packages as well but that is a different problem and can be solved either by having a meta-package like opentelemetry-all/opentelemetry-defaults or opentelemetry-bootstrap command can be extended to also install any SDK, Exporter or other component packages.

To clarify, I'm not suggesting opentelemtry-python to always default to OTLP exporter. Only suggesting that opentelemetry-instrumentation CLI command has sane defaults and doesn't force users to go and configure things (but certainly allows them to).