Clarify OpenTelemetry collector setup in our docs

[ppf2]

We are working with some users who are trying to set up OpenTelemetry collector to work with APM Server. They are familiar with our APM agents but not OpenTelemetry:
https://www.elastic.co/guide/en/apm/get-started/current/open-telemetry-elastic.html

It will be helpful to clarify the following directly in our docs (for users who do not know anything about OpenTelemetry):

• The collector is different from Elastic APM agents because it doesn't run within the app and needs to run in its own docker instance (for Kubernetes, this means running it as a daemonset in a pod)
• https://www.elastic.co/guide/en/apm/get-started/current/open-telemetry-elastic.html#open-telemetry-elastic-configure talks about setting up the yaml configuration file for the collector but does not indicate where users will reference this yaml file. Certainly, we are providing links to getting started docs (https://opentelemetry.io/docs/collector/getting-started/) on our documentation page. It can be helpful to provide some inline examples in our own docs, or provide a description of how this yaml is configured/read in by the collector before pointing to the OpenTelemetry docs for actual examples.

[graphaelli]

This page and guide are getting overhauled following the addition of native OpenTelemetry intake support. @bmorelli25 / @jalvz does this duplicate any open issues or can you link prs to this issue?

[bmorelli25]

Gil, we do have this issue for documenting native support for OTLP/gRPC: elastic/apm-server#4678

Pius, thanks for opening this issue! I really appreciate the level of detail.

(for users who do not know anything about OpenTelemetry)

To provide some background, when we initially doc'd the OptenTelemetry integration (way back in 7.7), we decided to focus only on users who were already using OpenTelemetry. We didn't want to make architecture recommendations or dive too deep into what OTel is and how it works. That's why there's a lot of linking off to the OTel docs to learn the basics, and a note that says: "This guide assumes you’ve already instrumented your services with the OpenTelemetry API and/or SDK..."

We were (and still are) worried about the maintenance cost of providing too much documentation (duplicating docs?) for products we can't control. But if getting started with OTel from scratch is a common ask from our users, then maybe we should reconsider the initial documentation decision. This has come up at least once before on Slack.

In the meantime, yes, your improvements seem easy enough to make (and low-cost to maintain) 😄

[tbragin]

cc @cyrille-leclerc

[bmorelli25]

Closing this as the OpenTelemetry Collector exporter for Elastic was deprecated in 7.13. See also elastic/observability-docs#467.

@ppf2 Please feel free to reopen this if you feel like anything is still missing.