[Feature]: Add OpenTelemetry distributed tracing

[ronensc]

🚀 The feature, motivation and pitch

This proposal suggests adding distributed tracing with OepnTelemetry, which will enable operators to export traces in a standard protocol and seamlessly connect them with visualization tools such as Jaeger, Zipkin and Instana

As an initial implementation, I suggest emitting a trace for each request, including the following data:

• Model name
• Request ID
• Sampling params
• Latency
• Number of input tokens
• Number of generated tokens

This approach will greatly enhance the observability and troubleshooting capabilities of vLLM.
I am willing to work on this and contribute it to the community.
I welcome any feedback or suggestions.

Alternatives

No response

Additional context

No response

[gyliu513]

Hi @ronensc , if you want to contribute, maybe here is a good repo https://github.com/traceloop/openllmetry/tree/main/packages , we already have bunch of LLM providers support here.

FYI @nirga

[nirga]

@ronensc would love to assist! Ping me on the community slack

[baggiponte]

@samuelcolvin for logfire!

[MrMegaMango]

Hi I am using Opentelemetry traces to crack down slowness in vllm call. Only this llm_request one span comes out with some attributes.

It only covers 10% of the time spent after a request to vllm's v1/completion endpoint. No nested span calls.
Presumably the start of vllm_request span comes from seq_group.metrics.arrival_time, why it is seconds later than request to v1/compelete?