HTTP Spans no longer emitted by AWS SDK v2 instrumentation

[jknollmeyer]

Describe the bug

Before v1.31*, calls using the AWS SDK would emit an HTTP Client Span. This span is now suppressed.

*I believe this behavior had existed since v1.28, based on my reading of this PR https://github.com/open-telemetry/opentelemetry-java-instrumentation/pull/8405/files#diff-ecc2a9e48ea1cac6e394aabfd4ece49b63cef5f08e950346e5e74cffd3c101e9

Steps to reproduce

1. Set up the auto-instrumentation on a Java application
2. Add AWS SDK v2 operations to the Java application
3. Trigger the AWS SDK v2 operation at runtime (I tested with both SQS and DDB)

Expected behavior

• AWS SDK Span is created and transmitted to the collector
• HTTP Client Span is created and transmitted to the collector

Actual behavior

• AWS SDK Span is created and transmitted to the collector
• HTTP Client Span is created, it is suppressed
  • The Java Agent does trace a span and it is visible in debug logs, but that span is suppressed
    • “DEBUG io.opentelemetry.javaagent.shaded.instrumentation.api.internal.SupportabilityMetrics - Suppressed Spans by 'io.opentelemetry.apache-httpclient-4.0'”

Javaagent or library instrumentation version

v1.31.0

Environment

JDK: Corretto-17.0.8.8.1
OS: Amazon Linux 2023

Additional context

I believe this is related to the addition of this commit in v1.31 6f0fd8e

I had been using the HTTP spans generated starting 1.28, so the suppression of the spans without any control caused a loss in visibility. As far as I know, the AWS SDK spans do not map one-to-one with requests made to the backend, so there’s a loss in visibility for retries.

The PR didn’t list any details, so it’s not clear to me why the nested HTTP Span needed to be suppressed - I would like to get some more context on that #9634

[laurit]

The presence of these spans was not intentional, the suppression behavior was inconsistent between sync and async clients. There is #8453 which describes the behavior you want. Implementing it is a bit tricky because with java agent we'd automatically get the http client spans when we stop suppression, but with library instrumentation we would not. Would need to investigate whether it is possible to also apply manual instrumentation to the underlying http client used by aws client or let the telemetry produced by the agent and library instrumentations diverge. An additional concern would be with sqs spans, how should the messaging spans and http client spans be connected?