Recommend pattern to redact query string parameters

[ericksoen]

Question

We're evaluating a migration from New Relic APM agents to OpenTelemetry. This APM agent provides fine-grained control over what properties are included or excluded to our telemetry. This includes properties like query string parameters, ports, etc.

I see that the dotnet Http instrumentation library provides filters to remove requests entirely and also an Enrich option to add additional properties.

I've been able to hijack this Enrich property to override the value for the http.url to remove query string params—this seems to accomplish the desired effect. I'm curious if there are any recommended patterns for how to accomplish this in the instrumentation library itself or if it is better handled via attribute processors in an OTEL config.

.AddHttpClientInstrumentation((options) => options.Enrich = (activity, eventName, rawObject) =>
{
    if (eventName.Equals("OnStartActivity"))
    {
        if (rawObject is HttpRequestMessage request)
        {
            var redactedUrl = request.RequestUri.OriginalString.Split("?")[0];
            activity.SetTag("http.url", redactedUrl);
        }
    }
})

Describe any aspect of your environment relevant to the question.

• Dotnet WebCore31 application
• OpenTelemetry.Instrumentation.Http version 1.0.0-rc8

What are you trying to achieve?

Remove query string parameters that are automatically added via HTTP client instrumentation (preferably with minimal application code on my part 😁)

Additional Context

N/A

[cijothomas]

open-telemetry/opentelemetry-dotnet-contrib#1747
open-telemetry/opentelemetry-dotnet-contrib#1794

Sharing some related issues logged earlier.

[stevejgordon]

@martinjt, this is another issue I think we can close. Things have moved on since this was opened.

For inbound HTTP requests, query string parameters are now redacted by default when added to the url.query attribute:

Activity.Tags:
    server.address: localhost
    server.port: 7036
    url.query: ?test1=Redacted&test2=Redacted
    http.request.method: GET
    url.scheme: https
    url.path: /weatherforecast
    network.protocol.version: 2
    user_agent.original: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
    http.route: /weatherforecast
    http.response.status_code: 200

For outbound HTTP via HttpClient, they are redacted in url.full:

Activity.Tags:
    http.request.method: GET
    server.address: www.google.com
    server.port: 443
    url.full: https://www.google.com/?query1=Redacted&query2=Redacted
    network.protocol.version: 1.1
    http.response.status_code: 200

[swythan]

That's a very blunt instrument, though. If you want to include everything except one query parameter (e.g. because chromium refuses to allow browser JS to set auth headers on websocket connections) then you have to "hijack" The enrichment mechanism as in the OP.

[abatishchev]

@stevejgordon

For inbound HTTP requests, query string parameters are now redacted by default

Is there a way to opt out?

[swythan]

@stevejgordon

For inbound HTTP requests, query string parameters are now redacted by default

Is there a way to opt out?

OTEL_DOTNET_EXPERIMENTAL_ASPNETCORE_DISABLE_URL_QUERY_REDACTION

See:

https://github.com/open-telemetry/opentelemetry-dotnet-instrumentation/blob/main/docs/config.md#instrumentation-options

#5532

[abatishchev]

thanks, @swythan! I asked and then found the corresponding environment variables myself.

I asked a question about them here: open-telemetry/semantic-conventions#860 (comment)