AI SDK does not collect HttpClient calls when response does not have content

[zakimaksyutov]

When an HTTP response contains only headers and not content (content-size = 0) then AI SDK (2.5-beta1) does not collect such calls as dependencies.

Repro

The following snippet for failure case should track 1) request (failed); 2) dependency (failed); 3) exception:

        public async Task<string> GetStringFromBackend(
            [FromUri] int numRetries,
            [FromUri] int dataStoreDelayInMS,
            [FromUri] double dataStoreFailureRatio)
        {
            while (true)
            {
                try
                {
                    string result = $"Hello from Backend: {DateTime.Now};";
                    using (var client = new HttpClient())
                    {
                        var response =
                            await
                                client.GetStringAsync(
                                        $"http://localhost.fiddler:8085/api/datastore/GetStringFromDataStore?delayInMS={dataStoreDelayInMS}&failureRatio={dataStoreFailureRatio}")
                                    .ConfigureAwait(false);
                        result += response;
                    }

                    return result;
                }
                catch (Exception ex)
                {
                    this.telemetry.TrackException(ex);

                    if (numRetries > 0)
                    {
                        --numRetries;
                        continue;
                    } 

                    throw new HttpResponseException(HttpStatusCode.InternalServerError);
                }
            }
        }

But with below response it tracks only request and exception but not dependency:

HTTP/1.1 500 Internal Server Error
Cache-Control: no-cache
Pragma: no-cache
Expires: -1
Server: Microsoft-IIS/8.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Sat, 11 Nov 2017 00:45:27 GMT
Content-Length: 0

After modification the response and including content AI SDK successfully tracks all three documents:

HTTP/1.1 500 Internal Server Error
Cache-Control: no-cache
Pragma: no-cache
Content-Length: 50
Content-Type: text/plain; charset=utf-8
Expires: -1
Server: Microsoft-IIS/8.5
X-AspNet-Version: 4.0.30319
Request-Context: appId=cid-v1:d79f37bd-6fea-487e-af26-f67978f70410
X-Powered-By: ASP.NET
Date: Sat, 11 Nov 2017 06:36:47 GMT

Reference ID: 346bacae-df64-4162-b55d-76711d062ae3

Workaround

Add some content to response.

[zakimaksyutov]

@cijothomas @Dmitry-Matveev looks like all functional tests get some content in response. We might want to consider adding tests covering this case (once it is fixed).

@lmolkova adding you to take a look once you have time

[lmolkova]

Note:

1. Affects .NET Framework (4.5, 4.6, 4.7) apps only - .NEt Core is not affected
2. Azure Web Apps and other apps with runtime instrumentation enabled are not affected either

[lmolkova]

1. HTTP desktop hook is called with response == null (https://github.com/dotnet/corefx/blob/master/src/System.Diagnostics.DiagnosticSource/src/System/Diagnostics/HttpHandlerDiagnosticListener.cs#L538), i.e. no event is sent to AI SDK.

2. It's called from HttpWebResponse ctor (no response is passed to CallDone):

            //if the returned contentlength is zero, preemptively invoke calldone on the stream.
            //this will wake up any pending reads.
            if (m_ContentLength == 0 && m_ConnectStream is ConnectStream) {
                ((ConnectStream)m_ConnectStream).CallDone();
            }

http://referencesource.microsoft.com/#System/net/System/Net/HttpWebResponse.cs,525

in case there is a response body, CallDone is called with response object.

I.e. by changing http desktop, we can react and send Stop event, however response object is not available.

[DeepakIoT]

How can we stop Request-Context: appId=cid-v1:91e46abb-4ce5-4d98-9375-02378f649011 from response header?

[DeepakIoT]

What is that Request-Context:appId=cid-v1 and why that is coming in Response Header?

Somehow in my request response header "Request-Context" coming and i tried to remove that using in web.config and Response.Headers.Remove("Request-Context"); in global.asax but that header is not getting removed.

In the value of that header I am getting some Appid and I am not sure from where that is coming.

Here is response header of my request.

Cache-Control:no-cache, no-store
Content-Encoding:gzip
Content-Length:140
Content-Type:application/json; charset=utf-8
Date:Tue, 20 Feb 2018 09:48:28 GMT
Pragma:no-cache
Request-Context:appId=cid-v1:b650ed48-297a-4ea2-af46-0a5a5d26a82b
Vary:Accept-Encoding

Any help appreciated. Thanks in Advance.

[lmolkova]

@DeepakIoT
Request-Context is used for cross-component correlation when 2 of your applications use different instrumentation keys.
In this case, knowing caller or callee appId (passed in the header) allows to build application map and trace correlated telemetry across instrumentation keys
Refer to this article for more info.

The header is added automatically by AppInsights SDK. Why would you want to remove it?

[DeepakIoT]

Thanks for your reply @lmolkova

I want to remove the header because we are using some IOT devices which use payload and these devices have limited memory.

Becuase of that payload size increased and iot device memory exhausted so I want to remove that unnecessary header which is not needed by us.

If you guide me to remove this it will be very helpful.

[lmolkova]

@DeepakIoT
Sorry to hear the header breaks your scenarios.
You may set RequestTrackingTelemetryModule.SetComponentCorrelationHttpHeaders to false to prevent header to be added to the response.
You can do it in applicationInsights xml file, just find RequestTrackingTelemetryModule element and add <SetComponentCorrelationHttpHeaders>false</SetComponentCorrelationHttpHeaders> under it.

Let me know if it helps.

[DeepakIoT]

@lmolkova

I tried to add in applicationInsights XML file but that still not getting removed. Even I tried to disable Application Insight on our App also delete that applicationInsights XML file but that was still coming in the header. After these changes, we restarted app service as well.

Do I miss something?

I am attaching my applicationInsights XML file in which I made changes, I am still looking for your help :)

[lmolkova]

@DeepakIoT

I tried to add in applicationInsights XML file but that still not getting removed. Even I tried to disable Application Insight on our App also delete that applicationInsights XML file but that was still coming in the header. After these changes, we restarted app service as well.

this sounds strange.
I cannot repro it locally, so if I do exactly the same changes in ApplicationInsights.config as you've done, AI SDK no longer adds the Request-Context header into the response.
Could you verify that it's a relevant config that you are changing and it's used at runtime?
Would you be able to provide the repro app?

It does not prevent the header to be set in incoming requests - could it be the problem in your case?

[lmolkova]

@DeepakIoT
By the way, which version if AI SDK are you using?

[DeepakIoT]

@lmolkova

Now it's working and this header remove now from adding false as mentioned by previous thread by you.

The problem was my end was that ApplicationInsights.config was residing at two places one in BIN and one to root directory so I was changing on BIN ones.

Today I changed in root folder one and it's removed now.

Thanks for all your help...!!!

[juho-hanhimaki]

@lmolkova can you advice how to remove the Request-Context header in ASP.NET Core?

I believe the RequestTrackingTelemetryModule doesn't exist in the ASP.NET Core SDK. At least I haven't found it.

Sorry for hijacking the issue.

[lmolkova]

@juho-hanhimaki, unfortunately, there is no such configuration option in AspNet Core SDK. Could you please create an issue in the https://github.com/Microsoft/ApplicationInsights-aspnetcore repo?

If your scenarios are broken because of this header, please describe it in the issue.
Thanks!

[juho-hanhimaki]

@lmolkova thanks for the info.

Opened an issue at: microsoft/ApplicationInsights-aspnetcore#613

[cijothomas]

Its possible (but not so straightforward) in asp.net core sdk to configure dependencymodule to not inject headers. Will use the issue opened in aspnet core for any comments.
https://github.com/Microsoft/ApplicationInsights-aspnetcore/wiki/Dependency-Tracking-and-Performance-Counter-Collection#configuring-telemetry-module-services

[lmolkova]

@cijothomas, thanks! The problem is it's not possible to prevent header injection into the response of incoming request. I.e. there is no configuration for request tracking layer in asp.net core sdk.