Provide managed APIs for our canonical set of runtime performance counters

[tommcdon]

AB#1117223
Provide the data needed to build managed performance counters

[shirhatti]

@pakrym To help provide list of counters that ASP.NET needs

[shirhatti]

@jorive @vancem to help provide list of counters

[vancem]

Currently these are the existing live metrics possible on Azure Portal

        Http 5XX, DataIn, DataOut, #Requests, ResponseTime.
        Ave Working set, CPU, Private Bytes
        Connections, Request in App Queue, Current Assemblies, Gen 0, 1, 2, Handle Count, Http Errors (various)
        Windows I/O Read, Write, Other, Thread Count, # AppDomain

They are a reasonable set.

Web site for ASP.NET perf counters https://stackify.com/asp-net-performance-counters/

Key counters mentioned

1. Managed Memory Used
2. Web Traffic.
3. Exceptions, % Time in GC, #Gen0, #Gen1, #Gen2,
4. CPU Time, Process Memory Used,

Desktop runtime counters https://docs.microsoft.com/en-us/dotnet/framework/debug-trace-profile/performance-counters

[shirhatti]

Final list of EventCounters as per our design meeting today

System Counters

• Working set memory
• CPU
• Handles

Exception performance counters

• Exceptions thrown

Thread performance counters

• Threads
• TPL threads
• Total queue length
• work items processed

Lock performance counters

• Lock contentions

Memory performance counters

• Bytes in all heaps
• Gen 0/1/2 collections
• % time in GC
• Gen 0/1/2 and LOH size
• Allocation rate

Networking performance counters

• Bytes received/sent
• Connections
• Requests queued/aborted/failed/succeeded

JIT Counters

• IL Bytes Jitted

Loader

• Current Assemblies loaded

In addition to this, we need counters in

• ASP.NET
• ADO.NET
• EF

[EDIT]: Some customer feedback from the XAP team, added the IL Bytes Jitted and Current Assemblies counters.

[tommcdon]

@noahfalk What are your thoughts on whether we should try to make the underlying API's that we use for perf counters public? Some are currently internal and others are public - for example CPU is internal (pre-existing), but workingset is public (pre-existing).

[noahfalk]

In general I think we should aim to have some API public. In the case of CPU it is internal because there is another API in System.Diagnostics.Process.dll in CoreFX that exposes the data to customers. We couldn't use it because of layering but customers won't have that constraint.

[noahfalk]

We have issue #20372 tracking a specific of runtime counters to be added for 3.0 whereas the scope of this issue is slightly broader including ASP.NET, ADO.NET and EF. I think we could afford to have the up-stack counters not appear during 3.0. Moving this to Future.
cc @tommcdon @sywhang

[Dotnet-GitSync-Bot]

I couldn't figure out the best area label to add to this issue. Please help me learn by adding exactly one area label.

[ghost]

Tagging subscribers to this area: @tommcdon
Notify danmosemsft if you want to be subscribed.

[tommcdon]

Depends on #34648, #31951, #1099, #1098

[tommcdon]

@sywhang do you have a PR for this issue?

[sywhang]

There are issues I filed for API reviews. Some of them were closed because there are ways to compute them using the managed APIs:

#36324 (JIT metrics): this one has some discussions going on regarding what metrics to expose. I think it might be better to hold off on this for 6.0 so that we can expose all the right metrics instead of adding just a couple of ones that we are currently using so that we have a more complete story, unless you think we should get it into 5.0?

#36068: there is a way to get this using managed API already.

#36071: same as above

#36069: same as above.

[tommcdon]

Thank you! I will move this to 6.0 based off of the current status of the JIT metrics issue (#36324)

[josalem]

closed via #55046