Add BuildCheck basic telemetry

[JanKrivanek]

Context

Let's understand how our users are using BuildChecks.
Possible examples of usefull metrics:

• Is BuildCheck enabled.
• Built in rules that run as part of build.
• Total ration of built-time vs buildcheck run.
• Number of distinct custom checks enabled.
• Number of distinct custom checks failed to load.

Work to be done

Inspiration: #9063

• Add and populate BuildCheck telemetry event
• Mount it properly in SDK
• Emit initial telemetry (to be captured in gdpr inventory tool: https://gdpr.datasmart.ms/)
  • once mounted in the SDK, run a build with the DOTNET_CLI_TELEMETRY_OPTOUT=0 env var set to actually send the telemetry so the GDPR tool will flag it.
• Approve telemetry in GDPR inventory tool
• Update SDK telemetry docs in dotnet/docs

FYI @baronfel

[baronfel]

Data that I want to be able to check (separately from your questions above).

• what % of builds are using buildchecks? We should be able to chart usage over time.
• What checks are being suppressed (turned down from their default warnings)? This is a signal that they are too painful/chatty.
• What checks are being elevated (turned up from their default warnings)? This is a signal they are valuable and should be considered for a severity increase in the next release.
• What checks are being run? General usage tracking to ensure that created checks are bringing value.
• What checks are actually flagging issues? General view of how correct a check is.

[JanKrivanek]

• What checks are actually flagging issues? General view of how correct a check is.

What is sample telemetry datapoint for this? List of build-in checks that emitted diagnostics?
We are not trying to measure correctness (false positives/negatives) here, are we?

[baronfel]

This is the question I am least sure of how to answer :D If we track BuildChecks that do flag errors, we'd expect that number to trend down to zero over time as users either suppress or fix all reported errors for the BuildCheck. Maybe that's useful enough of a metric?

[JanKrivanek]

@baronfel - can you have a quick look if those: https://github.com/dotnet/msbuild/pull/10652/files#diff-b2558de61139ff21a10b8149c39fd972bce0477b76f4deb6ed238d5e1e0abe07R14-R93 seem like might meet the needs?

I'm still working on testing and polishing the code - so PR is not yet fully ready, but it already has the proposal of data.
Main questions:

• Will change of existing telemetry event (adding BuildCheckEnabled to loggingConfiguration) force a need for re-classification (I suppose so). Is there any other concern?
• Are event names with slashes supported? The final eventname is with slashes - e.g. 'dotnet/cli/msbuild/loggingconfiguration'. But I'm not sure if specifying the last part with slashes matters.
• Do the proposed data look as meeting your requirements?

[baronfel]

• Responded to this point in the PR, but I think there's a better event to put BuildCheckEnabled onto ("msbuild/build")
• event names with slashes are indeed supported, so feel free to use whatever namespaces you need
• yes! the data I see so far should be able to answer all of the questions above

[JanKrivanek]

So I run couple samples with DOTNET_CLI_TELEMETRY_OPTOUT=0 through the local sdk with local patched msbuild (as well debugged through the sdk to confirm the events are coming in and data are populated). Yet I do not see the data yet in the classification tool:

https://gdpr.datasmart.ms/?q=ProductCode%20=%20%27ai.dotnetcli%27%20and%20DataClassification%20%3D%20%27Unclassified%27%20and%20EntityName%20like%20%27%25msbuild%25%27

Probably couple hours delay can be expected here?