CanIgnoreTemplateGroupsWithConstraints flaky test failing in CI

[ViktorHofer]

Edit by carlossanlop:

• I marked it as "BuildRetry" as it is failing intermittently.
• I don't know who the rightful owner is, if Area-CLI or Area-Templates, so I added both areas to get them notified.

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=955490
Build error leg or test failing: Microsoft.TemplateEngine.Cli.UnitTests.ParserTests.TabCompletionTests.CanIgnoreTemplateGroupsWithConstraints

Error Message

{
  "ErrorMessage" : ["Collections differ", "foo1"],
  "BuildRetry" : true,
  "ExcludeConsoleLog" : false
}

• PR: [ApiCompat] Warn if abstract keyword is removed from member #46797
• Job: TemplateEngine: linux (x64)
• Job result: https://dev.azure.com/dnceng-public/public/_build/results?buildId=955490&view=logs&j=abeb465d-bc0e-5635-8fbf-b3cd69aee9e2&t=5ede4cee-c78c-5b80-87c2-679aafff6b32
• Build output:

XUnit : error : Tests failed: /mnt/vss/_work/1/s/artifacts/TestResults/Release/Microsoft.TemplateEngine.Cli.UnitTests_net10.0_x64.html [net10.0|x64] [/mnt/vss/_work/1/s/test/Microsoft.TemplateEngine.Cli.UnitTests/Microsoft.TemplateEngine.Cli.UnitTests.csproj]
##[error]XUnit(0,0): error : (NETCORE_ENGINEERING_TELEMETRY=Test) Tests failed: /mnt/vss/_work/1/s/artifacts/TestResults/Release/Microsoft.TemplateEngine.Cli.UnitTests_net10.0_x64.html [net10.0|x64]

• Test output:

Error message
Assert.Equal() Failure: Collections differ\n           ↓ (pos 0)\nExpected: ["foo1"]\nActual:   []

Stack trace
   at Microsoft.TemplateEngine.Cli.UnitTests.ParserTests.TabCompletionTests.CanIgnoreTemplateGroupsWithConstraints() in /mnt/vss/_work/1/s/test/Microsoft.TemplateEngine.Cli.UnitTests/ParserTests/TabCompletionTests.cs:line 470
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=955490
Error message validated: [Collections differ foo1]
Result validation: ❌ Known issue did not match with the provided build.
Validation performed at: 2/18/2025 8:20:43 PM UTC

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

[carlossanlop]

I'm also seeing this in my SDK PRs. Unsure if KnownBuildError is already working in the repo, but I can add the json info to the description in case it does.