Fix StackOverflowException in Configuration.Binder source generator

[bart-vmware]

Fixes #103802.

No tests yet, work in progress...
Currently targets release/8.0 because main doesn't build.

[dotnet-issue-labeler]

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

[dotnet-issue-labeler]

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-extensions-configuration
See info in area-owners.md if you want to be subscribed.

[tarekgh]

@bart-vmware Thanks for helping with the fix.

Currently targets release/8.0 because main doesn't build.

What do you mean main doesn't build? If we need to proceed, we'll need to have the fix in main first. I'll wait to have you getting everything ready and then can review it.

[bart-vmware]

What do you mean main doesn't build?

On the main branch, I'm unable to build Microsoft.Extensions.Configuration.Binder.sln in VS Preview v17.10.3, running in a DevBox on Azure. I've installed the .NET 9 SDK, the latest Python for Windows, and added the C++ workload components to VS.

Based on the guidance here, this is what I tried (using PowerShell 7.4.3):

1. git clean -xdf
2. ./build.cmd -rc Release
3. ./build.cmd -projects src/libraries/Microsoft.Extensions.Configuration.Binder/Microsoft.Extensions.Configuration.Binder.sln
4. ./build.cmd -vs Microsoft.Extensions.Configuration.Binder

All the scripts succeed. But when loading the solution in VS, it makes several changes to the .sln file. Trying to build the solution from VS fails with the following error:

53>Microsoft.Extensions.Configuration.Binder.SourceGeneration.Tests -> C:\Source\Repos\runtime\artifacts\bin\Microsoft.Extensions.Configuration.Binder.SourceGeneration.Tests\Debug\net9.0\Microsoft.Extensions.Configuration.Binder.SourceGeneration.Tests.dll
52>Microsoft.Extensions.Configuration.Binder.Tests -> C:\Source\Repos\runtime\artifacts\bin\Microsoft.Extensions.Configuration.Binder.Tests\Debug\net462\Microsoft.Extensions.Configuration.Binder.Tests.dll
53>C:\Users\kbart\.nuget\packages\microsoft.net.compilers.toolset\4.11.0-3.24313.9\tasks\net472\Microsoft.CSharp.Core.targets(85,5): error :
53>C:\Users\kbart\.nuget\packages\microsoft.net.compilers.toolset\4.11.0-3.24313.9\tasks\net472\Microsoft.CSharp.Core.targets(85,5): error : Unhandled Exception: System.IO.DirectoryNotFoundException: Could not find a part of the path 'C:\Source\Repos\runtime\artifacts\obj\Microsoft.Extensions.Configuration.Binder.SourceGeneration.Tests\Debug\net462\generated\Microsoft.Extensions.Configuration.Binder.SourceGeneration\Microsoft.Extensions.Configuration.Binder.SourceGeneration.ConfigurationBindingGenerator'.
53>C:\Users\kbart\.nuget\packages\microsoft.net.compilers.toolset\4.11.0-3.24313.9\tasks\net472\Microsoft.CSharp.Core.targets(85,5): error :    at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
53>C:\Users\kbart\.nuget\packages\microsoft.net.compilers.toolset\4.11.0-3.24313.9\tasks\net472\Microsoft.CSharp.Core.targets(85,5): error :    at System.IO.Directory.InternalCreateDirectory(String fullPath, String path, Object dirSecurityObj, Boolean checkHost)
53>C:\Users\kbart\.nuget\packages\microsoft.net.compilers.toolset\4.11.0-3.24313.9\tasks\net472\Microsoft.CSharp.Core.targets(85,5): error :    at System.IO.Directory.InternalCreateDirectoryHelper(String path, Boolean checkHost)
53>C:\Users\kbart\.nuget\packages\microsoft.net.compilers.toolset\4.11.0-3.24313.9\tasks\net472\Microsoft.CSharp.Core.targets(85,5): error :    at Microsoft.CodeAnalysis.CommonCompiler.CompileAndEmit(TouchedFileLogger touchedFilesLogger, Compilation& compilation, ImmutableArray`1 analyzers, ImmutableArray`1 generators, ImmutableArray`1 additionalTextFiles, AnalyzerConfigSet analyzerConfigSet, ImmutableArray`1 sourceFileAnalyzerConfigOptions, ImmutableArray`1 embeddedTexts, DiagnosticBag diagnostics, ErrorLogger errorLogger, CancellationToken cancellationToken, CancellationTokenSource& analyzerCts, AnalyzerDriver& analyzerDriver, Nullable`1& generatorTimingInfo)
53>C:\Users\kbart\.nuget\packages\microsoft.net.compilers.toolset\4.11.0-3.24313.9\tasks\net472\Microsoft.CSharp.Core.targets(85,5): error :    at Microsoft.CodeAnalysis.CommonCompiler.RunCore(TextWriter consoleOutput, ErrorLogger errorLogger, CancellationToken cancellationToken)
53>C:\Users\kbart\.nuget\packages\microsoft.net.compilers.toolset\4.11.0-3.24313.9\tasks\net472\Microsoft.CSharp.Core.targets(85,5): error :    at Microsoft.CodeAnalysis.CommonCompiler.Run(TextWriter consoleOutput, CancellationToken cancellationToken)
53>C:\Users\kbart\.nuget\packages\microsoft.net.compilers.toolset\4.11.0-3.24313.9\tasks\net472\Microsoft.CSharp.Core.targets(85,5): error :    at Microsoft.CodeAnalysis.CSharp.CommandLine.Csc.<>c__DisplayClass1_0.<Run>b__0(TextWriter tw)
53>C:\Users\kbart\.nuget\packages\microsoft.net.compilers.toolset\4.11.0-3.24313.9\tasks\net472\Microsoft.CSharp.Core.targets(85,5): error :    at Microsoft.CodeAnalysis.CommandLine.ConsoleUtil.RunWithUtf8Output[T](Func`2 func)
53>C:\Users\kbart\.nuget\packages\microsoft.net.compilers.toolset\4.11.0-3.24313.9\tasks\net472\Microsoft.CSharp.Core.targets(85,5): error :    at Microsoft.CodeAnalysis.CSharp.CommandLine.Csc.Run(String[] args, BuildPaths buildPaths, TextWriter textWriter, IAnalyzerAssemblyLoader analyzerLoader)
53>C:\Users\kbart\.nuget\packages\microsoft.net.compilers.toolset\4.11.0-3.24313.9\tasks\net472\Microsoft.CSharp.Core.targets(85,5): error :    at Microsoft.CodeAnalysis.CommandLine.BuildClient.RunLocalCompilation(String[] arguments, BuildPaths buildPaths, TextWriter textWriter)
53>C:\Users\kbart\.nuget\packages\microsoft.net.compilers.toolset\4.11.0-3.24313.9\tasks\net472\Microsoft.CSharp.Core.targets(85,5): error :    at Microsoft.CodeAnalysis.CommandLine.BuildClient.RunCompilation(IEnumerable`1 originalArguments, BuildPaths buildPaths, TextWriter textWriter, String pipeName)
53>C:\Users\kbart\.nuget\packages\microsoft.net.compilers.toolset\4.11.0-3.24313.9\tasks\net472\Microsoft.CSharp.Core.targets(85,5): error :    at Microsoft.CodeAnalysis.CommandLine.BuildClient.Run(IEnumerable`1 arguments, RequestLanguage language, CompileFunc compileFunc, CompileOnServerFunc compileOnServerFunc, ICompilerServerLogger logger)
53>C:\Users\kbart\.nuget\packages\microsoft.net.compilers.toolset\4.11.0-3.24313.9\tasks\net472\Microsoft.CSharp.Core.targets(85,5): error :    at Microsoft.CodeAnalysis.CSharp.CommandLine.Program.MainCore(String[] args)
53>C:\Users\kbart\.nuget\packages\microsoft.net.compilers.toolset\4.11.0-3.24313.9\tasks\net472\Microsoft.CSharp.Core.targets(85,5): error :    at Microsoft.CodeAnalysis.CSharp.CommandLine.Program.Main(String[] args)
53>Done building project "Microsoft.Extensions.Configuration.Binder.SourceGeneration.Tests.csproj" -- FAILED.

The directory C:\Source\Repos\runtime\artifacts\obj\Microsoft.Extensions.Configuration.Binder.SourceGeneration.Tests\Debug\net462\generated\Microsoft.Extensions.Configuration.Binder.SourceGeneration\Microsoft.Extensions.Configuration.Binder.SourceGeneration.ConfigurationBindingGenerator exists on disk, which contains the file BindingExtensions.g.cs.

I've tried various combinations with switches like -f net462 and -f net9.0, but I can't get passed this. I've also seen VS popping up with a NullReferenceException when trying to load the test projects, but I don't remember the exact steps.

@tarekgh Can you help out?

[tarekgh]

I didn't try going through your steps, I'll try that later when I get a chance. Can you try to build and run from the command line? The easiest way is doing the steps:

• From the root build using .\build.cmd -s libs+clr -c Release
• The do pushd .\src\libraries\Microsoft.Extensions.Configuration.Binder\tests\SourceGenerationTests\
• run the tests using dotnet build /t:test /p:Configuration=Release. This should also build any changes in the source gen. Just ensure to kill dotnet processes taskkill /IM dotnet.exe /F as sometime running source gens keep dotnet processes alive for a while.

Let me know if you face any issue I can try to help with.

[bart-vmware]

I didn't try going through your steps, I'll try that later when I get a chance. Can you try to build and run from the command line?

Yes, that works. The problem is using Visual Studio. I'd like to be able to edit/debug the tests. Can you try building and running the tests from Visual Studio? Upon opening it, the following changes are automatically applied to the .sln:

And then the build fails.

[tarekgh]

Here is what you can do to use VS.

• In the command line from the root, do .\build.cmd -s libs+clr -c Release
• Ensure installing the .NET version required to run the tests. dotnet-install.ps1 -v 9.0.100-preview.5.24307.3.
• Manually open VS then choose to open a solution or project option.
• Choose the project [Your folder where you cloned the runtime repo]\runtime\src\libraries\Microsoft.Extensions.Configuration.Binder\tests\SourceGenerationTests\Microsoft.Extensions.Configuration.Binder.SourceGeneration.Tests.csproj
• Configure VS with the release build (as you compiled from the root) look at the image below. then open test explorer (using Test-> Test Explorer menu).
• In the test explorer choose which test you want to debug or run, double click on that test inside the test explorer to have VS open the source of the test case. Set a breaking point there. Right click on the test inside the test explorer then choose Debug.
• You should have VS stop on the breaking point and can debug the test as you wish.

Let me know if you need more help.

[bart-vmware]

@tarekgh Thanks for providing the detailed steps. Unfortunately, I'm still getting the same error. I've asked a coworker to try and he's observing the same problem. Telling VS to open the folder makes it load the .sln instead.

What should have been a simple effort from my side has already taken me hours, and I can't justify spending more time on it, so I'm giving up on this PR. Please feel free to take it from here. The chosen threshold of 10 is arbitrary, please adapt based on what compiler and runtime typically use.

[tarekgh]

@bart-vmware thanks for trying. I am closing the PR and will revisit when we focus on getting the fix. in.