Microsoft.XmlSerializer.Generator uses highest installed .NET Runtime, generates C# code incompatible with the SDK of the referencing project

[KalleOlaviNiemitalo]

Description

If .NET 7 Runtime has been installed, then the MSBuild integration in Microsoft.XmlSerializer.Generator generates code that uses ref struct types like this:

System.Span<bool> paramsRead = stackalloc bool[0];

It then runs the Csc task to compile the generated code, but it does not specify the C# language version.
If the project is being built in Visual Studio 2017, then the C# language version defaults to 7.0, which does not allow ref struct types, and the serializer assembly fails to build.
This happens even if the project that references Microsoft.XmlSerializer.Generator sets <LangVersion>7.2</LangVersion> for itself.

Reproduction Steps

XmlSer.csproj:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net48</TargetFramework>
    <LangVersion>7.2</LangVersion>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.XmlSerializer.Generator" Version="6.0.0">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>build; analyzers</IncludeAssets>
    </PackageReference>
    <PackageReference Include="System.Memory" Version="4.5.5" />
  </ItemGroup>

  <ItemGroup>
    <!--
      https://docs.microsoft.com/dotnet/core/additional-tools/xml-serializer-generator#add-another-itemgroup-section-for-net-cli-tool-support
    -->
    <DotNetCliToolReference Include="Microsoft.XmlSerializer.Generator" Version="6.0.0" />
  </ItemGroup>

</Project>

Root.cs:

using System.ComponentModel;
using System.Xml.Serialization;

namespace XmlSer
{
    [XmlRoot("Root", IsNullable = false)]
    public class Root
    {
        [XmlAttribute("id")]
        [DefaultValue(-1)]
        public int Id { get; set; }
    }
}

Build in Visual Studio 2017; either in the IDE, or with MSBuild.exe.

Expected behavior

The XML serializer assembly should be built without warnings.

Actual behavior

GenerateSerializationAssembly:
  Deleting file "obj\Debug\net48\XmlSer.XmlSerializers.cs".
  Running Serialization Tool
  dotnet Microsoft.XmlSerializer.Generator "obj\Debug\net48\XmlSer.dll" --force --quiet obj\Debug\net48\sgen.rsp
  .NET Xml Serialization Generation Utility, Version 6.0.0]
  Serialization Code File Name: C:\Projects\XmlSer\obj\Debug\net48\XmlSer.XmlSerializers.cs.
  Generated serialization code for assembly C:\Projects\XmlSer\obj\Debug\net48\XmlSer.dll --> 'C:\Projects\XmlSer\obj\Debug\net48\XmlSer.XmlSerializers.cs'.
  C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\15.0\bin\Roslyn\csc.exe /nowarn:1701,1702,219,162 /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFra
  mework\v4.8\mscorlib.dll" /reference:C:\Users\[REDACTED]\.nuget\packages\system.buffers\4.5.1\ref\net45\System.Buffers.dll /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramewor
  k\v4.8\System.Core.dll" /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.8\System.Data.dll" /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Fram
  ework\.NETFramework\v4.8\System.dll" /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.8\System.Drawing.dll" /reference:"C:\Program Files (x86)\Reference Assemblie
  s\Microsoft\Framework\.NETFramework\v4.8\System.IO.Compression.FileSystem.dll" /reference:C:\Users\[REDACTED]\.nuget\packages\system.memory\4.5.5\lib\net461\System.Memory.dll /reference:"C:\Program Files (x86)\R
  eference Assemblies\Microsoft\Framework\.NETFramework\v4.8\System.Numerics.dll" /reference:C:\Users\[REDACTED]\.nuget\packages\system.numerics.vectors\4.5.0\ref\net46\System.Numerics.Vectors.dll /reference:C:\Us
  ers\[REDACTED]\.nuget\packages\system.runtime.compilerservices.unsafe\4.5.3\ref\net461\System.Runtime.CompilerServices.Unsafe.dll /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETF
  ramework\v4.8\System.Runtime.Serialization.dll" /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.8\System.Xml.dll" /reference:"C:\Program Files (x86)\Reference As
  semblies\Microsoft\Framework\.NETFramework\v4.8\System.Xml.Linq.dll" /reference:obj\Debug\net48\XmlSer.dll /debug+ /out:obj\Debug\net48\XmlSer.XmlSerializers.dll /target:library obj\Debug\net48\XmlSer.XmlSe
  rializers.cs obj\Debug\net48\SgenAssemblyInfo.cs
obj\Debug\net48\XmlSer.XmlSerializers.cs(78,44): warning CS8107: Feature 'ref structs' is not available in C# 7.0. Please use language version 7.2 or greater. [C:\Projects\XmlSer\XmlSer.csproj]
  The previous error was converted to a warning because the task was called with ContinueOnError=true.
  Build continuing because "ContinueOnError" on the task "Csc" is set to "true".
C:\Users\[REDACTED]\.nuget\packages\microsoft.xmlserializer.generator\6.0.0\build\Microsoft.XmlSerializer.Generator.targets(55,5): warning : SGEN: Failed to generate the serializer for XmlSer.dll. Please follow th
e instructions at https://go.microsoft.com/fwlink/?linkid=858594 and try again. [C:\Projects\XmlSer\XmlSer.csproj]
  Deleting file "obj\Debug\net48\sgen.rsp".
Done Building Project "C:\Projects\XmlSer\XmlSer.csproj" (default targets).

Regression?

Yes! Uninstalling .NET 7 Runtime and keeping .NET 6 SDK makes the build work again.
(The bug is reproducible even without .NET 7 SDK, if .NET 7 Runtime is installed.)

Known Workarounds

Add to the project file:

<ItemGroup>
  <CscRspFile Include="-langversion:7.2" />
</ItemGroup>

But this seems risky because the CscRspFile name might be also used in packages other than Microsoft.XmlSerializer.Generator.

Configuration

Microsoft.XmlSerializer.Generator 6.0.0.
The dotnet Microsoft.XmlSerializer.Generator command is run using .NET SDK 7.0.400.
Visual Studio 2017 version 15.9.56, in which Csc is Microsoft (R) Visual C# Compiler version 2.10.0.0 (b9fb1610).
Windows 10 version 22H2 on amd64.

dotnet --info

C:\Projects>dotnet --info
.NET SDK:
 Version:   7.0.400
 Commit:    73bf45718d

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19045
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\7.0.400\

Host:
  Version:      7.0.10
  Architecture: x64
  Commit:       a6dbb800a4

.NET SDKs installed:
  2.1.526 [C:\Program Files\dotnet\sdk]
  6.0.413 [C:\Program Files\dotnet\sdk]
  7.0.110 [C:\Program Files\dotnet\sdk]
  7.0.400 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.21 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.10 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.21 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 6.0.2 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.21 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.10 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
  None

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

Other information

The specific incompatibility due to ref struct types (introduced in #66914) could be addressed by making Microsoft.XmlSerializer.Generator set the C# language version for the Csc task invocations, here:

runtime/src/libraries/Microsoft.XmlSerializer.Generator/src/build/Microsoft.XmlSerializer.Generator.targets

Lines 53 to 54 in 3bda6e0

<Csc Condition="Exists('$(_SerializerCsIntermediateFolder)') and !Exists('$(_CscRspFilePath)')" ContinueOnError="true" OutputAssembly="$(_SerializerDllIntermediateFolder)" References="@(ReferencePath);@(IntermediateAssembly)" Optimize="$(Optimize)" EmitDebugInformation="$(DebugSymbols)" Sources="$(_SerializerCsIntermediateFolder);$(_SerializerCsAssemblyInfoIntermediateFolder)" TargetType="Library" ToolExe="$(CscToolExe)" ToolPath="$(CscToolPath)" DisabledWarnings="$(_SerializationAssemblyDisabledWarnings)"/>

<Csc Condition="Exists('$(_SerializerCsIntermediateFolder)') and Exists('$(_CscRspFilePath)')" ContinueOnError="true" OutputAssembly="$(_SerializerDllIntermediateFolder)" References="@(ReferencePath);@(IntermediateAssembly)" Optimize="$(Optimize)" EmitDebugInformation="$(DebugSymbols)" Sources="$(_SerializerCsIntermediateFolder);$(_SerializerCsAssemblyInfoIntermediateFolder)" TargetType="Library" ResponseFiles="$(_CscRspFilePath)" ToolExe="$(CscToolExe)" ToolPath="$(CscToolPath)" DisabledWarnings="$(_SerializationAssemblyDisabledWarnings)"/>

Based on Csc Task documentation, LangVersion="7.2" should do the job. I think this should not depend on the $(LangVersion) of the referencing project, because the generated C# code doesn't depend on that either.

However, there may be further incompatibilities coming up, if even higher versions of .NET Runtime are installed.

[KalleOlaviNiemitalo]

I get the same error if I add a global.json file that specifies .NET SDK 6.0.413. However, even with that global.json, the "lib\netstandard2.0\dotnet-Microsoft.XmlSerializer.Generator.dll" tool in the package appears to be run by the .NET 7 runtime rather than .NET 6, so it uses the C# code generator from .NET 7. This issue then seems related to #66914, which shipped in .NET 7 and not in .NET 6.

More generally, if new versions of System.Private.Xml start generating code that requires a new version of C# or a new library, then there should be a way to keep old versions of Microsoft.XmlSerializer.Generator using old versions of System.Private.Xml, so that just installing a new version of .NET SDK side by side won't break people's builds.

[KalleOlaviNiemitalo]

Too bad you cannot simply change this dotnet Microsoft.XmlSerializer.Generator invocation here

runtime/src/libraries/Microsoft.XmlSerializer.Generator/src/build/Microsoft.XmlSerializer.Generator.targets

Line 51 in 4822e3c

<Exec Command="dotnet Microsoft.XmlSerializer.Generator &quot;$(IntermediateOutputPath)$(AssemblyName)$(TargetExt)&quot; --force --quiet $(_SgenRspFilePath)" ContinueOnError="true"/>

to run dotnet --fx-version 6.0 Microsoft.XmlSerializer.Generator instead, ensuring that it uses a runtime that generates C# code for a compatible language version; that syntax apparently requires a path-to-application argument that is a path rather than just a tool name.