Consolidate LINQ's internal IIListProvider/IPartition into base Iterator class

[stephentoub]

LINQ has an internal base Iterator class that's used when operators manually implement enumeration rather than having the compiler implement it with an iterator. That base class includes several abstract/virtual methods, including a Select and Where virtual method that have been present since the beginning of LINQ: those are used in a case of A().B(), where B is Select or Where and where A can then improve the processing of B by returning a customized implementation aware of some aspect of both A and B (e.g. the enumerable returned from .Where().Select() includes both the where and select functionality in that single object). Over the years, other specialization has been added to LINQ, in the form of additional internal interfaces: IIListProvider and IPartition. These interfaces similarly enable optimizing sequences A().B(), where B is other LINQ methods, e.g. IIListProvider enables optimizing ToArray/ToList/Count, and IPartition enables optimizing Skip/Take/First/Last/ElementAt. There was a complicated venn diagram of which types implemented which interfaces and base type.

This PR merges IIListProvider/IPartition into the base Iterator class. Everything from IPartition is virtual, enabling derivations to specialize just a subset of the functionality, and deduplicating some implementations that were providing the same implementation instead of having it shared in a base. Code that was type testing for the interfaces now type tests for the base class, which means we can delete some type testing where both the interfaces and the base class were previously being tested for. We no longer have this strange split across multiple optimization-focused internal implementation details, and instead have everything consolidated in the one base class. This also means that all of the calls that were previously interface dispatch are now virtual dispatch.

I renamed some types along the way to clarify what they represent.

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);

[MemoryDiagnoser(false)]
[HideColumns("Job", "Error", "StdDev", "Median", "RatioSD")]
public partial class Tests
{
    private IEnumerable<int> _arrayDistinct = Enumerable.Range(0, 1000).ToArray().Distinct();
    private IEnumerable<int> _appendSelect = Enumerable.Range(0, 1000).ToArray().Append(42).Select(i => i * 2);
    private IEnumerable<int> _rangeReverse = Enumerable.Range(0, 1000).Reverse();
    private IEnumerable<int> _listDefaultIfEmptySelect = Enumerable.Range(0, 1000).ToList().DefaultIfEmpty().Select(i => i * 2);
    private IEnumerable<int> _listSkipTake = Enumerable.Range(0, 1000).ToList().Skip(500).Take(100);
    private IEnumerable<int> _rangeUnion = Enumerable.Range(0, 1000).Union(Enumerable.Range(500, 1000));
    private IEnumerable<int> _selectWhereSelect = Enumerable.Range(0, 1000).Select(i => i * 2).Where(i => i % 2 == 0).Select(i => i * 2);

    [Benchmark] public int DistinctFirst() => _arrayDistinct.First();
    [Benchmark] public int AppendSelectLast() => _appendSelect.Last();
    [Benchmark] public int RangeReverseCount() => _rangeReverse.Count();
    [Benchmark] public int DefaultIfEmptySelectElementAt() => _listDefaultIfEmptySelect.ElementAt(999);
    [Benchmark] public int ListSkipTakeElementAt() => _listSkipTake.ElementAt(99);
    [Benchmark] public int RangeUnionFirst() => _rangeUnion.First();
    [Benchmark] public int SelectWhereSelectSum() => _selectWhereSelect.Sum();
}

Method	Toolchain	Mean	Ratio	Allocated	Alloc Ratio
DistinctFirst	\main\corerun.exe	88.210 ns	1.00	328 B	1.00
DistinctFirst	\pr\corerun.exe	21.502 ns	0.24	-	0.00
AppendSelectLast	\main\corerun.exe	13,222.276 ns	1.000	144 B	1.00
AppendSelectLast	\pr\corerun.exe	7.720 ns	0.001	-	0.00
RangeReverseCount	\main\corerun.exe	14.113 ns	1.00	-	NA
RangeReverseCount	\pr\corerun.exe	10.177 ns	0.72	-	NA
DefaultIfEmptySelectElementAt	\main\corerun.exe	13,417.511 ns	1.000	144 B	1.00
DefaultIfEmptySelectElementAt	\pr\corerun.exe	14.814 ns	0.001	-	0.00
ListSkipTakeElementAt	\main\corerun.exe	8.274 ns	1.00	-	NA
ListSkipTakeElementAt	\pr\corerun.exe	6.704 ns	0.81	-	NA
RangeUnionFirst	\main\corerun.exe	93.941 ns	1.00	344 B	1.00
RangeUnionFirst	\pr\corerun.exe	11.522 ns	0.12	-	0.00
SelectWhereSelectSum	\main\corerun.exe	12,936.309 ns	1.00	112 B	1.00
SelectWhereSelectSum	\pr\corerun.exe	11,674.326 ns	0.90	112 B	1.00

[ghost]

Tagging subscribers to this area: @dotnet/area-system-linq
See info in area-owners.md if you want to be subscribed.

Issue Details

---

LINQ has an internal base Iterator class that's used when operators manually implement enumeration rather than having the compiler implement it with an iterator. That base class includes several abstract/virtual methods, including a Select and Where virtual method that have been present since the beginning of LINQ: those are used in a case of A().B(), where B is Select or Where and where A can then improve the processing of B by returning a customized implementation aware of some aspect of both A and B (e.g. the enumerable returned from .Where().Select() includes both the where and select functionality in that single object). Over the years, other specialization has been added to LINQ, in the form of additional internal interfaces: IIListProvider and IPartition. These interfaces similarly enable optimizing sequences A().B(), where B is other LINQ methods, e.g. IIListProvider enables optimizing ToArray/ToList/Count, and IPartition enables optimizing Skip/Take/First/Last/ElementAt. There was a complicated venn diagram of which types implemented which interfaces and base type.

This PR merges IIListProvider/IPartition into the base Iterator class. Everything from IPartition is virtual, enabling derivations to specialize just a subset of the functionality, and deduplicating some implementations that were providing the same implementation instead of having it shared in a base. Code that was type testing for the interfaces now type tests for the base class, which means we can delete some type testing where both the interfaces and the base class were previously being tested for. We no longer have this strange split across multiple optimization-focused internal implementation details, and instead have everything consolidated in the one base class. This also means that all of the calls that were previously interface dispatch are now virtual dispatch.

I renamed some types along the way to clarify what they represent.

Author:	stephentoub
Assignees:	-
Labels:	area-System.Linq
Milestone:	9.0.0

[eiriktsarpalis]

Thanks!

[MichalStrehovsky]

This regressed the size of the "Stage 2" app by 0.5%. It may not look like much, but AFAIK we already got rid of a bunch of LINQ use in ASP.NET because of the size and don't use it much.

I experimentally tried to compile the Stage 2 app using the optimized for size version of LINQ we use on iDevices and WASM and the size dropped by 1.5% - i.e. the cost of the extra generics introduced here is proportionally a significant portion of the LINQ overhead in general.

It looks like this:

Here is the before/after MSTAT file (it's not just this PR, but official build before regression and after regression).

before.zip
after.zip

Cc @eerhardt

[eiriktsarpalis]

Superficially the issue seems to be that calling, say, Select will pull in all "SelectIterator" types since the source type is not inferred at compile time.

[stephentoub]

using the optimized for size version of LINQ

Note, too, that the "optimized for size" version doesn't actually exclude all of the select variations. They were part of the original LINQ implementation, and whether on purpose or by accident didn't get ifdef'd out along with all of the other specializations. I expect that 1.5% would increase even more if the Where/Select virtuals on Iterator were put onto the same plan as all of the other abstracts/virtuals that are there to optimize various sequences of operations.

Fundamentally, this is a tradeoff between size and throughput. Each of these specializations makes a very real throughput improvement. If we believe the size is more important, we can switch Native AOT over to a plan like that used by browser, omitting all of these throughput optimizations that bring in more code (though I'd prefer if finding a way to do so via a feature switch rather than having multiple builds.

Select will pull in all "SelectIterator" types since the source type is not inferred at compile time.

Yes, this is why @eerhardt was replacing use of Select in #98109. But if we're going to write specialized code that is effectively just the simple version of Select in order to avoid pulling in all the various specializations, then we should think about just having Native AOT use the simple version of Select.

[MichalStrehovsky]

Fundamentally, this is a tradeoff between size and throughput. Each of these specializations makes a very real throughput improvement

I would expect it also affects startup because it will be more type loads. When we're doing perf changes, we rarely look at perf outside throughput. It's not a problem specific to this PR.

I haven't filed this as an issue because I'm not sure we care enough. Making LINQ even bigger will build a stronger case for people to avoid it due to perf. I already recommend people to delete all the references to it if they care about perf and put a https://github.com/dotnet/corert/blob/master/src/Common/src/TypeSystem/Common/LinqPoison.cs in their project to stay clean.