Thoughts on comprenhension iterator-expressions?

[pedro-gilmora]

I think that it should be great to have some sugar syntax for simple collections dynamic initializers with the iterator-expression of your choice like

List<ItemDto> itemsCopy = [..foreach(var item in items) yield item.toDto()];

it could be while or for

[huoyaoyuan]

This feels like a combination of collection expression spreads and improved, pre-lowered LINQ.

Currently with LINQ, the followings are allowed:

xxx = [..items.Select(item => item.toDto())];

or with query syntax:

xxx = [..(from item in items select item.toDto())];

or method chain syntax:

xxx = items.Select(item => item.toDto()).ToXXCollection();

I think allowing LINQ to be lowered into more efficient form (#2482) should help more.

[HaloFour]

I showed you what extra things introduces such approach. I'm not saying it's wrong or too much expensive

Those are implementation details.

it's just that I feel I'm taking a cleaner approach of the existing scope, and maybe optimized.

That's your opinion. It's not enough to justify adding a second flavor of query comprehensions to the language. That's simply not going to happen. If you want to suggest specific enhancements to the existing LINQ query comprehensions then maybe those could be taken into advisement, and you'd automatically be able to compose that with collection literal expressions. However, if your goal is to have the language explicitly support a pattern which is considered non-idiomatic, it's very unlikely to gain any traction, especially since you can already bridge those patterns trivially.

[HaloFour]

If C# were to gain inline iterators, it would be orthogonal to collection literal expressions. It would be expected to be able to assign them directly to a variable, pass them around, use them with LINQ, etc. They would be expected to behave as normal iterators would, with deferred execution. As such, I don't see any proposal to add inline iterators within a collection literal to have any different semantics to any other iterator, whether it's declared as a local function or otherwise.

[pedro-gilmora]

oh man, that's your sight, I see you don't get the point, while I get yours. but not a problem, you must know the work/effort it takes, but I'm sure it's not bigger than the pattern matching one. So, no big deal. you're not thinking on multiple statements. and I don't think it's orthogonal, in fact you're just building an expression, what else would take to enrich with same natural languaje. Many others langs have list comprenhensions witout having to transform into methods like LINQ does

[HaloFour]

Ultimately, collection expressions already work with LINQ. The language already provides a solution here. Another solution isn't going to be considered. Solutions that benefit the whole of LINQ might be, if they can demonstrate that they are worth the effort.

But that's fine, your proposal is out there.

[pedro-gilmora]

I see it like a radical position, it would be nicer to mantain natural expressions, rather than potentiating a whole query lib. I see it both related. I know the benefits and the usage of LINQ of course. but I guess you don't think of engagement that would bring. then you'll have to potentiate query syntax to be more loveable and give more clauses and keyworkds, which I find rediculous if you already have the language itself

[KennethHoff]

It feels wrong inlining statements like this

[pedro-gilmora]

they don't need to be inlined if you want, but should yield the thing we need, that's why I set a poll option saying "can be improved"

[HaloFour]

C# already has this syntax, in the form of LINQ. You can already spread such an expression within a list literal. I would suggest that it's massively unlikely for the team to consider designing a completely different syntax to accomplish the same thing.

[pedro-gilmora]

I know lately LINQ had some improvements, but what I see it's something

Eg:

List<int> list = [1,2,3,4];

gets lowered into

List<int> list = new List<int>(4);
list.Add(1);
list.Add(2);
list.Add(3);
list.Add(4);

Then. something like

List<StudentDto> studentsDto= [..while(NextStudent(out var student)) yield ToStudentDto(student)];

whould go lower into something like

List<StudentDto> list = new List<StudentDto>();
Student student;
while (NextStudent(out student))
{
    list.Add(ToStudentDto(student));
}

[HaloFour]

Collection literals and LINQ are orthogonal concerns. I would not expect any changes to any comprehension syntax that would take place outside of LINQ.

[jnm2]

I'll also say, I'm not sure it's a common use case to do while (NextThing(out var thing)) to build a collection, but this use case is also something that a helper method would excel at:

List<StudentDto> list = Helpers.ToEnumerable(NextStudent).Select(ToStudentDto).ToList();

[pedro-gilmora]

Hey, it's OK to not being agree or expect it. I'm just getting some feedback about this proposal being considered as easy coding

[NetMage]

@HaloFour Unfortunately LINQ (especially query syntax) often feels like a sad, neglected corner of C#, though I have been glad to see some Enumerable extensions enhanced in the last few .Net (Core) releases. Without Expression improvements, LINQ is stuck in the old C# world.

[pedro-gilmora]

I'll also say, I'm not sure it's a common use case to do while (NextThing(out var thing)) to build a collection, but this use case is also something that a helper method would excel at:

List<StudentDto> list = Helpers.ToEnumerable(NextStudent).Select(ToStudentDto).ToList();

Ok, I see you're not very fan of sugar syntax.. 😅

[HaloFour]

or, if you find the query comprehension syntax a tad verbose for that simple case:

List<StudentDto> list = [.. NextStudent.AsEnumerable().Select(ToStudentDto)];

[pedro-gilmora]

you're just not exploding the potencial of comprenhension expressions and how much can reduce overhead at compile time. (I think)

List<StudentDto> list = [.. while(NextStudent(conditions, out var student) yield ToStudentDto(student, context)];

[HaloFour]

You're going to have to demonstrate that a completely new comprehension syntax actually provides those kinds of benefits. And frankly, even if it did, it would be something wrapped up into LINQ, not a completely new and completely superfluous syntactic form. C# isn't designed that way.

[pedro-gilmora]

I'm thinking of something like switch-expressions but thought as collections expressions (like builder pipeline, with multiple iterators and constants). but LINQ lambdas introduces extra types in compile time like I said before while my proposal don't.

[jnm2]

Ah, it's the compiler-generated closure classes that you're trying to avoid, using in-place codegen rather than using calls to Enumerable.Where/Select/Range/etc? I think that may have been investigated, but I'm not sure. If the performance benefits can be shown to speak for themselves, then LINQ is the place to do it because it'll improve perf everywhere.

[En3Tho]

F# has this for example and it's very useful:

let values = [
    for ... do
        yield ...

    while ... do
       yield ...

    yield 1

    if ... then
        yield 2
        yield 3

   etc ...
]

It would be nice to have a way to use something like that in C# too (not sure about syntax though):

List<int> values = [
     1, 2, 3,
     ..{ // basically an inlined IEnumeable syntax ?
          for (int i = 0; i < 10; i++) { yield return i; }
          if (...) yield return 1;
          while (...) yield return 2;
     }
]

[pedro-gilmora]

that's what I'm talking

[HaloFour]

C# isn't going to adopt F#'s syntax.

C# has a query comprehension syntax. It's not going to add a second query comprehension syntax.

[HaloFour]

At best you can argue for inline iterators, although the C# team hasn't been that interested in that before. They consider local functions to be suitable to meet those needs.

[pedro-gilmora]

I don't see them comfortable and also not taking a clean approach of the scope it belongs (capturing variables and introduce them in extra classes). Like I said, it reduces the extra type creations and I found it very useful and simple for collection initializers. that's the key point

[En3Tho]

@HaloFour I know C# won't. It was just an example. F# has much terser syntax anyway and such things look quite better there (not to mention that my example used "yield" keyword which could be omitted).

The thing I like about inline iterators is that they can:

1. Be optimized better (e.g. if compiler can deduce that there are 5 elements added from the loop then it can inline that instead)
2. Remain being structs and not get promoted to the heap in form of IEnumerable + IEnumerator or even better simply being inlined in the code right there

This is especially important now that .Net 8 has great struct promotion capabilities

I think overall this is a good addition to collection literals. I don't really like verbosity of it but at the same time IEnumerable syntax is the same so..

@CyrusNajmabadi What do you think about this?