Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[API Proposal]: PLINQ: add ParallelEnumerable extensions from Enumerable #98689

Open
timcassell opened this issue Feb 20, 2024 · 4 comments
Open
Labels
api-suggestion Early API idea and discussion, it is NOT ready for implementation area-System.Linq.Parallel needs-further-triage Issue has been initially triaged, but needs deeper consideration or reconsideration
Milestone

Comments

@timcassell
Copy link

timcassell commented Feb 20, 2024

Background and motivation

The documentation states "PLINQ implements the full set of LINQ standard query operators as extension methods for the System.Linq namespace and has additional operators for parallel operations.", but some new extensions were added to System.Linq.Enumerable that are missing in System.Linq.ParallelEnumerable. I propose they be added so that further queries will continue to be parallelizable (even if some extensions can't be parallelized, they should still return ParallelQuery<T>).

API Proposal

namespace System.Linq

public static partial class ParallelEnumerable
{
    // Append
    // Chunk
    // DistinctBy
    // ExceptBy
    // IntersectBy
    // MaxBy
    // MinBy
    // Order
    // OrderDescending
    // Prepend
    // SkipLast
    // TakeLast
    // ToHashSet
    // TryGetNonEnumeratedCount
    // UnionBy
}

All the missing extensions from System.Linq.Enumerable, with the same shape (I didn't bother to write out the full declarations for brevity).

And the new .Net 9 extensions:

namespace System.Linq

public static partial class ParallelEnumerable
{
    // AggregateBy
    // CountBy
    // Index
}

API Usage

Enumerable.Range(0, 100)
    .AsParallel()
    .Append(200)
    .TakeLast(20)
    .MinBy(x => x)

Alternative Designs

Use .AsSequential() before using the new extensions.

Risks

New extensions may not run in parallel (but that's already called out in PLINQ documentation).

@timcassell timcassell added the api-suggestion Early API idea and discussion, it is NOT ready for implementation label Feb 20, 2024
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Feb 20, 2024
@ghost
Copy link

ghost commented Feb 20, 2024

Tagging subscribers to this area: @dotnet/area-system-linq
See info in area-owners.md if you want to be subscribed.

Issue Details

Background and motivation

The documentation states PLINQ implements the full set of LINQ standard query operators as extension methods for the [System.Linq](https://learn.microsoft.com/en-us/dotnet/api/system.linq) namespace and has additional operators for parallel operations., but some new extensions were added to System.Linq.Enumerable that are missing in System.Linq.ParallelEnumerable. I propose they be added so that we don't need to use .AsSequential to use the new extensions.

API Proposal

namespace System.Linq

public static partial class ParallelEnumerable
{
    // Append
    // Chunk
    // DistinctBy
    // ExceptBy
    // IntersectBy
    // MaxBy
    // MinBy
    // Order
    // OrderDescending
    // Prepend
    // SkipLast
    // TakeLast
    // ToHashSet
    // TryGetNonEnumeratedCount
    // UnionBy
}

All the missing extensions from System.Linq.Enumerable, with the same shape (I didn't bother to write out the full declarations for brevity).

And the new .Net 9 extensions:

namespace System.Linq

public static partial class ParallelEnumerable
{
    // AggregateBy
    // CountBy
    // Index
}

API Usage

Enumerable.Range(0, 100)
    .AsParallel()
    .Append(200)
    .TakeLast(20)
    .MinBy(x => x)

Alternative Designs

Use .AsSequential() before using the new extensions.

Risks

New extensions may not run in parallel (but that's already called out in PLINQ documentation).

Author: timcassell
Assignees: -
Labels:

api-suggestion, area-System.Linq, untriaged

Milestone: -

@ghost
Copy link

ghost commented Feb 20, 2024

Tagging subscribers to this area: @dotnet/area-system-linq-parallel
See info in area-owners.md if you want to be subscribed.

Issue Details

Background and motivation

The documentation states "PLINQ implements the full set of LINQ standard query operators as extension methods for the System.Linq namespace and has additional operators for parallel operations.", but some new extensions were added to System.Linq.Enumerable that are missing in System.Linq.ParallelEnumerable. I propose they be added so that we don't need to use .AsSequential to use the new extensions.

API Proposal

namespace System.Linq

public static partial class ParallelEnumerable
{
    // Append
    // Chunk
    // DistinctBy
    // ExceptBy
    // IntersectBy
    // MaxBy
    // MinBy
    // Order
    // OrderDescending
    // Prepend
    // SkipLast
    // TakeLast
    // ToHashSet
    // TryGetNonEnumeratedCount
    // UnionBy
}

All the missing extensions from System.Linq.Enumerable, with the same shape (I didn't bother to write out the full declarations for brevity).

And the new .Net 9 extensions:

namespace System.Linq

public static partial class ParallelEnumerable
{
    // AggregateBy
    // CountBy
    // Index
}

API Usage

Enumerable.Range(0, 100)
    .AsParallel()
    .Append(200)
    .TakeLast(20)
    .MinBy(x => x)

Alternative Designs

Use .AsSequential() before using the new extensions.

Risks

New extensions may not run in parallel (but that's already called out in PLINQ documentation).

Author: timcassell
Assignees: -
Labels:

api-suggestion, area-System.Linq, area-System.Linq.Parallel, untriaged

Milestone: -

@eiriktsarpalis
Copy link
Member

Some of the methods that you listed such as Append, Prepend, Chunk, ToHashSet and TryGetNonEnumeratedCount are inherently non-parallelizable. I'm guessing they wouldn't be parallel as such, only accelerator methods mapping ParallelEnumerable values to the sequential implementations?

@timcassell
Copy link
Author

Some of the methods that you listed such as Append, Prepend, Chunk, ToHashSet and TryGetNonEnumeratedCount are inherently non-parallelizable. I'm guessing they wouldn't be parallel as such, only accelerator methods mapping ParallelEnumerable values to the sequential implementations?

Exactly.

Also, I'm sure I missed some new overloads for existing extensions (like ElementAt with a System.Index parameter). Basically I think the API surface should match like the intro-to-plinq documentation states, even though some can't actually be parallelized.

@jeffhandley jeffhandley added this to the 9.0.0 milestone Mar 21, 2024
@jeffhandley jeffhandley added needs-further-triage Issue has been initially triaged, but needs deeper consideration or reconsideration and removed untriaged New issue has not been triaged by the area owner labels Mar 21, 2024
@stephentoub stephentoub modified the milestones: 9.0.0, 10.0.0 Jun 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api-suggestion Early API idea and discussion, it is NOT ready for implementation area-System.Linq.Parallel needs-further-triage Issue has been initially triaged, but needs deeper consideration or reconsideration
Projects
None yet
Development

No branches or pull requests

4 participants