Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: Add docs to RepartitionExec and architecture guide #7003

Merged
merged 2 commits into from
Jul 19, 2023

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Jul 17, 2023

Which issue does this PR close?

N/A

Rationale for this change

While writing up #7001 I made some diagrams and descriptions about Repartition that I felt would add value as doc comments

What changes are included in this PR?

Add docs to RepartitionExec and architecture guide

Are these changes tested?

CI tests the doc comments

Are there any user-facing changes?

Docs

@github-actions github-actions bot added logical-expr Logical plan and expressions core Core DataFusion crate labels Jul 17, 2023
@@ -239,8 +239,67 @@ impl BatchPartitioner {
}
}

/// The repartition operator maps N input partitions to M output partitions based on a
/// partitioning scheme. No guarantees are made about the order of the resulting partitions.
/// Maps `N` input partitions to `M output partitions based on a
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screenshot 2023-07-17 at 12 21 43 PM

@alamb alamb marked this pull request as ready for review July 17, 2023 16:21
Copy link
Contributor

@r4ntix r4ntix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

Copy link
Contributor

@2010YOUY01 2010YOUY01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really nice doc 👍🏼

datafusion/core/src/physical_plan/repartition/mod.rs Outdated Show resolved Hide resolved
Co-authored-by: Yongting You <2010youy01@gmail.com>
@alamb alamb merged commit 7a029a5 into apache:main Jul 19, 2023
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate logical-expr Logical plan and expressions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants