Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bottom_k should not include nulls if the column contains at least k valid elements #16748

Closed
stinodego opened this issue Jun 5, 2024 · 0 comments · Fixed by #16804
Closed
Assignees
Labels
A-ops Area: operations accepted Ready for implementation enhancement New feature or an improvement of an existing feature
Milestone

Comments

@stinodego
Copy link
Member

In discussing #15238, we decided that the behavior for top_k/bottom_k with regards to nulls should be:

  1. Valid elements have priority over null values: if a column contains at least k valid elements, the result will not contain any nulls
  2. If the column contains fewer than k valid elements, nulls from the original column are used to reach k resulting elements.
  3. If k is larger than the size of the column, the result will be all the elements of the original column. We do not 'pad' the result with nulls until we have k elements.

This is currently the case for top_k, but not for bottom_k. This behavior should be updated for 1.0.0.

@stinodego stinodego added enhancement New feature or an improvement of an existing feature accepted Ready for implementation A-ops Area: operations labels Jun 5, 2024
@stinodego stinodego added this to the 1.0.0 milestone Jun 5, 2024
@stinodego stinodego added this to Backlog Jun 5, 2024
@github-project-automation github-project-automation bot moved this to Ready in Backlog Jun 5, 2024
@stinodego stinodego moved this from Ready to Next in Backlog Jun 5, 2024
@github-project-automation github-project-automation bot moved this from Next to Done in Backlog Jun 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-ops Area: operations accepted Ready for implementation enhancement New feature or an improvement of an existing feature
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants