Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix derivation of LocalExchange single stream property #15203

Merged
merged 2 commits into from
Nov 28, 2022

Conversation

sopel39
Copy link
Member

@sopel39 sopel39 commented Nov 25, 2022

Description

LocalExchange will preserve single stream property only if:

  1. global partitioning is either on coordinator or single-node
  2. LocalExchange is gathering

Release notes

( ) This is not user-visible or docs only and no release notes are required.
( ) Release notes are required, please propose a release note for me.
(x) Release notes are required, with the following suggested text:

# General
* Fix a correctness bug for queries with certain complex window operators used in sequence 

@martint
Copy link
Member

martint commented Nov 28, 2022

This still produces an incorrect plan:

 Fragment 0 [SINGLE]
     Output layout: [orderkey, count]
     Output partitioning: SINGLE []
     Output[columnNames = [orderkey, _col1]]
     │   Layout: [orderkey:bigint, count:bigint]
     │   Estimates: {rows: ? (?), cpu: 0, memory: 0B, network: 0B}
     │   _col1 := count
     └─ TopN[count = 5, orderBy = [orderkey ASC NULLS LAST]]
        │   Layout: [orderkey:bigint, count:bigint]
        └─ LocalExchange[partitioning = SINGLE]
           │   Layout: [orderkey:bigint, count:bigint]
           │   Estimates: {rows: 5 (90B), cpu: 0, memory: 0B, network: 0B}
           └─ LimitPartial[count = 5, inputPreSortedBy = [orderkey]]           <<<<< input to this limit is not pre-sorted by orderkey
              │   Layout: [orderkey:bigint, count:bigint]
              │   Estimates: {rows: 5 (90B), cpu: 90, memory: 0B, network: 0B}
              └─ LocalExchange[partitioning = ROUND_ROBIN]
                 │   Layout: [orderkey:bigint, count:bigint]
                 │   Estimates: {rows: ? (?), cpu: ?, memory: 0B, network: 0B}
                 └─ Window[orderBy = [orderkey ASC NULLS LAST]]
                    │   Layout: [orderkey:bigint, count:bigint]
                    │   count := count() RANGE UNBOUNDED_PRECEDING CURRENT_ROW
                    └─ LocalExchange[partitioning = SINGLE]
                       │   Layout: [orderkey:bigint]
                       │   Estimates: {rows: 15000 (131.84kB), cpu: 0, memory: 0B, network: 0B}
                       └─ RemoteSource[sourceFragmentIds = [1]]
                              Layout: [orderkey:bigint]

@martint
Copy link
Member

martint commented Nov 28, 2022

Actually, disregard. It's correct -- it's sorted at the stream level.

LocalExchange will preserve single stream property
only if:
1. global partitioning is either on coordinator or single-node
2. LocalExchange is gathering
@sopel39 sopel39 merged commit 0ebdaa0 into trinodb:master Nov 28, 2022
@sopel39 sopel39 deleted the ks/fix_deriv branch November 28, 2022 20:15
@github-actions github-actions bot added this to the 404 milestone Nov 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

3 participants