Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Iox 12409/df upgrade sept 4 eod #39

Open
wants to merge 11 commits into
base: iox-12409/df-base
Choose a base branch
from
Open

Iox 12409/df upgrade sept 4 eod #39

wants to merge 11 commits into from

Commits on Oct 15, 2024

  1. Test + workaround for SanityCheck plan

    @alamb @wiedld
    alamb authored and wiedld committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    e6142e8 View commit details
    Browse the repository at this point in the history

  2. Fix incorrect OFFSET during LIMIT pushdown. (apache#12399) 

    * test: demonstrate offset not applied correctly with limit pushdown on multiple input streams

* fix: do not pushdown when skip is applied

* test: update tests after fix

* chore: more doc cleanup

* chore: move LIMIT+OFFSET tests to proper sqllogic test case

* refactor: add global limit back (if there is a skip) during limit pushdown

* Apply suggestions from code review

* Add comment explaining why

---------

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
    @wiedld @alamb
    wiedld and alamb committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    f7000a5 View commit details
    Browse the repository at this point in the history

  3. Provide field and schema metadata missing on distinct aggregations. (a… 

    …pache#12691)

* test(12687): reproducer of missing metadata bug

* fix(12687): minimum change needed to fix the missing metadata
    @wiedld
    wiedld committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    21fe825 View commit details
    Browse the repository at this point in the history

  4. Provide field and schema metadata missing on cross joins, and union w… 

    …ith null fields. (apache#12729)

* test: reproducer for missing schema metadata on cross join

* fix: pass thru schema metadata on cross join

* fix: preserve metadata when transforming to view types

* test: reproducer for missing field metadata in left hand NULL field of union

* fix: preserve field metadata from right side of union

* chore: safe indexing
    @wiedld
    wiedld committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    6ff8146 View commit details
    Browse the repository at this point in the history

  5. fix(iox-11401): temporary patch to permit count col be nullable

    @wiedld
    wiedld committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    da17fed View commit details
    Browse the repository at this point in the history

  6. fix: when extracting metadata from expr, handle CastExpr

    @wiedld
    wiedld committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    9d32fd2 View commit details
    Browse the repository at this point in the history

  7. fix: handle when the left side of the union has no fields (e.g. an em… 

    …pty projection)
    @wiedld
    wiedld committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    cb2e845 View commit details
    Browse the repository at this point in the history

  8. fix: because either the left or right fields may be chosen, add metad… 

    …ata from both to each other
    @wiedld
    wiedld committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    50bba14 View commit details
    Browse the repository at this point in the history

  9. fix: now() UDF is not nullable

    @wiedld
    wiedld committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    9aa8976 View commit details
    Browse the repository at this point in the history

  10. chore: temp patch to convert errors to logged warnings, for schema mi… 

    …smatch in aggregates
    @wiedld
    wiedld committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    0c0a52d View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2024

  1. Update to arrow/parquet 53.0.0, tonic, prost, `object_store… 

    …`, `pyo3` (apache#12032)

* Update prost, prost-derive, pbjson

* udpate more

* Update datafusion/substrait/Cargo.toml

Co-authored-by: tison <wander4096@gmail.com>

* Update vendored code

* revert upgrade in datafusion-examples until arrow-flight is updated

* Pin to pre-release arrow-rs

* update pyo3

* Update to use new arrow apis

* update for new api

* Update tonic in examples

* update prost

* update datafusion-cli/cargo

* update test output

* update

* updates

* updates

* update math

* update more

* fix scalar tests

* Port statistics to use new API

* factor into a function

* update generated files

* Update test

* add new test

* update tests

* tapelo format

* Update other tests

* Update datafusion pin

* Update for API change

* Update to arrow 53.0.0 sha

* Update cli deps

* update cargo.lock

* Update expected output

* Remove patch

* update datafusion-cli cargo

* Pin some aws sdks whose update caused CI failures

---------

Co-authored-by: tison <wander4096@gmail.com>
    @alamb @tisonkun @wiedld
    2 people authored and wiedld committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    7024e5d View commit details
    Browse the repository at this point in the history