From c0f08174a3d411c4cb01709be41b72e163a12d13 Mon Sep 17 00:00:00 2001 From: Andrew Lamb Date: Thu, 10 Feb 2022 15:25:16 -0500 Subject: [PATCH] Update CHANGELOG.md, update release scripts --- CHANGELOG.md | 1 - datafusion/CHANGELOG.md | 287 ++++++++++++++++++++++++ dev/release/README.md | 4 +- dev/release/update_change_log-all.sh | 1 - dev/release/update_change_log-python.sh | 32 --- dev/release/update_change_log.sh | 8 +- 6 files changed, 293 insertions(+), 40 deletions(-) delete mode 100755 dev/release/update_change_log-python.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index 467cddc62c88..ed77f4bad26e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,7 +21,6 @@ Changelogs are maintained separately for each subproject. Please check out the changelog file within each subproject folder for more details: * [Datafusion CHANGELOG](./datafusion/CHANGELOG.md) -* [Datafusion Python Binding CHANGELOG](./python/CHANGELOG.md) * [Ballista CHANGELOG](./ballista/CHANGELOG.md) For older versions, see [apache/arrow/CHANGELOG.md](https://github.com/apache/arrow/blob/master/CHANGELOG.md). diff --git a/datafusion/CHANGELOG.md b/datafusion/CHANGELOG.md index c22b0553474c..8e04f75eaf5a 100644 --- a/datafusion/CHANGELOG.md +++ b/datafusion/CHANGELOG.md @@ -19,6 +19,293 @@ # Changelog +## [7.0.0](https://github.com/apache/arrow-datafusion/tree/7.0.0) (2022-02-10) + +[Full Changelog](https://github.com/apache/arrow-datafusion/compare/6.0.0...7.0.0) + +**Breaking changes:** + +- Consolidate various configurations options, remove unrelated `batch_size` [\#1565](https://github.com/apache/arrow-datafusion/issues/1565) +- Extract logical plans in LogicalPlan as independent struct [\#1228](https://github.com/apache/arrow-datafusion/issues/1228) +- Update `ExecutionPlan` to know about sortedness and repartitioning optimizer pass respect the invariants [\#1776](https://github.com/apache/arrow-datafusion/pull/1776) ([alamb](https://github.com/alamb)) +- Update to `arrow 8.0.0` [\#1673](https://github.com/apache/arrow-datafusion/pull/1673) ([alamb](https://github.com/alamb)) +- Remove non idiomatic `DataFusionError::into_arrow_external_error` in favor of From conversion [\#1645](https://github.com/apache/arrow-datafusion/pull/1645) ([alamb](https://github.com/alamb)) +- Remove `Accumulator::update` and `Accumulator::merge` [\#1582](https://github.com/apache/arrow-datafusion/pull/1582) ([Jimexist](https://github.com/Jimexist)) +- implement `Hash` for various types and replace `PartialOrd` [\#1580](https://github.com/apache/arrow-datafusion/pull/1580) ([Jimexist](https://github.com/Jimexist)) +- Replace `DatafusionError` with `GenericError` in `ObjectStore` interface [\#1541](https://github.com/apache/arrow-datafusion/pull/1541) ([matthewmturner](https://github.com/matthewmturner)) +- Make `FLOAT` SQL type map to `Float32` rather than `Float64` [\#1423](https://github.com/apache/arrow-datafusion/pull/1423) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([liukun4515](https://github.com/liukun4515)) +- Map `REAL` SQL type to `Float32` rather than `Float64` to be consistent with pg [\#1390](https://github.com/apache/arrow-datafusion/pull/1390) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([hntd187](https://github.com/hntd187)) + +**Implemented enhancements:** + +- Create new `datafusion_expr` crate [\#1753](https://github.com/apache/arrow-datafusion/issues/1753) +- Create new `datafusion_common` crate [\#1752](https://github.com/apache/arrow-datafusion/issues/1752) +- API to get Expr's type and nullability without a `DFSchema` [\#1725](https://github.com/apache/arrow-datafusion/issues/1725) +- Cleaner API to create `Expr::ScalarFunction` programatically [\#1718](https://github.com/apache/arrow-datafusion/issues/1718) +- Introduce a `Vec` based row-wise representation for DataFusion [\#1708](https://github.com/apache/arrow-datafusion/issues/1708) +- Simplify creating new `ListingTable` [\#1705](https://github.com/apache/arrow-datafusion/issues/1705) +- Implement TableProvider for DataFrameImpl to allow registration of logical plans [\#1698](https://github.com/apache/arrow-datafusion/issues/1698) +- Public Expr simplification API [\#1694](https://github.com/apache/arrow-datafusion/issues/1694) +- Query Optimizer: Add OUTER --\> INNER join conversion [\#1670](https://github.com/apache/arrow-datafusion/issues/1670) +- Support reading from CSV, Avro and Json files that have mergeable/compatible, but not identical schemas [\#1669](https://github.com/apache/arrow-datafusion/issues/1669) +- Remove `DataFusionError::into_arrow_external_error` in favor of `From` conversion [\#1644](https://github.com/apache/arrow-datafusion/issues/1644) +- Include join type in display implementation for logical plan [\#1620](https://github.com/apache/arrow-datafusion/issues/1620) +- Switch datafusion to using `eq_dyn_scalar`, etc kernels [\#1610](https://github.com/apache/arrow-datafusion/issues/1610) +- Proposal: Remove `Accumulator::update` and `Accumulator::merge` [\#1549](https://github.com/apache/arrow-datafusion/issues/1549) +- Replace DataFusionError/Result with impl Error for ObjectStore and Reader [\#1540](https://github.com/apache/arrow-datafusion/issues/1540) +- Add `approx_quantile` support [\#1538](https://github.com/apache/arrow-datafusion/issues/1538) +- support sorting decimal data type [\#1522](https://github.com/apache/arrow-datafusion/issues/1522) +- Keep all datafusion's packages up to date with Dependabot [\#1472](https://github.com/apache/arrow-datafusion/issues/1472) +- ExecutionContext support init ExecutionContextState with `new(state: Arc>)` method [\#1439](https://github.com/apache/arrow-datafusion/issues/1439) +- support the decimal scalar value [\#1393](https://github.com/apache/arrow-datafusion/issues/1393) +- Documentation for using scalar functions with the the DataFrame API [\#1364](https://github.com/apache/arrow-datafusion/issues/1364) +- Support `boolean == boolean` and `boolean != boolean` operators [\#1159](https://github.com/apache/arrow-datafusion/issues/1159) +- Support DataType::Decimal\(15, 2\) in TPC-H benchmark [\#174](https://github.com/apache/arrow-datafusion/issues/174) +- Make `MemoryStream` public [\#150](https://github.com/apache/arrow-datafusion/issues/150) +- Add support for Parquet schema merging [\#132](https://github.com/apache/arrow-datafusion/issues/132) +- Add SQL support for IN expression [\#118](https://github.com/apache/arrow-datafusion/issues/118) +- Add logging to datafusion-cli [\#1789](https://github.com/apache/arrow-datafusion/pull/1789) ([alamb](https://github.com/alamb)) +- Add `approx_median()` aggregate function [\#1729](https://github.com/apache/arrow-datafusion/pull/1729) ([realno](https://github.com/realno)) +- Add join type for logical plan display [\#1674](https://github.com/apache/arrow-datafusion/pull/1674) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([xudong963](https://github.com/xudong963)) +- Fix null comparison for Parquet pruning predicate [\#1595](https://github.com/apache/arrow-datafusion/pull/1595) ([viirya](https://github.com/viirya)) +- Add `corr` aggregate function [\#1561](https://github.com/apache/arrow-datafusion/pull/1561) ([realno](https://github.com/realno)) +- Add `covar`, `covar_pop` and `covar_samp` aggregate functions [\#1551](https://github.com/apache/arrow-datafusion/pull/1551) ([realno](https://github.com/realno)) +- Add `approx_quantile()` aggregation function [\#1539](https://github.com/apache/arrow-datafusion/pull/1539) ([domodwyer](https://github.com/domodwyer)) +- Initial MemoryManager and DiskManager APIs for query execution + External Sort implementation [\#1526](https://github.com/apache/arrow-datafusion/pull/1526) ([yjshen](https://github.com/yjshen)) +- Add `stddev` and `variance` [\#1525](https://github.com/apache/arrow-datafusion/pull/1525) ([realno](https://github.com/realno)) +- Add `rem` operation for Expr [\#1467](https://github.com/apache/arrow-datafusion/pull/1467) ([liukun4515](https://github.com/liukun4515)) +- support decimal data type in create table [\#1431](https://github.com/apache/arrow-datafusion/pull/1431) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([liukun4515](https://github.com/liukun4515)) +- Ordering by index in select expression [\#1419](https://github.com/apache/arrow-datafusion/pull/1419) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([hntd187](https://github.com/hntd187)) +- Add support for `ORDER BY` on unprojected columns [\#1415](https://github.com/apache/arrow-datafusion/pull/1415) ([viirya](https://github.com/viirya)) +- Support decimal for `min` and `max` aggregate [\#1407](https://github.com/apache/arrow-datafusion/pull/1407) ([liukun4515](https://github.com/liukun4515)) +- Consolidate `ConstantFolding` and `SimplifyExpression` [\#1375](https://github.com/apache/arrow-datafusion/pull/1375) ([alamb](https://github.com/alamb)) +- Datafusion cli quiet mode command to contain option bool [\#1345](https://github.com/apache/arrow-datafusion/pull/1345) ([Jimexist](https://github.com/Jimexist)) +- Implement `array_agg` aggregate function [\#1300](https://github.com/apache/arrow-datafusion/pull/1300) ([viirya](https://github.com/viirya)) +- Add a command to switch output format in cli [\#1284](https://github.com/apache/arrow-datafusion/pull/1284) ([capkurmagati](https://github.com/capkurmagati)) +- Support `=`, `<`, `<=`, `>`, `>=`, `!=`, `is distinct from`, `is not distinct from` for `BooleanArray` [\#1163](https://github.com/apache/arrow-datafusion/pull/1163) ([alamb](https://github.com/alamb)) + +**Fixed bugs:** + +- Unsupported data type in hasher: Timestamp\(Second, None\) [\#1768](https://github.com/apache/arrow-datafusion/issues/1768) +- SQL column identifiers should be converted to lowercase when unquoted [\#1746](https://github.com/apache/arrow-datafusion/issues/1746) +- Data type Dictionary\(Int32, Utf8\) not supported for binary operation 'eq' on dyn arrays [\#1605](https://github.com/apache/arrow-datafusion/issues/1605) +- datafusion doesn't process predicate pushdown correctly when there is outer join [\#1586](https://github.com/apache/arrow-datafusion/issues/1586) +- casting `Int64` to `Float64` unsuccessfully caused tpch8 to fail [\#1576](https://github.com/apache/arrow-datafusion/issues/1576) +- CTE/WITH .. UNION ALL confuses name resolution in WHERE [\#1509](https://github.com/apache/arrow-datafusion/issues/1509) +- ORDER BY min\(x\) results in error `Plan("No field named 'foo.x'. Valid fields are 'MIN(foo.x)'.")` [\#1479](https://github.com/apache/arrow-datafusion/issues/1479) +- Sort discards field metadata on the output schema [\#1476](https://github.com/apache/arrow-datafusion/issues/1476) +- Datafusion should not strip out timezone information from existing types [\#1454](https://github.com/apache/arrow-datafusion/issues/1454) +- Error on some queries: "column types must match schema types, expected XXX but found YYY" [\#1447](https://github.com/apache/arrow-datafusion/issues/1447) +- Query failing to return any results when filter is an equality check on strings \(bad statistics in parquet\) [\#1433](https://github.com/apache/arrow-datafusion/issues/1433) +- Field names containing period such as `f.c1` cannot be named in SQL query [\#1432](https://github.com/apache/arrow-datafusion/issues/1432) +- `Select *` returns an unexpected result [\#1412](https://github.com/apache/arrow-datafusion/issues/1412) +- Turn off unused default features of chrono and ahash [\#1398](https://github.com/apache/arrow-datafusion/issues/1398) +- real data type is float32 in PG database, but in the datafusion it is as float64 [\#1380](https://github.com/apache/arrow-datafusion/issues/1380) +- TPC-H q10 performance regression \(expression for filter with added alias is not pushed down\) [\#1367](https://github.com/apache/arrow-datafusion/issues/1367) +- ProjectionExec Loses Field Metadata [\#1361](https://github.com/apache/arrow-datafusion/issues/1361) +- Support Filter on unprojected columns [\#1351](https://github.com/apache/arrow-datafusion/issues/1351) +- NULLS ORDER is inconsistent with postgres [\#1343](https://github.com/apache/arrow-datafusion/issues/1343) +- Fix bug while merging `RecordBatch`, add `SortPreservingMerge` fuzz tester [\#1678](https://github.com/apache/arrow-datafusion/pull/1678) ([alamb](https://github.com/alamb)) +- fix a cte block with same name for many times [\#1639](https://github.com/apache/arrow-datafusion/pull/1639) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([xudong963](https://github.com/xudong963)) +- fix: casting Int64 to Float64 unsuccessfully caused tpch8 to fail [\#1601](https://github.com/apache/arrow-datafusion/pull/1601) ([xudong963](https://github.com/xudong963)) +- Fix single\_distinct\_to\_groupby for arbitrary expressions [\#1519](https://github.com/apache/arrow-datafusion/pull/1519) ([james727](https://github.com/james727)) +- Fix SortExec discards field metadata on the output schema [\#1477](https://github.com/apache/arrow-datafusion/pull/1477) ([alamb](https://github.com/alamb)) +- fix calculate in many\_to\_many\_hash\_partition test. [\#1463](https://github.com/apache/arrow-datafusion/pull/1463) ([Ted-Jiang](https://github.com/Ted-Jiang)) +- Add Timezone to Scalar::Time\* types, and better timezone awareness to Datafusion's time types [\#1455](https://github.com/apache/arrow-datafusion/pull/1455) ([maxburke](https://github.com/maxburke)) +- Support identifiers with `.` in them [\#1449](https://github.com/apache/arrow-datafusion/pull/1449) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([alamb](https://github.com/alamb)) +- Fixes for working with functions in dataframes, additional documentation [\#1430](https://github.com/apache/arrow-datafusion/pull/1430) ([tobyhede](https://github.com/tobyhede)) +- \[Minor\] Fix `send_time` metric for hash-repartition [\#1421](https://github.com/apache/arrow-datafusion/pull/1421) ([Dandandan](https://github.com/Dandandan)) +- fix: Select \* returns an unexpected result [\#1413](https://github.com/apache/arrow-datafusion/pull/1413) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([xudong963](https://github.com/xudong963)) +- Make cli handle multiple whitespaces [\#1388](https://github.com/apache/arrow-datafusion/pull/1388) ([capkurmagati](https://github.com/capkurmagati)) +- Metadata is kept in projections for non-derived columns [\#1378](https://github.com/apache/arrow-datafusion/pull/1378) ([hntd187](https://github.com/hntd187)) +- Fix Predicate Pushdown: split\_members should be able to split aliased predicate [\#1368](https://github.com/apache/arrow-datafusion/pull/1368) ([viirya](https://github.com/viirya)) +- Change the arg names and make parameters more meaningful [\#1357](https://github.com/apache/arrow-datafusion/pull/1357) ([liukun4515](https://github.com/liukun4515)) +- collect table stats by default for listing table [\#1347](https://github.com/apache/arrow-datafusion/pull/1347) ([houqp](https://github.com/houqp)) +- fix: make nulls-order consistent with postgres [\#1344](https://github.com/apache/arrow-datafusion/pull/1344) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([xudong963](https://github.com/xudong963)) +- Avoid changing expression names during constant folding [\#1319](https://github.com/apache/arrow-datafusion/pull/1319) ([viirya](https://github.com/viirya)) +- improve error message for invalid create table statement [\#1294](https://github.com/apache/arrow-datafusion/pull/1294) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([houqp](https://github.com/houqp)) +- Forbid creating the table with the same name [\#1288](https://github.com/apache/arrow-datafusion/pull/1288) ([liukun4515](https://github.com/liukun4515)) + +**Documentation updates:** + +- Clarify docs about `Accumulator::update` and `Accumulator::update_batch` [\#1542](https://github.com/apache/arrow-datafusion/pull/1542) ([alamb](https://github.com/alamb)) +- Fix duplicated `cargo run --example parquet_sql` [\#1482](https://github.com/apache/arrow-datafusion/pull/1482) ([sergey-melnychuk](https://github.com/sergey-melnychuk)) +- add documentation to Datafusion cli's new commands [\#1348](https://github.com/apache/arrow-datafusion/pull/1348) ([liukun4515](https://github.com/liukun4515)) +- fix some clippy warnings from nightly channel [\#1277](https://github.com/apache/arrow-datafusion/pull/1277) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([Jimexist](https://github.com/Jimexist)) + +**Performance improvements:** + +- Parquet pruning predicate for `IS NULL` [\#1591](https://github.com/apache/arrow-datafusion/issues/1591) +- Fix predicate pushdown for outer joins [\#1618](https://github.com/apache/arrow-datafusion/pull/1618) ([james727](https://github.com/james727)) +- fix: sql planner creates cross join instead of inner join from select predicates [\#1566](https://github.com/apache/arrow-datafusion/pull/1566) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([xudong963](https://github.com/xudong963)) +- Split fetch\_metadata into fetch\_statistics and fetch\_schema [\#1365](https://github.com/apache/arrow-datafusion/pull/1365) ([Dandandan](https://github.com/Dandandan)) +- Optimize the performance queries with a single distinct aggregate [\#1315](https://github.com/apache/arrow-datafusion/pull/1315) ([ic4y](https://github.com/ic4y)) +- Left join could use bitmap for left join instead of Vec\ [\#1291](https://github.com/apache/arrow-datafusion/pull/1291) ([boazberman](https://github.com/boazberman)) + +**Closed issues:** + +- Add `release compile` to CI [\#1728](https://github.com/apache/arrow-datafusion/issues/1728) +- DiskManager and TempFiles getting created several times per query [\#1690](https://github.com/apache/arrow-datafusion/issues/1690) +- Add a test for the `pyarrow` feature in CI [\#1635](https://github.com/apache/arrow-datafusion/issues/1635) +- SQL tests for when sorting exceeded available memory and had to spill to disk [\#1573](https://github.com/apache/arrow-datafusion/issues/1573) +- Consolidate the N-way merging code and `SortPreservingMergeStream` \(which has quite good tests of what is often quite tricky code, and it will be performance critical\) [\#1572](https://github.com/apache/arrow-datafusion/issues/1572) +- Consolidate the `SortExec` code \(so there is only a single sort operator that does in memory sorting if it has enough memory budget but then spills to disk if needed\). [\#1571](https://github.com/apache/arrow-datafusion/issues/1571) +- Track memory usage in Non Limited Operators [\#1569](https://github.com/apache/arrow-datafusion/issues/1569) +- \[Question\] Why does ballista store tables in the client instead of in the SchedulerServer [\#1473](https://github.com/apache/arrow-datafusion/issues/1473) +- Consolidate Projection for Schema and RecordBatch [\#1425](https://github.com/apache/arrow-datafusion/issues/1425) +- Support Sort on unprojected columns [\#1372](https://github.com/apache/arrow-datafusion/issues/1372) +- Unused code in hash\_aggregate [\#1362](https://github.com/apache/arrow-datafusion/issues/1362) +- Why use the expr types before coercion to get the result type? [\#1358](https://github.com/apache/arrow-datafusion/issues/1358) +- A problem about the projection\_push\_down optimizer gathers valid columns [\#1312](https://github.com/apache/arrow-datafusion/issues/1312) +- apply constant folding to `LogicalPlan::Values` [\#1170](https://github.com/apache/arrow-datafusion/issues/1170) +- reduce usage of `IntoIterator` in logical plan builder window fn [\#372](https://github.com/apache/arrow-datafusion/issues/372) +- Why does DataFusion throw a Tokio 0.2 runtime error? [\#176](https://github.com/apache/arrow-datafusion/issues/176) +- TPC-H Query 14 [\#165](https://github.com/apache/arrow-datafusion/issues/165) +- Length kernel returns bytes not character length [\#156](https://github.com/apache/arrow-datafusion/issues/156) +- Split the logical operators out into separate source files [\#115](https://github.com/apache/arrow-datafusion/issues/115) + +**Merged pull requests:** + +- Update versions for split crates [\#1803](https://github.com/apache/arrow-datafusion/pull/1803) ([matthewmturner](https://github.com/matthewmturner)) +- Improve the error message and UX of tpch benchmark program [\#1800](https://github.com/apache/arrow-datafusion/pull/1800) ([alamb](https://github.com/alamb)) +- rename references of expr in logical plan module after datafusion-expr split [\#1797](https://github.com/apache/arrow-datafusion/pull/1797) ([Jimexist](https://github.com/Jimexist)) +- \[split/13\] move rest of expr to expr\_fn in datafusion-expr module [\#1794](https://github.com/apache/arrow-datafusion/pull/1794) ([Jimexist](https://github.com/Jimexist)) +- Update datafusion versions [\#1793](https://github.com/apache/arrow-datafusion/pull/1793) ([matthewmturner](https://github.com/matthewmturner)) +- Less verbose plans in debug logging [\#1787](https://github.com/apache/arrow-datafusion/pull/1787) ([alamb](https://github.com/alamb)) +- \[split/11\] split expr type and null info to be expr-schemable [\#1784](https://github.com/apache/arrow-datafusion/pull/1784) ([Jimexist](https://github.com/Jimexist)) +- Introduce `Row` format backed by raw bytes [\#1782](https://github.com/apache/arrow-datafusion/pull/1782) ([yjshen](https://github.com/yjshen)) +- rewrite predicates before pushing to union inputs [\#1781](https://github.com/apache/arrow-datafusion/pull/1781) ([korowa](https://github.com/korowa)) +- \[split/10\] split up expr for rewriting, visiting, and simplification traits [\#1774](https://github.com/apache/arrow-datafusion/pull/1774) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([Jimexist](https://github.com/Jimexist)) +- \#1768 Support TimeUnit::Second in hasher [\#1769](https://github.com/apache/arrow-datafusion/pull/1769) ([jychen7](https://github.com/jychen7)) +- TPC-H benchmark can optionally write JSON output file with benchmark summary [\#1766](https://github.com/apache/arrow-datafusion/pull/1766) ([andygrove](https://github.com/andygrove)) +- \[split/8\] move `Accumulator` and `ColumnarValue` to datafusion-expr [\#1765](https://github.com/apache/arrow-datafusion/pull/1765) ([Jimexist](https://github.com/Jimexist)) +- \[split/7\] move built-in scalar function to datafusion-expr [\#1764](https://github.com/apache/arrow-datafusion/pull/1764) ([Jimexist](https://github.com/Jimexist)) +- \[split/6\] move signature, type signature, volatility to datafusion-expr [\#1763](https://github.com/apache/arrow-datafusion/pull/1763) ([Jimexist](https://github.com/Jimexist)) +- \[split/9+12\] move udf, udaf, `Expr` to datafusion-expr module [\#1762](https://github.com/apache/arrow-datafusion/pull/1762) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([Jimexist](https://github.com/Jimexist)) +- \[split/5\] move window frame and operator to datafusion-expr module [\#1761](https://github.com/apache/arrow-datafusion/pull/1761) ([Jimexist](https://github.com/Jimexist)) +- \[split/4\] move scalar value to datafusion-common [\#1760](https://github.com/apache/arrow-datafusion/pull/1760) ([Jimexist](https://github.com/Jimexist)) +- \[split/3\] split datafusion expr module and move aggregate and window function expr [\#1759](https://github.com/apache/arrow-datafusion/pull/1759) ([Jimexist](https://github.com/Jimexist)) +- \[split/2\] move column and dfschema to datafusion-common module [\#1758](https://github.com/apache/arrow-datafusion/pull/1758) ([Jimexist](https://github.com/Jimexist)) +- Use ordered-float 2.10 [\#1756](https://github.com/apache/arrow-datafusion/pull/1756) ([andygrove](https://github.com/andygrove)) +- \[split/1\] split datafusion-common module [\#1751](https://github.com/apache/arrow-datafusion/pull/1751) ([Jimexist](https://github.com/Jimexist)) +- use clap 3 style args parsing for datafusion cli [\#1749](https://github.com/apache/arrow-datafusion/pull/1749) ([Jimexist](https://github.com/Jimexist)) +- fix: Case insensitive unquoted identifiers in SQL [\#1747](https://github.com/apache/arrow-datafusion/pull/1747) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([mkmik](https://github.com/mkmik)) +- Move more tests out of context.rs [\#1743](https://github.com/apache/arrow-datafusion/pull/1743) ([alamb](https://github.com/alamb)) +- Move optimize test out of context.rs [\#1742](https://github.com/apache/arrow-datafusion/pull/1742) ([alamb](https://github.com/alamb)) +- Fix typos in crate documentation [\#1739](https://github.com/apache/arrow-datafusion/pull/1739) ([r4ntix](https://github.com/r4ntix)) +- add `cargo check --release` to ci [\#1737](https://github.com/apache/arrow-datafusion/pull/1737) ([xudong963](https://github.com/xudong963)) +- Update parking\_lot requirement from 0.11 to 0.12 [\#1735](https://github.com/apache/arrow-datafusion/pull/1735) ([dependabot[bot]](https://github.com/apps/dependabot)) +- Create built-in scalar functions programmatically [\#1734](https://github.com/apache/arrow-datafusion/pull/1734) ([HaoYang670](https://github.com/HaoYang670)) +- Prevent repartitioning of certain operator's direct children \(\#1731\) [\#1732](https://github.com/apache/arrow-datafusion/pull/1732) ([tustvold](https://github.com/tustvold)) +- API to get Expr's type and nullability without a `DFSchema` [\#1726](https://github.com/apache/arrow-datafusion/pull/1726) ([alamb](https://github.com/alamb)) +- minor: fix `cargo run --release` error [\#1723](https://github.com/apache/arrow-datafusion/pull/1723) ([xudong963](https://github.com/xudong963)) +- substitute `parking_lot::Mutex` for `std::sync::Mutex` [\#1720](https://github.com/apache/arrow-datafusion/pull/1720) ([xudong963](https://github.com/xudong963)) +- Convert boolean case expressions to boolean logic [\#1719](https://github.com/apache/arrow-datafusion/pull/1719) ([tustvold](https://github.com/tustvold)) +- Add Expression Simplification API [\#1717](https://github.com/apache/arrow-datafusion/pull/1717) ([alamb](https://github.com/alamb)) +- Create ListingTableConfig which includes file format and schema inference [\#1715](https://github.com/apache/arrow-datafusion/pull/1715) ([matthewmturner](https://github.com/matthewmturner)) +- make `select_to_plan` clearer [\#1714](https://github.com/apache/arrow-datafusion/pull/1714) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([xudong963](https://github.com/xudong963)) +- Add upper bound for public function `signature` [\#1713](https://github.com/apache/arrow-datafusion/pull/1713) ([HaoYang670](https://github.com/HaoYang670)) +- Add tests and CI for optional pyarrow module [\#1711](https://github.com/apache/arrow-datafusion/pull/1711) ([wjones127](https://github.com/wjones127)) +- Create SchemaAdapter trait to map table schema to file schemas [\#1709](https://github.com/apache/arrow-datafusion/pull/1709) ([thinkharderdev](https://github.com/thinkharderdev)) +- refine test in repartition.rs & coalesce\_batches.rs [\#1707](https://github.com/apache/arrow-datafusion/pull/1707) ([xudong963](https://github.com/xudong963)) +- Fuzz test for spillable sort [\#1706](https://github.com/apache/arrow-datafusion/pull/1706) ([yjshen](https://github.com/yjshen)) +- Support `create_physical_expr` and `ExecutionContextState` or `DefaultPhysicalPlanner` for faster speed [\#1700](https://github.com/apache/arrow-datafusion/pull/1700) ([alamb](https://github.com/alamb)) +- Implement TableProvider for DataFrameImpl [\#1699](https://github.com/apache/arrow-datafusion/pull/1699) ([cpcloud](https://github.com/cpcloud)) +- Move timestamp related tests out of context.rs and into sql integration test [\#1696](https://github.com/apache/arrow-datafusion/pull/1696) ([alamb](https://github.com/alamb)) +- Lazy TempDir creation in DiskManager [\#1695](https://github.com/apache/arrow-datafusion/pull/1695) ([alamb](https://github.com/alamb)) +- Add `MemTrackingMetrics` to ease memory tracking for non-limited memory consumers [\#1691](https://github.com/apache/arrow-datafusion/pull/1691) ([yjshen](https://github.com/yjshen)) +- \(minor\) Reduce memory manager and disk manager logs from `info!` to `debug!` [\#1689](https://github.com/apache/arrow-datafusion/pull/1689) ([alamb](https://github.com/alamb)) +- Make `SortPreservingMergeStream` stable on input stream order [\#1687](https://github.com/apache/arrow-datafusion/pull/1687) ([alamb](https://github.com/alamb)) +- Incorporate dyn scalar kernels [\#1685](https://github.com/apache/arrow-datafusion/pull/1685) ([matthewmturner](https://github.com/matthewmturner)) +- Move `information_schema` tests out of execution/context.rs to `sql_integration` tests [\#1684](https://github.com/apache/arrow-datafusion/pull/1684) ([alamb](https://github.com/alamb)) +- Add a new metric type: `Gauge` + `CurrentMemoryUsage` to metrics [\#1682](https://github.com/apache/arrow-datafusion/pull/1682) ([yjshen](https://github.com/yjshen)) +- refactor array\_agg to not to have `update` and `merge` [\#1681](https://github.com/apache/arrow-datafusion/pull/1681) ([Jimexist](https://github.com/Jimexist)) +- Use NamedTempFile rather than `String` in DiskManager [\#1680](https://github.com/apache/arrow-datafusion/pull/1680) ([alamb](https://github.com/alamb)) +- upgrade clap to version 3 [\#1672](https://github.com/apache/arrow-datafusion/pull/1672) ([Jimexist](https://github.com/Jimexist)) +- Improve configuration and resource use of `MemoryManager` and `DiskManager` [\#1668](https://github.com/apache/arrow-datafusion/pull/1668) ([alamb](https://github.com/alamb)) +- feat: Support quarter granularity in date\_trunc function [\#1667](https://github.com/apache/arrow-datafusion/pull/1667) ([ovr](https://github.com/ovr)) +- Fix can not load parquet table form spark in datafusion-cli. [\#1665](https://github.com/apache/arrow-datafusion/pull/1665) ([Ted-Jiang](https://github.com/Ted-Jiang)) +- Make `MemoryManager` and `MemoryStream` public [\#1664](https://github.com/apache/arrow-datafusion/pull/1664) ([yjshen](https://github.com/yjshen)) +- \[Cleanup\] Move `AggregatedMetricsSet` to `metrics` for further reuse [\#1663](https://github.com/apache/arrow-datafusion/pull/1663) ([yjshen](https://github.com/yjshen)) +- fix: substr - correct behaivour with negative start pos [\#1660](https://github.com/apache/arrow-datafusion/pull/1660) ([ovr](https://github.com/ovr)) +- suppport bitwise and as an example [\#1653](https://github.com/apache/arrow-datafusion/pull/1653) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([liukun4515](https://github.com/liukun4515)) +- refine match pattern related code [\#1650](https://github.com/apache/arrow-datafusion/pull/1650) ([xudong963](https://github.com/xudong963)) +- update md-5, sha2, blake2 [\#1647](https://github.com/apache/arrow-datafusion/pull/1647) ([xudong963](https://github.com/xudong963)) +- Add `DataFusionError` -\> `ArrowError` conversion [\#1643](https://github.com/apache/arrow-datafusion/pull/1643) ([alamb](https://github.com/alamb)) +- Add `spill_count` and `spilled_bytes` to `BaselineMetrics`, test sort with spill [\#1641](https://github.com/apache/arrow-datafusion/pull/1641) ([yjshen](https://github.com/yjshen)) +- support hash decimal array and group by [\#1640](https://github.com/apache/arrow-datafusion/pull/1640) ([liukun4515](https://github.com/liukun4515)) +- Consolidate Schema and RecordBatch projection [\#1638](https://github.com/apache/arrow-datafusion/pull/1638) ([alamb](https://github.com/alamb)) +- Update hashbrown requirement from 0.11 to 0.12 [\#1631](https://github.com/apache/arrow-datafusion/pull/1631) ([dependabot[bot]](https://github.com/apps/dependabot)) +- Update pyo3 requirement from 0.14 to 0.15 [\#1627](https://github.com/apache/arrow-datafusion/pull/1627) ([dependabot[bot]](https://github.com/apps/dependabot)) +- Optimize `SortPreservingMergeStream` to avoid `SortKeyCursor` sharing [\#1624](https://github.com/apache/arrow-datafusion/pull/1624) ([yjshen](https://github.com/yjshen)) +- Handle merging of evolved schemas in ParquetExec [\#1622](https://github.com/apache/arrow-datafusion/pull/1622) ([thinkharderdev](https://github.com/thinkharderdev)) +- feat: Support Substring\(str \[from int\] \[for int\]\) [\#1621](https://github.com/apache/arrow-datafusion/pull/1621) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([ovr](https://github.com/ovr)) +- feat: Support complex interval via IntervalMonthDayNano [\#1615](https://github.com/apache/arrow-datafusion/pull/1615) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([ovr](https://github.com/ovr)) +- consolidate binary\_expr coercion rule code into `binary_rule.rs` module [\#1607](https://github.com/apache/arrow-datafusion/pull/1607) ([alamb](https://github.com/alamb)) +- Fix comparison of dictionary arrays [\#1606](https://github.com/apache/arrow-datafusion/pull/1606) ([alamb](https://github.com/alamb)) +- add test for decimal to decimal [\#1603](https://github.com/apache/arrow-datafusion/pull/1603) ([liukun4515](https://github.com/liukun4515)) +- update nightly version [\#1597](https://github.com/apache/arrow-datafusion/pull/1597) ([Jimexist](https://github.com/Jimexist)) +- Consolidate sort and external\_sort [\#1596](https://github.com/apache/arrow-datafusion/pull/1596) ([yjshen](https://github.com/yjshen)) +- support from\_slice for binary, string, and boolean array types [\#1589](https://github.com/apache/arrow-datafusion/pull/1589) ([Jimexist](https://github.com/Jimexist)) +- add from\_slice trait to ease arrow2 migration [\#1588](https://github.com/apache/arrow-datafusion/pull/1588) ([Jimexist](https://github.com/Jimexist)) +- Implement ARRAY\_AGG\(DISTINCT ...\) [\#1579](https://github.com/apache/arrow-datafusion/pull/1579) ([james727](https://github.com/james727)) +- Rename sql integration tests from `mod` to `sql_integration` [\#1575](https://github.com/apache/arrow-datafusion/pull/1575) ([alamb](https://github.com/alamb)) +- minor: improve the benchmark readme [\#1567](https://github.com/apache/arrow-datafusion/pull/1567) ([xudong963](https://github.com/xudong963)) +- Consolidate `batch_size` configuration in `ExecutionConfig`, `RuntimeConfig` and `PhysicalPlanConfig` [\#1562](https://github.com/apache/arrow-datafusion/pull/1562) ([yjshen](https://github.com/yjshen)) +- Update to rust 1.58 [\#1557](https://github.com/apache/arrow-datafusion/pull/1557) ([xudong963](https://github.com/xudong963)) +- support mathematics operation for decimal data type [\#1554](https://github.com/apache/arrow-datafusion/pull/1554) ([liukun4515](https://github.com/liukun4515)) +- Address clippy warnings [\#1553](https://github.com/apache/arrow-datafusion/pull/1553) ([sergey-melnychuk](https://github.com/sergey-melnychuk)) +- enhance arithmetic operation for array with scalar [\#1552](https://github.com/apache/arrow-datafusion/pull/1552) ([liukun4515](https://github.com/liukun4515)) +- Remove unused `update` and `merge` implementations from Aggregates and supporting `ScalarValue` arithmetic [\#1550](https://github.com/apache/arrow-datafusion/pull/1550) ([alamb](https://github.com/alamb)) +- Add batch operations to stddev [\#1547](https://github.com/apache/arrow-datafusion/pull/1547) ([realno](https://github.com/realno)) +- Mark ARRAY\_AGG\(DISTINCT ...\) not implemented [\#1534](https://github.com/apache/arrow-datafusion/pull/1534) ([james727](https://github.com/james727)) +- Update to arrow-7.0.0 [\#1523](https://github.com/apache/arrow-datafusion/pull/1523) ([alamb](https://github.com/alamb)) +- Fix ORDER BY on aggregate [\#1506](https://github.com/apache/arrow-datafusion/pull/1506) ([viirya](https://github.com/viirya)) +- Add example on how to query multiple parquet files [\#1497](https://github.com/apache/arrow-datafusion/pull/1497) ([nitisht](https://github.com/nitisht)) +- Refactor testing modules [\#1491](https://github.com/apache/arrow-datafusion/pull/1491) ([hntd187](https://github.com/hntd187)) +- add rfcs for datafusion [\#1490](https://github.com/apache/arrow-datafusion/pull/1490) ([xudong963](https://github.com/xudong963)) +- support comparison for decimal data type and refactor the binary coercion rule [\#1483](https://github.com/apache/arrow-datafusion/pull/1483) ([liukun4515](https://github.com/liukun4515)) +- Minor: Rename `predicate_builder` --\> `pruning_predicate` for consistency [\#1481](https://github.com/apache/arrow-datafusion/pull/1481) ([alamb](https://github.com/alamb)) +- Tests for support try\_cast/cast decimal to numeric [\#1465](https://github.com/apache/arrow-datafusion/pull/1465) ([liukun4515](https://github.com/liukun4515)) +- Avoid send empty batches for Hash partitioning. [\#1459](https://github.com/apache/arrow-datafusion/pull/1459) ([Ted-Jiang](https://github.com/Ted-Jiang)) +- Planner code cleanup [\#1450](https://github.com/apache/arrow-datafusion/pull/1450) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([alamb](https://github.com/alamb)) +- Fix bug in projection: "column types must match schema types, expected XXX but found YYY" [\#1448](https://github.com/apache/arrow-datafusion/pull/1448) ([alamb](https://github.com/alamb)) +- Update arrow-rs to 6.4.0 and replace boolean comparison in datafusion with arrow compute kernel [\#1446](https://github.com/apache/arrow-datafusion/pull/1446) ([xudong963](https://github.com/xudong963)) +- support cast/try\_cast for decimal: signed numeric to decimal [\#1442](https://github.com/apache/arrow-datafusion/pull/1442) ([liukun4515](https://github.com/liukun4515)) +- Consolidate decimal error checking and improve error messages [\#1438](https://github.com/apache/arrow-datafusion/pull/1438) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([alamb](https://github.com/alamb)) +- use 0.13 sql parser [\#1435](https://github.com/apache/arrow-datafusion/pull/1435) ([Jimexist](https://github.com/Jimexist)) +- Minor Code cleanups [\#1428](https://github.com/apache/arrow-datafusion/pull/1428) ([alamb](https://github.com/alamb)) +- Clarify communication on bi-weekly sync [\#1427](https://github.com/apache/arrow-datafusion/pull/1427) ([alamb](https://github.com/alamb)) +- support sum/avg agg for decimal, change sum\(float32\) --\> float64 [\#1408](https://github.com/apache/arrow-datafusion/pull/1408) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([liukun4515](https://github.com/liukun4515)) +- Fix bugs with nullability during rewrites: Combine `simplify` and `Simplifier` [\#1401](https://github.com/apache/arrow-datafusion/pull/1401) ([alamb](https://github.com/alamb)) +- Minimize features [\#1399](https://github.com/apache/arrow-datafusion/pull/1399) ([carols10cents](https://github.com/carols10cents)) +- Update rust vesion to 1.57 [\#1395](https://github.com/apache/arrow-datafusion/pull/1395) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([xudong963](https://github.com/xudong963)) +- support decimal scalar value [\#1394](https://github.com/apache/arrow-datafusion/pull/1394) ([liukun4515](https://github.com/liukun4515)) +- Add coercion rules for AggregateFunctions [\#1387](https://github.com/apache/arrow-datafusion/pull/1387) ([liukun4515](https://github.com/liukun4515)) +- upgrade the arrow-rs version [\#1385](https://github.com/apache/arrow-datafusion/pull/1385) ([liukun4515](https://github.com/liukun4515)) +- add array agg name [\#1382](https://github.com/apache/arrow-datafusion/pull/1382) ([liukun4515](https://github.com/liukun4515)) +- Make tests for `simplify` and `Simplifer` consistent [\#1376](https://github.com/apache/arrow-datafusion/pull/1376) ([alamb](https://github.com/alamb)) +- Refactor: Consolidate expression simplification code in `simplify_expression.rs` [\#1374](https://github.com/apache/arrow-datafusion/pull/1374) ([alamb](https://github.com/alamb)) +- remove unused code in hash\_aggregate [\#1370](https://github.com/apache/arrow-datafusion/pull/1370) ([ic4y](https://github.com/ic4y)) +- Use `BufReader` for LocalFileReader to revert performance regression in parquet reading [\#1366](https://github.com/apache/arrow-datafusion/pull/1366) ([Dandandan](https://github.com/Dandandan)) +- Add unit test for constant folding on values [\#1355](https://github.com/apache/arrow-datafusion/pull/1355) ([viirya](https://github.com/viirya)) +- Extract logical plan: rename the plan name \(follow up\) [\#1354](https://github.com/apache/arrow-datafusion/pull/1354) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([liukun4515](https://github.com/liukun4515)) +- Moved aggr\_test\_schema to test\_utils [\#1338](https://github.com/apache/arrow-datafusion/pull/1338) ([rdettai](https://github.com/rdettai)) +- upgrade arrow-rs to 6.2.0 [\#1334](https://github.com/apache/arrow-datafusion/pull/1334) ([liukun4515](https://github.com/liukun4515)) +- Update release instructions [\#1331](https://github.com/apache/arrow-datafusion/pull/1331) ([alamb](https://github.com/alamb)) +- \#1268: allow datafusion-cli to toggle quiet flag within CLI [\#1330](https://github.com/apache/arrow-datafusion/pull/1330) ([jgoday](https://github.com/jgoday)) +- Extract Aggregate, Sort, and Join to struct from AggregatePlan [\#1326](https://github.com/apache/arrow-datafusion/pull/1326) ([matthewmturner](https://github.com/matthewmturner)) +- Extract `EmptyRelation`, `Limit`, `Values` from `LogicalPlan` [\#1325](https://github.com/apache/arrow-datafusion/pull/1325) ([liukun4515](https://github.com/liukun4515)) +- Extract CrossJoin, Repartition, Union in LogicalPlan [\#1322](https://github.com/apache/arrow-datafusion/pull/1322) ([liukun4515](https://github.com/liukun4515)) +- Fifth batch of updating sql tests to use assert\_batches\_eq [\#1318](https://github.com/apache/arrow-datafusion/pull/1318) ([matthewmturner](https://github.com/matthewmturner)) +- Extract Explain, Analyze, Extension in LogicalPlan as independent struct [\#1317](https://github.com/apache/arrow-datafusion/pull/1317) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([xudong963](https://github.com/xudong963)) +- Extract CreateMemoryTable, DropTable, CreateExternalTable in LogicalPlan as independent struct [\#1311](https://github.com/apache/arrow-datafusion/pull/1311) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([liukun4515](https://github.com/liukun4515)) +- Extract Projection, Filter, Window in LogicalPlan as independent struct [\#1309](https://github.com/apache/arrow-datafusion/pull/1309) ([ic4y](https://github.com/ic4y)) +- Add PSQL comparison tests for except, intersect [\#1292](https://github.com/apache/arrow-datafusion/pull/1292) ([mrob95](https://github.com/mrob95)) +- Extract logical plans in LogicalPlan as independent struct: TableScan [\#1290](https://github.com/apache/arrow-datafusion/pull/1290) ([xudong963](https://github.com/xudong963)) +- Add statement helper command to cli [\#1285](https://github.com/apache/arrow-datafusion/pull/1285) ([matthewmturner](https://github.com/matthewmturner)) +- Python bindings for window functions [\#819](https://github.com/apache/arrow-datafusion/pull/819) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([jgoday](https://github.com/jgoday)) + ## [6.0.0](https://github.com/apache/arrow-datafusion/tree/6.0.0) (2021-11-13) [Full Changelog](https://github.com/apache/arrow-datafusion/compare/5.0.0...6.0.0) diff --git a/dev/release/README.md b/dev/release/README.md index 7930e2bc2d5e..86d36a0c0b7a 100644 --- a/dev/release/README.md +++ b/dev/release/README.md @@ -21,7 +21,7 @@ ## Sub-projects -The Datafusion repo contains 3 different releasable sub-projects: Datafusion, Ballista and Datafusion python binding. +The Datafusion repo contains 2 different releasable sub-projects: Datafusion, Ballista We use Datafusion release to drive the release for the other sub-projects. As a result, Datafusion version bump is required for every release while version @@ -64,7 +64,7 @@ The CHANGELOG is managed automatically using This script creates a changelog using github PRs and issues based on the labels associated with them. -## Prepare release comimts and PR +## Prepare release commits and PR Prepare a PR to update `CHANGELOG.md` and versions to reflect the planned release. diff --git a/dev/release/update_change_log-all.sh b/dev/release/update_change_log-all.sh index c5639cc23b70..0d4cf0071a27 100755 --- a/dev/release/update_change_log-all.sh +++ b/dev/release/update_change_log-all.sh @@ -28,4 +28,3 @@ SOURCE_TOP_DIR="$(cd "${SOURCE_DIR}/../../" && pwd)" ${SOURCE_DIR}/update_change_log-datafusion.sh ${SOURCE_DIR}/update_change_log-ballista.sh -${SOURCE_DIR}/update_change_log-python.sh diff --git a/dev/release/update_change_log-python.sh b/dev/release/update_change_log-python.sh deleted file mode 100755 index 6d428e808d9b..000000000000 --- a/dev/release/update_change_log-python.sh +++ /dev/null @@ -1,32 +0,0 @@ -#!/bin/bash -# -# Licensed to the Apache Software Foundation (ASF) under one -# or more contributor license agreements. See the NOTICE file -# distributed with this work for additional information -# regarding copyright ownership. The ASF licenses this file -# to you under the Apache License, Version 2.0 (the -# "License"); you may not use this file except in compliance -# with the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, -# software distributed under the License is distributed on an -# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -# KIND, either express or implied. See the License for the -# specific language governing permissions and limitations -# under the License. -# - -# Usage: -# CHANGELOG_GITHUB_TOKEN= ./update_change_log-python.sh - -SOURCE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -SOURCE_TOP_DIR="$(cd "${SOURCE_DIR}/../../" && pwd)" - -CURRENT_VER=$(grep version "${SOURCE_TOP_DIR}/python/Cargo.toml" | head -n 1 | awk '{print $3}' | tr -d '"') -${SOURCE_DIR}/update_change_log.sh \ - python \ - python-0.3.0 \ - --exclude-tags-regex "ballista-.+" \ - --future-release "python-${CURRENT_VER}" diff --git a/dev/release/update_change_log.sh b/dev/release/update_change_log.sh index 1d1570d68a62..07ba6f9794c0 100755 --- a/dev/release/update_change_log.sh +++ b/dev/release/update_change_log.sh @@ -50,14 +50,14 @@ OUTPUT_PATH="${PROJECT}/CHANGELOG.md" pushd ${SOURCE_TOP_DIR} # reset content in changelog -git co "${SINCE_TAG}" "${OUTPUT_PATH}" +git checkout "${SINCE_TAG}" "${OUTPUT_PATH}" # remove license header so github-changelog-generator has a clean base to append -sed -i '1,18d' "${OUTPUT_PATH}" +sed -i.bak '1,18d' "${OUTPUT_PATH}" docker run -it --rm \ -e CHANGELOG_GITHUB_TOKEN=$CHANGELOG_GITHUB_TOKEN \ -v "$(pwd)":/usr/local/src/your-app \ - githubchangeloggenerator/github-changelog-generator:1.16.2 \ + githubchangeloggenerator/github-changelog-generator \ --user apache \ --project arrow-datafusion \ --since-tag "${SINCE_TAG}" \ @@ -66,7 +66,7 @@ docker run -it --rm \ --output "${OUTPUT_PATH}" \ "$@" -sed -i "s/\\\n/\n\n/" "${OUTPUT_PATH}" +sed -i.bak "s/\\\n/\n\n/" "${OUTPUT_PATH}" echo '