ARROW-15683: [Website] [DataFusion] DataFusion 7.0.0 blog post #193

matthewmturner · 2022-02-14T15:41:47Z

based on crowd sourced content https://docs.google.com/document/d/17uB1GIN58xOehQP5XpJH8J3qA7KDS1WQJzyHzW7L1kU/edit?usp=sharing

github-actions · 2022-02-14T15:42:06Z

Thanks for opening a pull request!

Could you open an issue for this pull request on JIRA?
https://issues.apache.org/jira/browse/ARROW

Then could you also rename pull request title in the following format?

ARROW-${JIRA_ID}: [${COMPONENT}] ${SUMMARY}

See also:

matthewmturner · 2022-02-14T15:42:38Z

FYI @alamb @xudong963

alamb · 2022-02-14T15:59:55Z

Thanks @matthewmturner ! I'll plan to take a look at this carefully this afternoon

github-actions · 2022-02-14T21:49:07Z

https://issues.apache.org/jira/browse/ARROW-15683

alamb

Thank you @matthewmturner this is looking good -- I'll also send a note to the mailing list / slack channel highlighting this as well

_posts/2022-02-14-datafusion-7.0.0.md

xudong963

Thanks @matthewmturner

xudong963 · 2022-02-15T00:46:56Z

_posts/2022-02-14-datafusion-7.0.0.md

@@ -0,0 +1,154 @@
+---
+layout: post
+title: Apache Arrow DataFusion 6.0.0 Release


Suggested change

title: Apache Arrow DataFusion 6.0.0 Release

title: Apache Arrow DataFusion 7.0.0 Release

that is a great catch. 🦅 👁️ 👍

xudong963 · 2022-02-15T00:47:53Z

_posts/2022-02-14-datafusion-7.0.0.md

+
+DataFusion's  SQL, `DataFrame`, and manual `PlanBuilder` API let users access a sophisticated query optimizer and execution engine capable of fast, resource efficient, and parallel execution that takes optimal advantage of todays multicore hardware. Being written in Rust means DataFusion can offer *both* the safety of dynamic languages as well as the resource efficiency of a compiled language.
+
+The Apache Arrow team is pleased to announce the DataFusion 7.0.0 release. This covers 4 months of development work


xudong963 · 2022-02-15T00:51:09Z

_posts/2022-02-14-datafusion-7.0.0.md

+```
+    44  Andrew Lamb
+    24  Kun Liu
+    23  Jiayu Liu
+    12  xudong.w
+    11  Yijie Shen
+     9  Matthew Turner
+     7  Liang-Chi Hsieh
+     5  Lin Ma
+     5  Carlos
+     4  Stephen Carman
+     4  James Katz
+     4  Dmitry Patsura
+     4  QP Hou
+     3  dependabot[bot]
+     3  Remzi Yang
+     3  Yang
+     3  ic4y
+     3  Daniël Heres
+     2  Andy Grove
+     2  Raphael Taylor-Davies
+     2  Jason Tianyi Wang
+     2  Dan Harris
+     2  Sergey Melnychuk
+     1  Nitish Tiwari
+     1  Dom
+     1  Eduard Karacharov
+     1  Javier Goday
+     1  Boaz
+     1  Marko Mikulicic
+     1  Max Burke
+     1  Carol (Nichols || Goulding)
+     1  Phillip Cloud
+     1  Rich
+     1  Toby Hede
+     1  Will Jones
+     1  r.4ntix
+     1  rdettai
+```


Carlos and xudong.w both are me, lol

Suggested change

```

44 Andrew Lamb

24 Kun Liu

23 Jiayu Liu

12 xudong.w

11 Yijie Shen

9 Matthew Turner

7 Liang-Chi Hsieh

5 Lin Ma

5 Carlos

4 Stephen Carman

4 James Katz

4 Dmitry Patsura

4 QP Hou

3 dependabot[bot]

3 Remzi Yang

3 Yang

3 ic4y

3 Daniël Heres

2 Andy Grove

2 Raphael Taylor-Davies

2 Jason Tianyi Wang

2 Dan Harris

2 Sergey Melnychuk

1 Nitish Tiwari

1 Dom

1 Eduard Karacharov

1 Javier Goday

1 Boaz

1 Marko Mikulicic

1 Max Burke

1 Carol (Nichols || Goulding)

1 Phillip Cloud

1 Rich

1 Toby Hede

1 Will Jones

1 r.4ntix

1 rdettai

```

44 Andrew Lamb

24 Kun Liu

23 Jiayu Liu

17 xudong.w

11 Yijie Shen

9 Matthew Turner

7 Liang-Chi Hsieh

5 Lin Ma

4 Stephen Carman

4 James Katz

4 Dmitry Patsura

4 QP Hou

3 dependabot[bot]

3 Remzi Yang

3 Yang

3 ic4y

3 Daniël Heres

2 Andy Grove

2 Raphael Taylor-Davies

2 Jason Tianyi Wang

2 Dan Harris

2 Sergey Melnychuk

1 Nitish Tiwari

1 Dom

1 Eduard Karacharov

1 Javier Goday

1 Boaz

1 Marko Mikulicic

1 Max Burke

1 Carol (Nichols || Goulding)

1 Phillip Cloud

1 Rich

1 Toby Hede

1 Will Jones

1 r.4ntix

1 rdettai

Carlos and xudong.w both are me, lol

sry, meant to update that. thx for catching.

xudong963 · 2022-02-15T00:52:24Z

_posts/2022-02-14-datafusion-7.0.0.md

+      - [DataFusion-ObjectStore-s3](https://github.com/datafusion-contrib/datafusion-objectstore-s3)
+    - New Features
+      - [DataFusion-Streams](https://github.com/datafusion-contrib/datafusion-streams)
+- [Arrow2](https://github.com/jorgecarleitao/arrow2)


liukun4515 · 2022-02-15T05:51:00Z

_posts/2022-02-14-datafusion-7.0.0.md

+     1  rdettai
+```
+
+The following section highlights some of the improvements in this release. Of course, many other bug fixes and improvements have also been made and we refer you to the complete [changelog](https://github.com/apache/arrow-datafusion/blob/7.0.0/datafusion/CHANGELOG.md) for the full detail.


The link of the changelog is invalid.

I thought it would be valid once officially published - @alamb can you confirm?

That is correct -- I will create a 7.0.0 tag in the datafusion-repo once it has been published

Right now, you can preview the log here:
https://github.com/apache/arrow-datafusion/blob/7.0.0-rc2/datafusion/CHANGELOG.md

https://github.com/apache/arrow-datafusion/releases/tag/7.0.0 is now available, so the change log link now works:

https://github.com/apache/arrow-datafusion/blob/7.0.0/datafusion/CHANGELOG.md

alamb · 2022-02-17T14:03:10Z

_posts/2022-02-14-datafusion-7.0.0.md

+---
+layout: post
+title: Apache Arrow DataFusion 7.0.0 Release
+date: "2022-02-14 00:00:00"


Suggested change

date: "2022-02-14 00:00:00"

date: "2022-02-17 00:00:00"

I plan to release 7.0.0 in about 1 hour (after the required 72 hour voting time period has elapsed)

Updating now

houqp

LGTM, thanks @matthewmturner !

houqp · 2022-02-18T05:12:52Z

_posts/2022-02-17-datafusion-7.0.0.md

+  - Arrow’s dyn scalar kernels are now used to enable efficient operations on `DictionaryArray`s [#1685](https://github.com/apache/arrow-datafusion/pull/1685)
+  - Switch from `std::sync::Mutex` to `parking_lot::Mutex` [#1720](https://github.com/apache/arrow-datafusion/pull/1720)
+- New Features
+  - Better support for limiting resource usage


Perhaps rephrase this as the following since it's a pretty major feature:

Suggested change

- Better support for limiting resource usage

- Support for memory tracking and spilling to disk

liukun4515

LGTM
thanks @matthewmturner

alamb · 2022-02-27T20:58:59Z

Shall we publish this one now?

matthewmturner · 2022-02-28T00:12:31Z

Let's do it!

_posts/2022-02-17-datafusion-7.0.0.md

alamb · 2022-02-28T14:18:41Z

🚀

kou · 2022-02-28T21:11:12Z

Could you also update date in path?
("2022-02-17" in https://github.com/apache/arrow-site/blob/master/_posts/2022-02-17-datafusion-7.0.0.md )

alamb · 2022-02-28T21:30:13Z

Could you also update date in path?

Sorry about that. PR in #196

@kou

…lished date As suggested by @kou in #193 (comment) Closes #196 from alamb/alamb/fix_date: Authored-by: Andrew Lamb <andrew@nerdnetworks.org> Signed-off-by: Sutou Kouhei <kou@clear-code.com>

v1 blog

a066409

Fix intro

34b6795

alamb changed the title ~~DataFusion 7.0.0 blog post~~ ARROW-15683: [Website] [DataFusion] DataFusion 7.0.0 blog post Feb 14, 2022

alamb reviewed Feb 14, 2022

View reviewed changes

Update for feedback

4e585e4

xudong963 reviewed Feb 15, 2022

View reviewed changes

More feedback

ed98d6b

liukun4515 reviewed Feb 15, 2022

View reviewed changes

Parquet schema evolution comment

6d9df52

alamb reviewed Feb 17, 2022

View reviewed changes

Update date

4ac95f5

alamb mentioned this pull request Feb 17, 2022

update datafusion site to get latest example #195

Merged

houqp approved these changes Feb 18, 2022

View reviewed changes

Update wording

b1baf55

liukun4515 approved these changes Feb 20, 2022

View reviewed changes

alamb reviewed Feb 28, 2022

View reviewed changes

_posts/2022-02-17-datafusion-7.0.0.md Outdated Show resolved Hide resolved

Update _posts/2022-02-17-datafusion-7.0.0.md

ee52d1c

alamb merged commit cc0b0b8 into apache:master Feb 28, 2022

alamb mentioned this pull request Feb 28, 2022

ARROW-15683: [Website] [DataFusion] Fix name of post to match its published date #196

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ARROW-15683: [Website] [DataFusion] DataFusion 7.0.0 blog post #193

ARROW-15683: [Website] [DataFusion] DataFusion 7.0.0 blog post #193

matthewmturner commented Feb 14, 2022 •

edited by alamb

Loading

github-actions bot commented Feb 14, 2022

matthewmturner commented Feb 14, 2022

alamb commented Feb 14, 2022

github-actions bot commented Feb 14, 2022

alamb left a comment

xudong963 left a comment

xudong963 Feb 15, 2022

alamb Feb 15, 2022

xudong963 Feb 15, 2022

xudong963 Feb 15, 2022

matthewmturner Feb 15, 2022

xudong963 Feb 15, 2022

liukun4515 Feb 15, 2022

matthewmturner Feb 15, 2022

alamb Feb 15, 2022

alamb Feb 17, 2022

alamb Feb 17, 2022

matthewmturner Feb 17, 2022

houqp left a comment

houqp Feb 18, 2022

matthewmturner Feb 18, 2022

liukun4515 left a comment

alamb commented Feb 27, 2022

matthewmturner commented Feb 28, 2022

alamb commented Feb 28, 2022

kou commented Feb 28, 2022

alamb commented Feb 28, 2022 •

edited

Loading

	title: Apache Arrow DataFusion 6.0.0 Release
	title: Apache Arrow DataFusion 7.0.0 Release


		DataFusion's SQL, `DataFrame`, and manual `PlanBuilder` API let users access a sophisticated query optimizer and execution engine capable of fast, resource efficient, and parallel execution that takes optimal advantage of todays multicore hardware. Being written in Rust means DataFusion can offer both the safety of dynamic languages as well as the resource efficiency of a compiled language.

		The Apache Arrow team is pleased to announce the DataFusion 7.0.0 release. This covers 4 months of development work

	- Better support for limiting resource usage
	- Support for memory tracking and spilling to disk

ARROW-15683: [Website] [DataFusion] DataFusion 7.0.0 blog post #193

ARROW-15683: [Website] [DataFusion] DataFusion 7.0.0 blog post #193

Conversation

matthewmturner commented Feb 14, 2022 • edited by alamb Loading

github-actions bot commented Feb 14, 2022

matthewmturner commented Feb 14, 2022

alamb commented Feb 14, 2022

github-actions bot commented Feb 14, 2022

alamb left a comment

Choose a reason for hiding this comment

xudong963 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

houqp left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liukun4515 left a comment

Choose a reason for hiding this comment

alamb commented Feb 27, 2022

matthewmturner commented Feb 28, 2022

alamb commented Feb 28, 2022

kou commented Feb 28, 2022

alamb commented Feb 28, 2022 • edited Loading

matthewmturner commented Feb 14, 2022 •

edited by alamb

Loading

alamb commented Feb 28, 2022 •

edited

Loading