chore(toolchain): bump to 2022-12-12 #6159

BugenZhao · 2022-11-01T17:22:11Z

I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.

What's changed and what's your intention?

Bump the toolchain to 2022-12-12, which should contain the fix to rust-lang/rust#103423. We're now able to revert some workarounds, as explained in #6103.

This is mainly to fix the excessively slow compilation speed under release profile, like https://buildkite.com/risingwavelabs/docker/builds/13181.

Checklist

I have written necessary rustdoc comments
I have added necessary unit tests and integration tests
All checks passed in ./risedev check (or alias, ./risedev c)

Refer to a related PR or issue link (optional)

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

This reverts commit ea98e26.

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

codecov · 2022-11-01T17:57:45Z

Codecov Report

Merging #6159 (074a330) into main (f2975a3) will increase coverage by 0.00%.
The diff coverage is 76.19%.

@@           Coverage Diff           @@
##             main    #6159   +/-   ##
=======================================
  Coverage   73.24%   73.24%           
=======================================
  Files        1032     1031    -1     
  Lines      164648   164587   -61     
=======================================
- Hits       120598   120555   -43     
+ Misses      44050    44032   -18

Flag	Coverage Δ
rust	`73.24% <76.19%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
.../batch/src/task/consistent_hash_shuffle_channel.rs	`0.00% <ø> (ø)`
src/connector/src/source/filesystem/s3/s3_dir.rs	`0.00% <0.00%> (ø)`
src/meta/src/lib.rs	`2.40% <ø> (ø)`
src/risedevtool/src/compose.rs	`0.00% <0.00%> (ø)`
src/risedevtool/src/compose_deploy.rs	`0.00% <0.00%> (ø)`
src/risedevtool/src/task/grafana_service.rs	`0.00% <0.00%> (ø)`
src/risedevtool/src/task/kafka_service.rs	`0.00% <0.00%> (ø)`
src/risedevtool/src/task/prometheus_service.rs	`0.00% <0.00%> (ø)`
src/risedevtool/src/task/zookeeper_service.rs	`0.00% <0.00%> (ø)`
...storage/src/hummock/compactor/compaction_filter.rs	`96.55% <ø> (ø)`
... and 41 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

BugenZhao · 2022-12-08T08:45:51Z

@huangjw806 Could you please help to test whether building the docker image under release profile finishes in a reasonable time, under this branch?

huangjw806

LGTM

huangjw806 · 2022-12-08T08:47:35Z

ou please help to test whether building the docker image under release profile finishes in a reasonable time, under this branch?

Sure.

BugenZhao · 2022-12-08T09:03:38Z

It cannot compile with debug assertions on.

rust-lang/rust#105009

huangjw806 · 2022-12-08T09:32:42Z

@huangjw806 Could you please help to test whether building the docker image under release profile finishes in a reasonable time, under this branch?

I tested the docker build of this branch, it was successful and fast.
https://buildkite.com/risingwavelabs/docker/builds/13184#_

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

BugenZhao · 2022-12-08T10:32:09Z

It seems like it's #6613 that causes the slow compilation before bumping and ICE after bumping.

cc @wenym1 Would you please help to take a look and clean some suspicious sugars or magics in that PR? 🥵

wenym1 · 2022-12-09T04:54:04Z

It seems like it's #6613 that causes the slow compilation before bumping and ICE after bumping.
cc @wenym1 Would you please help to take a look and clean some suspicious sugars or magics in that PR? 🥵

In #6613, we only wrap the previous StateStoreIter with a try_stream. I am not sure whether this will cause the slow compilation. Can you provide me a way to reproduce the problem locally?

BugenZhao · 2022-12-09T05:09:54Z

Can you provide me a way to reproduce the problem locally?

I guess compiling with release profile under Linux can reproduce this, just like what we do when building image on CI.

risingwave/docker/Dockerfile

Line 35 in 353837d

    
           RUN cargo build -p risingwave_cmd -p risingwave_cmd_all --release --features "static-link static-log-level" && \

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

BugenZhao · 2022-12-12T07:21:55Z

rust-lang/rust#105009 has been fixed and I've successfully bumped the toolchain to 12-12. Seems that everything is working well.

cc @wenym1 There's no need to do anything with #6613 now. :)

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

src/storage/src/lib.rs

BugenZhao · 2022-12-12T07:27:30Z

src/storage/src/table/streaming_table/mem_table.rs

@@ -43,7 +43,7 @@ pub enum MemTableError {
    },
 }

-type Result<T> = std::result::Result<T, MemTableError>;
+type Result<T> = std::result::Result<T, Box<MemTableError>>;


This error is also large.

BugenZhao · 2022-12-12T07:27:49Z

src/stream/src/executor/managed_state/join/mod.rs

-            #[for_await]
-            for row_and_degree in zipped_iter {
-                let (row, degree): (Cow<'_, Row>, Cow<'_, Row>) = row_and_degree?;
+            for (row, degree) in table_iter.zip(degree_table_iter) {


cc @yuhao-su Am I correct here?

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

…wave into bz/bump-to-11-01

xxchan · 2022-12-12T09:46:56Z

e2e_test/ch_benchmark/batch/ch_benchmark.slt

@@ -5,9 +5,11 @@ statement ok
 SET QUERY_MODE TO distributed;

 # Create tpch tables
+


Revert this change? (& sqllogctest master preserves newline now 🤪

xxchan · 2022-12-12T09:49:04Z

e2e_test/ch_benchmark/batch/q3.slt.part

@@ -16,21 +16,21 @@ where 	 c_state like 'A%'
 group by ol_o_id, ol_w_id, ol_d_id, o_entry_d
 order by revenue desc, o_entry_d;
 ----
-    2111        1        4       21  2015-11-22 00:00:00


order by revenue desc, o_entry_d seems not enough to be deterministic..

Oops. Will keep rowsort here.

xxchan · 2022-12-12T09:51:19Z

e2e_test/ch_benchmark/batch/q10.slt.part

@@ -10,26 +10,25 @@ where	 c_id = o_c_id
    	 and o_entry_d >= '2007-01-02 00:00:00.000000'
    	 and o_entry_d <= ol_delivery_d
    	 and n_nationkey = ascii(substr(c_state,1,1)) - 48
-group by c_id, c_last, c_city, c_phone, n_name
-order by revenue desc;


Why to remove the order by

I think order by makes no sense under rowsort?

Anyway, I will revert this.

Adding order by can test sort executor produces results.. 🤪🤪🤪

xxchan · 2022-12-12T09:57:32Z

src/frontend/src/expr/type_inference/func.rs

-fn infer_type_name<'a, 'b>(
+fn infer_type_name<'a>(
    sig_map: &'a FuncSigMap,
    func_type: ExprType,
-    inputs: &'b [Option<DataTypeName>],
+    inputs: &[Option<DataTypeName>],


Is this change really correct? 🤔

Oh yes

Each elided lifetime in input position becomes a distinct lifetime parameter.

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

BugenZhao · 2022-12-12T10:26:02Z

ci/Dockerfile

@@ -28,7 +28,7 @@ RUN rustup component add rustfmt llvm-tools-preview clippy

 # install build tools
 RUN cargo install cargo-llvm-cov cargo-nextest cargo-udeps cargo-hakari cargo-sort cargo-make cargo-cache \
-    && cargo install --git https://github.com/risinglightdb/sqllogictest-rs --bin sqllogictest \
+    && cargo install --git https://github.com/risinglightdb/sqllogictest-rs --rev 65b122f --bin sqllogictest \


I'd like to specify the revision to make the bump of sqllogictest more explicit. 😄 It's the lastest revision on main now. cc @xxchan

BugenZhao added 5 commits November 1, 2022 15:57

bump toolchain

00804d2

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

fix clippy

9a4495f

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

use catalog error

ea98e26

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

Revert "use catalog error"

cb5aa4a

This reverts commit ea98e26.

update image version

19fc7ae

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

github-actions bot added the type/chore label Nov 1, 2022

do not use toml

0d5e293

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

BugenZhao added 4 commits December 8, 2022 13:16

bump to 1208

ce3b81d

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

Merge remote-tracking branch 'origin/main' into bz/bump-to-11-01

18ea87a

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

switch to 11-21

445f210

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

fix clippy

39d21df

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

BugenZhao changed the title ~~chore(toolchain): bump to 2022-11-xx~~ chore(toolchain): bump to 2022-11-21 Dec 8, 2022

generate hakari

b47eed0

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

BugenZhao marked this pull request as ready for review December 8, 2022 08:43

BugenZhao requested review from xxchan, wangrunji0408 and huangjw806 December 8, 2022 08:43

huangjw806 approved these changes Dec 8, 2022

View reviewed changes

xxchan approved these changes Dec 8, 2022

View reviewed changes

wangrunji0408 approved these changes Dec 8, 2022

View reviewed changes

bump docker version

9b59c71

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

xxchan mentioned this pull request Dec 9, 2022

compiler bug(?): daily release build takes 2 hours #6835

Closed

BugenZhao added 2 commits December 12, 2022 06:44

fix show

80b655d

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

fix hash zip

3693e6c

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

BugenZhao changed the title ~~chore(toolchain): bump to 2022-11-21~~ chore(toolchain): bump to 2022-12-12 Dec 12, 2022

BugenZhao requested review from yuhao-su and wangrunji0408 December 12, 2022 07:20

fix e2es

c195295

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

BugenZhao commented Dec 12, 2022

View reviewed changes

src/storage/src/lib.rs Outdated Show resolved Hide resolved

BugenZhao commented Dec 12, 2022

View reviewed changes

BugenZhao added 5 commits December 12, 2022 08:41

fix rowsort

59a684b

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

fix simulation

754782f

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

no need for row sort

fc6bf2c

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

fix ch benchmark

6609c93

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

Update src/storage/src/lib.rs

e8b3206

BugenZhao requested a review from xxchan December 12, 2022 09:34

BugenZhao added 3 commits December 12, 2022 09:37

remove order by

cf56b29

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

Merge branch 'bz/bump-to-11-01' of github.com:singularity-data/rising…

4e7db20

…wave into bz/bump-to-11-01

Merge remote-tracking branch 'origin/main' into bz/bump-to-11-01

756fabc

BugenZhao added the mergify/can-merge label Dec 12, 2022

xxchan reviewed Dec 12, 2022

View reviewed changes

BugenZhao removed the mergify/can-merge label Dec 12, 2022

BugenZhao added 3 commits December 12, 2022 10:15

revert and fix batch ch

f50a2df

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

fix streaming ch

9a43978

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

specify revision for sqllogictest

4558d3b

Signed-off-by: Bugen Zhao <i@bugenzhao.com>

BugenZhao commented Dec 12, 2022

View reviewed changes

BugenZhao added the mergify/can-merge label Dec 12, 2022

Merge branch 'main' into bz/bump-to-11-01

074a330

mergify bot merged commit 885f864 into main Dec 12, 2022

mergify bot deleted the bz/bump-to-11-01 branch December 12, 2022 11:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(toolchain): bump to 2022-12-12 #6159

chore(toolchain): bump to 2022-12-12 #6159

BugenZhao commented Nov 1, 2022 •

edited

Loading

codecov bot commented Nov 1, 2022 •

edited

Loading

BugenZhao commented Dec 8, 2022

huangjw806 left a comment

huangjw806 commented Dec 8, 2022

BugenZhao commented Dec 8, 2022 •

edited

Loading

huangjw806 commented Dec 8, 2022

BugenZhao commented Dec 8, 2022

wenym1 commented Dec 9, 2022 •

edited

Loading

BugenZhao commented Dec 9, 2022 •

edited

Loading

BugenZhao commented Dec 12, 2022

BugenZhao Dec 12, 2022

BugenZhao Dec 12, 2022

xxchan Dec 12, 2022

xxchan Dec 12, 2022

BugenZhao Dec 12, 2022

xxchan Dec 12, 2022

BugenZhao Dec 12, 2022

BugenZhao Dec 12, 2022

xxchan Dec 12, 2022

xxchan Dec 12, 2022

xxchan Dec 12, 2022

BugenZhao Dec 12, 2022

xxchan Dec 12, 2022

		@@ -5,9 +5,11 @@ statement ok
		SET QUERY_MODE TO distributed;

		# Create tpch tables

chore(toolchain): bump to 2022-12-12 #6159

chore(toolchain): bump to 2022-12-12 #6159

Conversation

BugenZhao commented Nov 1, 2022 • edited Loading

What's changed and what's your intention?

Checklist

Refer to a related PR or issue link (optional)

codecov bot commented Nov 1, 2022 • edited Loading

Codecov Report

BugenZhao commented Dec 8, 2022

huangjw806 left a comment

Choose a reason for hiding this comment

huangjw806 commented Dec 8, 2022

BugenZhao commented Dec 8, 2022 • edited Loading

huangjw806 commented Dec 8, 2022

BugenZhao commented Dec 8, 2022

wenym1 commented Dec 9, 2022 • edited Loading

BugenZhao commented Dec 9, 2022 • edited Loading

BugenZhao commented Dec 12, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BugenZhao commented Nov 1, 2022 •

edited

Loading

codecov bot commented Nov 1, 2022 •

edited

Loading

BugenZhao commented Dec 8, 2022 •

edited

Loading

wenym1 commented Dec 9, 2022 •

edited

Loading

BugenZhao commented Dec 9, 2022 •

edited

Loading