Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: speed up win64 test #8728

Merged
merged 41 commits into from
Jan 6, 2024
Merged
Changes from 40 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
4748977
Change back to rust 1.74.1 for testing
Jefffrey Jan 3, 2024
18908d5
Run only tpcds planning for faster feedback
Jefffrey Jan 3, 2024
c70faae
test
Jefffrey Jan 3, 2024
2b006c5
test
Jefffrey Jan 3, 2024
a358c5a
test
Jefffrey Jan 3, 2024
f17e064
test
Jefffrey Jan 3, 2024
efe74c6
test
Jefffrey Jan 3, 2024
9efc1d5
test
Jefffrey Jan 3, 2024
5931a34
test
Jefffrey Jan 3, 2024
3da2aa5
test
Jefffrey Jan 3, 2024
48d62a1
test
Jefffrey Jan 3, 2024
b213b55
test
Jefffrey Jan 3, 2024
b8b445a
test
Jefffrey Jan 3, 2024
ef96467
test
Jefffrey Jan 3, 2024
632081e
test
Jefffrey Jan 3, 2024
bfb94bc
test
Jefffrey Jan 3, 2024
8926c54
test
Jefffrey Jan 3, 2024
c85e67c
test
Jefffrey Jan 3, 2024
2e99e21
bad
Jefffrey Jan 3, 2024
f67bfce
isolate test
Jefffrey Jan 3, 2024
b7438f0
Merge branch 'main' into test_win_ci
Jefffrey Jan 3, 2024
4ef39d0
isolate test bad
Jefffrey Jan 3, 2024
9ea64d4
isolate test bad narrow to physical q44
Jefffrey Jan 3, 2024
b36c204
isolate test good narrow to physical q44
Jefffrey Jan 3, 2024
b1f125b
physical q44 bad no backtrace
Jefffrey Jan 3, 2024
c55a09c
Trigger
Jefffrey Jan 3, 2024
5e1fcdd
physical q44 good no backtrace
Jefffrey Jan 3, 2024
7ed65d2
physical q44 bad with backtrace with stdout
Jefffrey Jan 4, 2024
e29e092
physical q44 1.76.0 with backtrace
Jefffrey Jan 4, 2024
f6d493e
physical q44 latest nightly with backtrace
Jefffrey Jan 4, 2024
0c7a6f8
physical q44 bad with backtrace with stdout
Jefffrey Jan 4, 2024
ccf3949
Merge branch 'main' into test_win_ci
Jefffrey Jan 6, 2024
951063b
try with debuginfo=line-tables-only
Jefffrey Jan 6, 2024
b93d7fc
Reenable other checks
Jefffrey Jan 6, 2024
e196b5b
Revert win CI check to running all tests
Jefffrey Jan 6, 2024
c928ed6
Disable coverage CI
Jefffrey Jan 6, 2024
a50b089
Change only debuginfo
Jefffrey Jan 6, 2024
1ddcd05
Commens
Jefffrey Jan 6, 2024
3770594
win remove other rustflag optimizations
Jefffrey Jan 6, 2024
9afd349
Fix comments
Jefffrey Jan 6, 2024
d24f851
Trigger
Jefffrey Jan 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 4 additions & 5 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -310,11 +310,10 @@ jobs:
cd datafusion-cli
cargo test --lib --tests --bins --all-features
env:
# do not produce debug symbols to keep memory usage down
# use higher optimization level to overcome Windows rust slowness for tpc-ds
# and speed builds: https://github.com/apache/arrow-datafusion/issues/8696
# Cargo profile docs https://doc.rust-lang.org/cargo/reference/profiles.html?profile-settings#profile-settings
RUSTFLAGS: "-C debuginfo=0 -C opt-level=1 -C target-feature=+crt-static -C incremental=false -C codegen-units=256"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing the other optimization flags as it caused compilation to take 27 min

Without them, it takes 8 min

# Minimize producing debug symbols to keep memory usage down
# Set debuginfo=line-tables-only as debuginfo=0 causes immensely slow build
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 thank you

# See for more details: https://github.com/rust-lang/rust/issues/119560
RUSTFLAGS: "-C debuginfo=line-tables-only"
RUST_BACKTRACE: "1"
# avoid rust stack overflows on tpc-ds tests
RUST_MINSTACK: "3000000"
Expand Down
Loading