Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run a single huge par_body_owners instead of many small ones after each other. #122140

Merged
merged 3 commits into from
Mar 11, 2024

Conversation

oli-obk
Copy link
Contributor

@oli-obk oli-obk commented Mar 7, 2024

This improves parallel rustc parallelism by avoiding the bottleneck after each individual par_body_owners (because it needs to wait for queries to finish, so if there is one long running one, a lot of cores will be idle while waiting for the single query).

@rustbot
Copy link
Collaborator

rustbot commented Mar 7, 2024

r? @davidtwco

rustbot has assigned @davidtwco.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels Mar 7, 2024
@oli-obk
Copy link
Contributor Author

oli-obk commented Mar 7, 2024

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Mar 7, 2024
@bors
Copy link
Contributor

bors commented Mar 7, 2024

⌛ Trying commit 164473b with merge c0e49b5...

bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 7, 2024
Run a single huge par_body_owners instead of many small ones after each other.

This improves parallel rustc parallelism by avoiding the bottleneck after each individual `par_body_owners` (because it needs to wait for queries to finish, so if there is one long running one, a lot of cores will be idle while waiting for the single query).

based on rust-lang#121500
@bors
Copy link
Contributor

bors commented Mar 7, 2024

☀️ Try build successful - checks-actions
Build commit: c0e49b5 (c0e49b5e16f7b562786f83d7702b40f242942aee)

@rust-timer

This comment has been minimized.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (c0e49b5): comparison URL.

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
0.2% [0.2%, 0.3%] 3
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-0.5% [-0.8%, -0.2%] 5
Improvements ✅
(secondary)
-0.3% [-0.5%, -0.2%] 12
All ❌✅ (primary) -0.2% [-0.8%, 0.3%] 8

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
2.4% [2.4%, 2.4%] 1
Improvements ✅
(primary)
-1.9% [-2.5%, -0.9%] 8
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) -1.9% [-2.5%, -0.9%] 8

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
4.3% [1.1%, 14.8%] 65
Regressions ❌
(secondary)
12.3% [4.8%, 18.6%] 18
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 4.3% [1.1%, 14.8%] 65

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 648.501s -> 652.39s (0.60%)
Artifact size: 172.69 MiB -> 172.63 MiB (-0.03%)

@rustbot rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Mar 7, 2024
@oli-obk
Copy link
Contributor Author

oli-obk commented Mar 8, 2024

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Mar 8, 2024
@bors
Copy link
Contributor

bors commented Mar 8, 2024

⌛ Trying commit c531269 with merge 702bd6d...

bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 8, 2024
Run a single huge par_body_owners instead of many small ones after each other.

This improves parallel rustc parallelism by avoiding the bottleneck after each individual `par_body_owners` (because it needs to wait for queries to finish, so if there is one long running one, a lot of cores will be idle while waiting for the single query).

based on rust-lang#121500
@bors
Copy link
Contributor

bors commented Mar 8, 2024

☀️ Try build successful - checks-actions
Build commit: 702bd6d (702bd6de43d33c2a791c1a2f6acccd68f55d3afd)

@rust-timer

This comment has been minimized.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (702bd6d): comparison URL.

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
0.3% [0.3%, 0.3%] 2
Regressions ❌
(secondary)
1.7% [1.7%, 1.7%] 1
Improvements ✅
(primary)
-0.3% [-0.3%, -0.3%] 1
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 0.1% [-0.3%, 0.3%] 3

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
1.1% [1.1%, 1.1%] 1
Regressions ❌
(secondary)
5.7% [5.1%, 6.2%] 2
Improvements ✅
(primary)
-1.7% [-2.4%, -0.7%] 5
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) -1.2% [-2.4%, 1.1%] 6

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
3.9% [1.1%, 13.7%] 71
Regressions ❌
(secondary)
11.7% [2.9%, 20.6%] 20
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 3.9% [1.1%, 13.7%] 71

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 648.792s -> 653.885s (0.78%)
Artifact size: 172.56 MiB -> 172.55 MiB (-0.01%)

@rustbot rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Mar 8, 2024
…each other.

This improves parallel rustc parallelism by avoiding the bottleneck after each individual `par_body_owners` (because it needs to wait for queries to finish, so if there is one long running one, a lot of cores will be idle while waiting for the single query).
Copy link
Member

@davidtwco davidtwco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r=me if this is ready

@oli-obk
Copy link
Contributor Author

oli-obk commented Mar 11, 2024

@bors r=davidtwco

@bors
Copy link
Contributor

bors commented Mar 11, 2024

📌 Commit 55ea944 has been approved by davidtwco

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 11, 2024
@bors
Copy link
Contributor

bors commented Mar 11, 2024

⌛ Testing commit 55ea944 with merge 65cd843...

@bors
Copy link
Contributor

bors commented Mar 11, 2024

☀️ Test successful - checks-actions
Approved by: davidtwco
Pushing 65cd843 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Mar 11, 2024
@bors bors merged commit 65cd843 into rust-lang:master Mar 11, 2024
12 checks passed
@rustbot rustbot added this to the 1.78.0 milestone Mar 11, 2024
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (65cd843): comparison URL.

Overall result: ❌✅ regressions and improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
1.7% [1.7%, 1.7%] 1
Improvements ✅
(primary)
-0.4% [-0.6%, -0.3%] 2
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) -0.4% [-0.6%, -0.3%] 2

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-1.8% [-2.3%, -0.9%] 4
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) -1.8% [-2.3%, -0.9%] 4

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
4.2% [1.3%, 13.8%] 68
Regressions ❌
(secondary)
12.0% [2.1%, 20.9%] 19
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-5.9% [-9.0%, -3.6%] 8
All ❌✅ (primary) 4.2% [1.3%, 13.8%] 68

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 647.596s -> 652.28s (0.72%)
Artifact size: 310.01 MiB -> 309.94 MiB (-0.02%)

@rustbot rustbot removed the perf-regression Performance regression. label Mar 11, 2024
@Kobzol
Copy link
Contributor

Kobzol commented Mar 11, 2024

Umm, did this help for the parallel frontend (and how much?). Otherwise, this looks like a severe regression for the single-threaded version.

@bjorn3
Copy link
Member

bjorn3 commented Mar 11, 2024

(for those that got confused, the linked perf report is for the cycles count rather than the instructions count)

@Kobzol
Copy link
Contributor

Kobzol commented Mar 11, 2024

Ah yes, sorry, forgot to mention that. The same regression is for walltime.

@oli-obk oli-obk deleted the track_errors13 branch March 11, 2024 19:40
@oli-obk
Copy link
Contributor Author

oli-obk commented Mar 11, 2024

That's super weird. Did I trash CPU caching of the query tables or sth? (Edit: Jup, I did https://perf.rust-lang.org/compare.html?start=d255c6a57c393db6221b1ff700daea478436f1cd&end=65cd843ae06ad00123c131a431ed5304e4cd577a&stat=cache-misses) Before this PR we invoked all queries of the same kind right after each other, now we invoke all queries for the same id together.

Let's revert and revisit.

oli-obk added a commit to oli-obk/rust that referenced this pull request Mar 11, 2024
…vidtwco"

This reverts commit 65cd843, reversing
changes made to d255c6a.
bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 14, 2024
…rrors

Revert "Auto merge of rust-lang#122140 - oli-obk:track_errors13, r=davidtwco"

This reverts commit 65cd843, reversing changes made to d255c6a.

reverts rust-lang#122140

It was a large regression in wall time due to trashing CPU caches
github-actions bot pushed a commit to rust-lang/miri that referenced this pull request Mar 14, 2024
Revert "Auto merge of #122140 - oli-obk:track_errors13, r=davidtwco"

This reverts commit 65cd843ae06ad00123c131a431ed5304e4cd577a, reversing changes made to d255c6a57c393db6221b1ff700daea478436f1cd.

reverts rust-lang/rust#122140

It was a large regression in wall time due to trashing CPU caches
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants