bootstrap: allow setting `--jobs` in config.toml #131838

jieyouxu · 2024-10-17T15:06:25Z

Allow setting --jobs in config.toml's [build] section.

[build]
jobs = 0

If this is unset or set to zero in config.toml, we look at --jobs flag. If that is also unset, then we fallback to std::thread::available_parallelism. If that is not available, then we default to 1. The flags and available_parallelism fallback are already setup, this PR just adds a config.toml option to wire that up.

Closes #131836.

r? bootstrap

rustbot · 2024-10-17T15:06:37Z

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

This PR modifies config.example.toml.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

jieyouxu · 2024-10-17T16:08:13Z

@rustbot author

jieyouxu · 2024-10-18T03:07:33Z

@rustbot ready

onur-ozkan · 2024-10-18T04:23:26Z

src/bootstrap/src/core/config/config.rs

+        if let Some(jobs) = jobs {
+            if jobs > 0 {
+                config.jobs = Some(jobs);
+            }
+        }


We should handle it here:

rust/src/bootstrap/src/core/config/config.rs

Line 1293 in e057ca5

config.jobs = Some(threads_from_config(flags.jobs as u32));

Note that there's two layers of precedence, the line there is the top priority --jobs flag. I changed the logic to better reflect the precedence such that:

config.jobs and flags.jobs are initially None.

config.jobs is Some based on --jobs=N if N > 0 and still None otherwise.

If config.jobs is still None, then we fallback to --set build.jobs then [build] jobs. Note that the build toml section is read below this.

I added a precedence test to check this.

(Remark: I found this config layering somewhat hard to follow)

src/bootstrap/src/core/config/tests.rs

config.example.toml

src/bootstrap/src/core/config/config.rs

jieyouxu · 2024-10-18T06:22:34Z

@rustbot author

jieyouxu · 2024-10-18T10:32:33Z

@rustbot ready

jieyouxu · 2024-10-19T05:22:37Z

@rustbot ready

onur-ozkan

r=me with/without the nit suggestion

src/bootstrap/src/core/config/config.rs

onur-ozkan · 2024-10-19T05:41:14Z

Thanks!

@bors r+

bors · 2024-10-19T05:41:17Z

📌 Commit 65458ae has been approved by onur-ozkan

It is now in the queue for this repository.

bootstrap: allow setting `--jobs` in config.toml Allow setting `--jobs` in config.toml's `[build]` section. ```toml [build] jobs = 0 ``` If this is unset or set to zero in config.toml, we look at `--jobs` flag. If that is also unset, then we fallback to `std::thread::available_parallelism`. If that is not available, then we default to `1`. The flags and `available_parallelism` fallback are already setup, this PR just adds a config.toml option to wire that up. Closes rust-lang#131836. r? bootstrap

…kingjubilee Rollup of 8 pull requests Successful merges: - rust-lang#127462 (std: uefi: Add basic Env variables) - rust-lang#131537 (Fix range misleading field access) - rust-lang#131838 (bootstrap: allow setting `--jobs` in config.toml) - rust-lang#131871 (x86-32 float return for 'Rust' ABI: treat all float types consistently) - rust-lang#131876 (compiler: Use LLVM's Comdat support) - rust-lang#131890 (Update `use` keyword docs to describe precise capturing) - rust-lang#131899 (Mark unexpected variant res suggestion as having placeholders) - rust-lang#131908 (rustdoc: Switch from FxHash to sha256 for static file hashing.) r? `@ghost` `@rustbot` modify labels: rollup

bootstrap: allow setting `--jobs` in config.toml Allow setting `--jobs` in config.toml's `[build]` section. ```toml [build] jobs = 0 ``` If this is unset or set to zero in config.toml, we look at `--jobs` flag. If that is also unset, then we fallback to `std::thread::available_parallelism`. If that is not available, then we default to `1`. The flags and `available_parallelism` fallback are already setup, this PR just adds a config.toml option to wire that up. Closes rust-lang#131836. r? bootstrap

…iaskrgr Rollup of 10 pull requests Successful merges: - rust-lang#127462 (std: uefi: Add basic Env variables) - rust-lang#131537 (Fix range misleading field access) - rust-lang#131838 (bootstrap: allow setting `--jobs` in config.toml) - rust-lang#131871 (x86-32 float return for 'Rust' ABI: treat all float types consistently) - rust-lang#131876 (compiler: Use LLVM's Comdat support) - rust-lang#131890 (Update `use` keyword docs to describe precise capturing) - rust-lang#131899 (Mark unexpected variant res suggestion as having placeholders) - rust-lang#131908 (rustdoc: Switch from FxHash to sha256 for static file hashing.) - rust-lang#131916 (small interpreter error cleanup) - rust-lang#131919 (zero-sized accesses are fine on null pointers) r? `@ghost` `@rustbot` modify labels: rollup

bootstrap: allow setting `--jobs` in config.toml Allow setting `--jobs` in config.toml's `[build]` section. ```toml [build] jobs = 0 ``` If this is unset or set to zero in config.toml, we look at `--jobs` flag. If that is also unset, then we fallback to `std::thread::available_parallelism`. If that is not available, then we default to `1`. The flags and `available_parallelism` fallback are already setup, this PR just adds a config.toml option to wire that up. Closes rust-lang#131836. r? bootstrap

…iaskrgr Rollup of 9 pull requests Successful merges: - rust-lang#127462 (std: uefi: Add basic Env variables) - rust-lang#131537 (Fix range misleading field access) - rust-lang#131838 (bootstrap: allow setting `--jobs` in config.toml) - rust-lang#131871 (x86-32 float return for 'Rust' ABI: treat all float types consistently) - rust-lang#131890 (Update `use` keyword docs to describe precise capturing) - rust-lang#131899 (Mark unexpected variant res suggestion as having placeholders) - rust-lang#131908 (rustdoc: Switch from FxHash to sha256 for static file hashing.) - rust-lang#131916 (small interpreter error cleanup) - rust-lang#131919 (zero-sized accesses are fine on null pointers) r? `@ghost` `@rustbot` modify labels: rollup

…iaskrgr Rollup of 8 pull requests Successful merges: - rust-lang#127462 (std: uefi: Add basic Env variables) - rust-lang#131537 (Fix range misleading field access) - rust-lang#131838 (bootstrap: allow setting `--jobs` in config.toml) - rust-lang#131890 (Update `use` keyword docs to describe precise capturing) - rust-lang#131899 (Mark unexpected variant res suggestion as having placeholders) - rust-lang#131908 (rustdoc: Switch from FxHash to sha256 for static file hashing.) - rust-lang#131916 (small interpreter error cleanup) - rust-lang#131919 (zero-sized accesses are fine on null pointers) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of rust-lang#131838 - jieyouxu:boopjob, r=onur-ozkan bootstrap: allow setting `--jobs` in config.toml Allow setting `--jobs` in config.toml's `[build]` section. ```toml [build] jobs = 0 ``` If this is unset or set to zero in config.toml, we look at `--jobs` flag. If that is also unset, then we fallback to `std::thread::available_parallelism`. If that is not available, then we default to `1`. The flags and `available_parallelism` fallback are already setup, this PR just adds a config.toml option to wire that up. Closes rust-lang#131836. r? bootstrap

rustbot assigned onur-ozkan Oct 17, 2024

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Oct 17, 2024

jieyouxu force-pushed the boopjob branch from 49fcc6b to 6a6d20f Compare October 17, 2024 15:07

jieyouxu mentioned this pull request Oct 17, 2024

Bootstrap feature request: set --jobs in config.toml #131836

Closed

This comment has been minimized.

Sign in to view

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 17, 2024

jieyouxu force-pushed the boopjob branch 2 times, most recently from 8302cba to e057ca5 Compare October 18, 2024 02:54

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 18, 2024

onur-ozkan reviewed Oct 18, 2024

View reviewed changes

config.example.toml Outdated Show resolved Hide resolved

jieyouxu force-pushed the boopjob branch 2 times, most recently from 4adaf51 to 2fdc3b1 Compare October 18, 2024 05:25

jieyouxu commented Oct 18, 2024

View reviewed changes

src/bootstrap/src/core/config/config.rs Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 18, 2024

jieyouxu force-pushed the boopjob branch from 2fdc3b1 to a18e389 Compare October 18, 2024 08:38

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 18, 2024

jieyouxu added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 18, 2024

jieyouxu force-pushed the boopjob branch from a18e389 to 667d659 Compare October 19, 2024 04:03

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 19, 2024

onur-ozkan approved these changes Oct 19, 2024

View reviewed changes

src/bootstrap/src/core/config/config.rs Outdated Show resolved Hide resolved

bootstrap: allow setting --jobs in config.toml

65458ae

jieyouxu force-pushed the boopjob branch from 667d659 to 65458ae Compare October 19, 2024 05:36

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 19, 2024

workingjubilee mentioned this pull request Oct 19, 2024

Rollup of 8 pull requests #131914

Closed

matthiaskrgr mentioned this pull request Oct 19, 2024

Rollup of 10 pull requests #131922

Closed

matthiaskrgr mentioned this pull request Oct 19, 2024

Rollup of 9 pull requests #131924

Closed

matthiaskrgr mentioned this pull request Oct 19, 2024

Rollup of 8 pull requests #131934

Merged

bors merged commit 21475b7 into rust-lang:master Oct 19, 2024
6 checks passed

rustbot added this to the 1.84.0 milestone Oct 19, 2024

jieyouxu deleted the boopjob branch October 21, 2024 08:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bootstrap: allow setting `--jobs` in config.toml #131838

bootstrap: allow setting `--jobs` in config.toml #131838

jieyouxu commented Oct 17, 2024

rustbot commented Oct 17, 2024

This comment has been minimized.

jieyouxu commented Oct 17, 2024

jieyouxu commented Oct 18, 2024

onur-ozkan Oct 18, 2024

jieyouxu Oct 18, 2024 •

edited

Loading

jieyouxu Oct 18, 2024

This comment has been minimized.

jieyouxu commented Oct 18, 2024

jieyouxu commented Oct 18, 2024

jieyouxu commented Oct 19, 2024

onur-ozkan left a comment

onur-ozkan commented Oct 19, 2024

bors commented Oct 19, 2024

bootstrap: allow setting --jobs in config.toml #131838

bootstrap: allow setting --jobs in config.toml #131838

Conversation

jieyouxu commented Oct 17, 2024

rustbot commented Oct 17, 2024

This comment has been minimized.

jieyouxu commented Oct 17, 2024

jieyouxu commented Oct 18, 2024

onur-ozkan Oct 18, 2024

Choose a reason for hiding this comment

jieyouxu Oct 18, 2024 • edited Loading

Choose a reason for hiding this comment

jieyouxu Oct 18, 2024

Choose a reason for hiding this comment

This comment has been minimized.

jieyouxu commented Oct 18, 2024

jieyouxu commented Oct 18, 2024

jieyouxu commented Oct 19, 2024

onur-ozkan left a comment

Choose a reason for hiding this comment

onur-ozkan commented Oct 19, 2024

bors commented Oct 19, 2024

bootstrap: allow setting `--jobs` in config.toml #131838

bootstrap: allow setting `--jobs` in config.toml #131838

jieyouxu Oct 18, 2024 •

edited

Loading