Make target-cpu=native detect individual features #80749

as-com · 2021-01-06T08:32:24Z

This PR makes target-cpu=native check for and enable/disable individual features instead of detecting and targeting a CPU by name. This brings the flag's behavior more in line with clang and gcc and ensures that the host actually supports each feature that we are compiling for.

This should resolve issues with miscompilations on e.g. "Haswell" Pentiums and Celerons that lack support for AVX, and also enable support for aes on Broadwell processors that support it. It should also resolve issues with failing to detect feature support in newer CPUs that aren't yet known by LLVM (see: #80633).

Fixes #54688
Fixes #48464
Fixes #38218

rust-highfive · 2021-01-06T08:32:27Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @oli-obk (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

nagisa · 2021-01-07T16:07:19Z

r? @nagisa

This seems broadly LGTM, however I do have a concern with use of std::detect – in particular it only supports a few specific architectures (x86, x86_64, arm, aarch64, powerpc, powerpc64, mips and mips64). Furthermore it only iterates features that stdsimd knows about. This overall may lead to pessimizing the generated code.

With that in mind using LLVM's getHostCPUFeatures is probably better? It is also guaranteed that its output is suitable for passing to LLVM, avoiding the need to maintain a denylist of features like what you did with tsc here.

as-com · 2021-01-07T17:08:30Z

@nagisa Using getHostCPUFeatures would require an update to the C++ shim layer that translates the StringMap data structure. I'll take a look later to see if it's possible to do that without too much of a mess.

nagisa · 2021-01-07T18:05:47Z

There's a C API of this as well LLVMGetHostCPUFeatures, that could let you avoid doing this, but all in all its no huge deal if you find it better to adjust rustc_llvm's RustWrapper to expose an API that is more convenient to consume from the Rust side (e.g. by returning string-length pairs rather than null-terminated strings).

as-com · 2021-01-08T04:30:54Z

Alright, I added LLVMGetHostCPUFeatures and LLVMDisposeMessage to ffi.rs, and updated the implementation accordingly. It works on my machine(tm) and should be more robust than the previous version.

compiler/rustc_codegen_llvm/src/llvm_util.rs

nagisa · 2021-01-08T17:13:50Z

@bors r+ rollup=iffy

Thanks!

bors · 2021-01-08T17:13:52Z

📌 Commit 80ca198 has been approved by nagisa

bors · 2021-01-08T21:06:56Z

⌛ Testing commit 80ca198 with merge 2d5c7a78104c9fbd6643bc6124bf1a1bda24ce55...

bors · 2021-01-08T23:34:00Z

💔 Test failed - checks-actions

rust-log-analyzer · 2021-01-08T23:34:02Z

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

nagisa · 2021-01-08T23:40:16Z

@bors retry

Looks like a timeout

JohnTitor · 2021-01-09T04:36:19Z

bors stuck?

@bors retry

bors · 2021-01-09T04:36:33Z

⌛ Testing commit 80ca198 with merge c87ef0a...

bors · 2021-01-09T07:48:48Z

☀️ Test successful - checks-actions
Approved by: nagisa
Pushing c87ef0a to master...

Pkgsrc changes: * Remove one SunOS patch, apparently no longer needed. * Adapt one patch for Darwin, adjust cargo checksum accordingly. * Adjust bootstraps to version 1.50.0. Version 1.51.0 (2021-03-25) ============================ Language -------- - [You can now parameterize items such as functions, traits, and `struct`s by constant values in addition to by types and lifetimes.][79135] Also known as "const generics" E.g. you can now write the following. Note: Only values of primitive integers, `bool`, or `char` types are currently permitted. ```rust struct GenericArray<T, const LENGTH: usize> { inner: [T; LENGTH] } impl<T, const LENGTH: usize> GenericArray<T, LENGTH> { const fn last(&self) -> Option<&T> { if LENGTH == 0 { None } else { Some(&self.inner[LENGTH - 1]) } } } ``` Compiler -------- - [Added the `-Csplit-debuginfo` codegen option for macOS platforms.][79570] This option controls whether debug information is split across multiple files or packed into a single file. **Note** This option is unstable on other platforms. - [Added tier 3\* support for `aarch64_be-unknown-linux-gnu`, `aarch64-unknown-linux-gnu_ilp32`, and `aarch64_be-unknown-linux-gnu_ilp32` targets.][81455] - [Added tier 3 support for `i386-unknown-linux-gnu` and `i486-unknown-linux-gnu` targets.][80662] - [The `target-cpu=native` option will now detect individual features of CPUs.][80749] - [Rust now uses `inline-asm` for stack probes when used with LLVM 11.0.1+][77885] \* Refer to Rust's [platform support page][forge-platform-support] for more information on Rust's tiered platform support. Libraries --------- - [`Box::downcast` is now also implemented for any `dyn Any + Send + Sync` object.][80945] - [`str` now implements `AsMut<str>`.][80279] - [`u64` and `u128` now implement `From<char>`.][79502] - [`Error` is now implemented for `&T` where `T` implements `Error`.][75180] - [`Poll::{map_ok, map_err}` are now implemented for `Poll<Option<Result<T, E>>>`.][80968] - [`unsigned_abs` is now implemented for all signed integer types.][80959] - [`io::Empty` now implements `io::Seek`.][78044] - [`rc::Weak<T>` and `sync::Weak<T>`'s methods such as `as_ptr` are now implemented for `T: ?Sized` types.][80764] Stabilized APIs --------------- - [`Arc::decrement_strong_count`] - [`Arc::increment_strong_count`] - [`Once::call_once_force`] - [`Peekable::next_if_eq`] - [`Peekable::next_if`] - [`Seek::stream_position`] - [`array::IntoIter`] - [`panic::panic_any`] - [`ptr::addr_of!`] - [`ptr::addr_of_mut!`] - [`slice::fill_with`] - [`slice::split_inclusive_mut`] - [`slice::split_inclusive`] - [`slice::strip_prefix`] - [`slice::strip_suffix`] - [`str::split_inclusive`] - [`sync::OnceState`] - [`task::Wake`] Cargo ----- - [Added the `split-debuginfo` profile option to control the -Csplit-debuginfo codegen option.][cargo/9112] - [Added the `resolver` field to `Cargo.toml` to enable the new feature resolver and CLI option behavior.][cargo/8997] Version 2 of the feature resolver will try to avoid unifying features of dependencies where that unification could be unwanted. Such as using the same dependency with a `std` feature in a build scripts and proc-macros, while using the `no-std` feature in the final binary. See the [Cargo book documentation][feature-resolver@2.0] for more information on the feature. Rustdoc ------- - [Rustdoc will now include documentation for methods available from `Deref` traits.][80653] - [You can now provide a `--default-theme` flag which sets the default theme to use for documentation.][79642] Various improvements to intra-doc links: - [You can link to non-path primitives such as `slice`.][80181] - [You can link to associated items.][74489] - [You can now include generic parameters when linking to items, like `Vec<T>`.][76934] Misc ---- - [You can now pass `--include-ignored` to tests (e.g. with `cargo test -- --include-ignored`) to include testing tests marked `#[ignore]`.][80053] Compatibility Notes ------------------- - [WASI platforms no longer use the `wasm-bindgen` ABI, and instead use the wasm32 ABI.][79998] - [`rustc` no longer promotes division, modulo and indexing operations to `const` that could fail.][80579] - [The minimum version of glibc for the following platforms has been bumped to version 2.31 for the distributed artifacts.][81521] - `armv5te-unknown-linux-gnueabi` - `sparc64-unknown-linux-gnu` - `thumbv7neon-unknown-linux-gnueabihf` - `armv7-unknown-linux-gnueabi` - `x86_64-unknown-linux-gnux32` Internal Only ------------- - [Consistently avoid constructing optimized MIR when not doing codegen][80718] [79135]: rust-lang/rust#79135 [74489]: rust-lang/rust#74489 [76934]: rust-lang/rust#76934 [79570]: rust-lang/rust#79570 [80181]: rust-lang/rust#80181 [79642]: rust-lang/rust#79642 [80945]: rust-lang/rust#80945 [80279]: rust-lang/rust#80279 [80053]: rust-lang/rust#80053 [79502]: rust-lang/rust#79502 [75180]: rust-lang/rust#75180 [79135]: rust-lang/rust#79135 [81521]: rust-lang/rust#81521 [80968]: rust-lang/rust#80968 [80959]: rust-lang/rust#80959 [80718]: rust-lang/rust#80718 [80653]: rust-lang/rust#80653 [80579]: rust-lang/rust#80579 [79998]: rust-lang/rust#79998 [78044]: rust-lang/rust#78044 [81455]: rust-lang/rust#81455 [80764]: rust-lang/rust#80764 [80749]: rust-lang/rust#80749 [80662]: rust-lang/rust#80662 [77885]: rust-lang/rust#77885 [cargo/8997]: rust-lang/cargo#8997 [cargo/9112]: rust-lang/cargo#9112 [feature-resolver@2.0]: https://doc.rust-lang.org/nightly/cargo/reference/features.html#feature-resolver-version-2 [`Once::call_once_force`]: https://doc.rust-lang.org/stable/std/sync/struct.Once.html#method.call_once_force [`sync::OnceState`]: https://doc.rust-lang.org/stable/std/sync/struct.OnceState.html [`panic::panic_any`]: https://doc.rust-lang.org/stable/std/panic/fn.panic_any.html [`slice::strip_prefix`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.strip_prefix [`slice::strip_suffix`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.strip_prefix [`Arc::increment_strong_count`]: https://doc.rust-lang.org/nightly/std/sync/struct.Arc.html#method.increment_strong_count [`Arc::decrement_strong_count`]: https://doc.rust-lang.org/nightly/std/sync/struct.Arc.html#method.decrement_strong_count [`slice::fill_with`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.fill_with [`ptr::addr_of!`]: https://doc.rust-lang.org/nightly/std/ptr/macro.addr_of.html [`ptr::addr_of_mut!`]: https://doc.rust-lang.org/nightly/std/ptr/macro.addr_of_mut.html [`array::IntoIter`]: https://doc.rust-lang.org/nightly/std/array/struct.IntoIter.html [`slice::split_inclusive`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.split_inclusive [`slice::split_inclusive_mut`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.split_inclusive_mut [`str::split_inclusive`]: https://doc.rust-lang.org/nightly/std/primitive.str.html#method.split_inclusive [`task::Wake`]: https://doc.rust-lang.org/nightly/std/task/trait.Wake.html [`Seek::stream_position`]: https://doc.rust-lang.org/nightly/std/io/trait.Seek.html#method.stream_position [`Peekable::next_if`]: https://doc.rust-lang.org/nightly/std/iter/struct.Peekable.html#method.next_if [`Peekable::next_if_eq`]: https://doc.rust-lang.org/nightly/std/iter/struct.Peekable.html#method.next_if_eq

Pkgsrc changes: * Add support for the big-endian arm64 NetBSD target (aarch64_be). * On NetBSD/i386, use the i586 (pentium) bootstrap kit variant in preference to i686. * Adjust patches, re-compute line offsets, re-compute crate checksums. * Remove a patch which was either integrated upstream and/or no longer applies. * Bump bootstraps to 1.50.0. * Move conditionals until after bsd.prefs.mk so that they work... * Default to "dist" build target if cross-compiling, but allow also to override via rust.BUILD_TARGET. * Allow overriding MAKE_JOBS_SAFE via rust.MAKE_JOBS_SAFE if you want a different trade-off between occasional breakage and performance. * Adjust platform.mk according to work already done in wip/rust/ * Add a patch to optimize the install.sh script used to install binary bootstraps to not do so many forks; use case/esac and parameter expansion instead of grep, sed and cut. * Drop building documentation for the binary bootstrap kits. This will also impact the lang/rust-bin package. For full documentation, build or install lang/rust as a package. Upstream changes: Version 1.51.0 (2021-03-25) ============================ Language -------- - [You can now parameterize items such as functions, traits, and `struct`s by constant values in addition to by types and lifetimes.][79135] Also known as "const generics" E.g. you can now write the following. Note: Only values of primitive integers, `bool`, or `char` types are currently permitted. ```rust struct GenericArray<T, const LENGTH: usize> { inner: [T; LENGTH] } impl<T, const LENGTH: usize> GenericArray<T, LENGTH> { const fn last(&self) -> Option<&T> { if LENGTH == 0 { None } else { Some(&self.inner[LENGTH - 1]) } } } ``` Compiler -------- - [Added the `-Csplit-debuginfo` codegen option for macOS platforms.][79570] This option controls whether debug information is split across multiple files or packed into a single file. **Note** This option is unstable on other platforms. - [Added tier 3\* support for `aarch64_be-unknown-linux-gnu`, `aarch64-unknown-linux-gnu_ilp32`, and `aarch64_be-unknown-linux-gnu_ilp32` targets.][81455] - [Added tier 3 support for `i386-unknown-linux-gnu` and `i486-unknown-linux-gnu` targets.][80662] - [The `target-cpu=native` option will now detect individual features of CPUs.][80749] \* Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [`Box::downcast` is now also implemented for any `dyn Any + Send + Sync` object.][80945] - [`str` now implements `AsMut<str>`.][80279] - [`u64` and `u128` now implement `From<char>`.][79502] - [`Error` is now implemented for `&T` where `T` implements `Error`.][75180] - [`Poll::{map_ok, map_err}` are now implemented for `Poll<Option<Result<T,E>>>`.][80968] - [`unsigned_abs` is now implemented for all signed integer types.][80959] - [`io::Empty` now implements `io::Seek`.][78044] - [`rc::Weak<T>` and `sync::Weak<T>`'s methods such as `as_ptr` are now implemented for `T: ?Sized` types.][80764] - [`Div` and `Rem` by their `NonZero` variant is now implemented for all unsigned integers.][79134] Stabilized APIs --------------- - [`Arc::decrement_strong_count`] - [`Arc::increment_strong_count`] - [`Once::call_once_force`] - [`Peekable::next_if_eq`] - [`Peekable::next_if`] - [`Seek::stream_position`] - [`array::IntoIter`] - [`panic::panic_any`] - [`ptr::addr_of!`] - [`ptr::addr_of_mut!`] - [`slice::fill_with`] - [`slice::split_inclusive_mut`] - [`slice::split_inclusive`] - [`slice::strip_prefix`] - [`slice::strip_suffix`] - [`str::split_inclusive`] - [`sync::OnceState`] - [`task::Wake`] - [`VecDeque::range`] - [`VecDeque::range_mut`] Cargo ----- - [Added the `split-debuginfo` profile option to control the -Csplit-debuginfo codegen option.][cargo/9112] - [Added the `resolver` field to `Cargo.toml` to enable the new feature resolver and CLI option behavior.][cargo/8997] Version 2 of the feature resolver will try to avoid unifying features of dependencies where that unification could be unwanted. Such as using the same dependency with a `std` feature in a build scripts and proc-macros, while using the `no-std` feature in the final binary. See the [Cargo book documentation][feature-resolver@2.0] for more information on the feature. Rustdoc ------- - [Rustdoc will now include documentation for methods available from _nested_ `Deref` traits.][80653] - [You can now provide a `--default-theme` flag which sets the default theme to use for documentation.][79642] Various improvements to intra-doc links: - [You can link to non-path primitives such as `slice`.][80181] - [You can link to associated items.][74489] - [You can now include generic parameters when linking to items, like `Vec<T>`.][76934] Misc ---- - [You can now pass `--include-ignored` to tests (e.g. with `cargo test -- --include-ignored`) to include testing tests marked `#[ignore]`.][80053] Compatibility Notes ------------------- - [WASI platforms no longer use the `wasm-bindgen` ABI, and instead use the wasm32 ABI.][79998] - [`rustc` no longer promotes division, modulo and indexing operations to `const` that could fail.][80579] - [The minimum version of glibc for the following platforms has been bumped to version 2.31 for the distributed artifacts.][81521] - `armv5te-unknown-linux-gnueabi` - `sparc64-unknown-linux-gnu` - `thumbv7neon-unknown-linux-gnueabihf` - `armv7-unknown-linux-gnueabi` - `x86_64-unknown-linux-gnux32` - [`atomic::spin_loop_hint` has been deprecated.][80966] It's recommended to use `hint::spin_loop` instead. Internal Only ------------- - [Consistently avoid constructing optimized MIR when not doing codegen][80718] [79135]: rust-lang/rust#79135 [74489]: rust-lang/rust#74489 [76934]: rust-lang/rust#76934 [79570]: rust-lang/rust#79570 [80181]: rust-lang/rust#80181 [79642]: rust-lang/rust#79642 [80945]: rust-lang/rust#80945 [80279]: rust-lang/rust#80279 [80053]: rust-lang/rust#80053 [79502]: rust-lang/rust#79502 [75180]: rust-lang/rust#75180 [79135]: rust-lang/rust#79135 [81521]: rust-lang/rust#81521 [80968]: rust-lang/rust#80968 [80959]: rust-lang/rust#80959 [80718]: rust-lang/rust#80718 [80653]: rust-lang/rust#80653 [80579]: rust-lang/rust#80579 [79998]: rust-lang/rust#79998 [78044]: rust-lang/rust#78044 [81455]: rust-lang/rust#81455 [80764]: rust-lang/rust#80764 [80749]: rust-lang/rust#80749 [80662]: rust-lang/rust#80662 [79134]: rust-lang/rust#79134 [80966]: rust-lang/rust#80966 [cargo/8997]: rust-lang/cargo#8997 [cargo/9112]: rust-lang/cargo#9112 [feature-resolver@2.0]: https://doc.rust-lang.org/nightly/cargo/reference/features.html#feature-resolver-version-2 [`Once::call_once_force`]: https://doc.rust-lang.org/stable/std/sync/struct.Once.html#method.call_once_force [`sync::OnceState`]: https://doc.rust-lang.org/stable/std/sync/struct.OnceState.html [`panic::panic_any`]: https://doc.rust-lang.org/stable/std/panic/fn.panic_any.html [`slice::strip_prefix`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.strip_prefix [`slice::strip_suffix`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.strip_prefix [`Arc::increment_strong_count`]: https://doc.rust-lang.org/nightly/std/sync/struct.Arc.html#method.increment_strong_count [`Arc::decrement_strong_count`]: https://doc.rust-lang.org/nightly/std/sync/struct.Arc.html#method.decrement_strong_count [`slice::fill_with`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.fill_with [`ptr::addr_of!`]: https://doc.rust-lang.org/nightly/std/ptr/macro.addr_of.html [`ptr::addr_of_mut!`]: https://doc.rust-lang.org/nightly/std/ptr/macro.addr_of_mut.html [`array::IntoIter`]: https://doc.rust-lang.org/nightly/std/array/struct.IntoIter.html [`slice::split_inclusive`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.split_inclusive [`slice::split_inclusive_mut`]: https://doc.rust-lang.org/nightly/std/primitive.slice.html#method.split_inclusive_mut [`str::split_inclusive`]: https://doc.rust-lang.org/nightly/std/primitive.str.html#method.split_inclusive [`task::Wake`]: https://doc.rust-lang.org/nightly/std/task/trait.Wake.html [`Seek::stream_position`]: https://doc.rust-lang.org/nightly/std/io/trait.Seek.html#method.stream_position [`Peekable::next_if`]: https://doc.rust-lang.org/nightly/std/iter/struct.Peekable.html#method.next_if [`Peekable::next_if_eq`]: https://doc.rust-lang.org/nightly/std/iter/struct.Peekable.html#method.next_if_eq [`VecDeque::range`]: https://doc.rust-lang.org/nightly/std/collections/struct.VecDeque.html#method.range [`VecDeque::range_mut`]: https://doc.rust-lang.org/nightly/std/collections/struct.VecDeque.html#method.range_mut

Make target-cpu=native detect individual features

bf80159

rust-highfive assigned oli-obk Jan 6, 2021

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jan 6, 2021

as-com mentioned this pull request Jan 6, 2021

target-cpu=native should also check all the target-features #54688

Closed

rust-highfive assigned nagisa and unassigned oli-obk Jan 7, 2021

Use LLVMGetHostCPUFeatures instead of stdsimd

bc4c5ba

nagisa reviewed Jan 8, 2021

View reviewed changes

compiler/rustc_codegen_llvm/src/llvm_util.rs Outdated Show resolved Hide resolved

Check if the pointer is null/string is not utf8

80ca198

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 8, 2021

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jan 8, 2021

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 8, 2021

bors added the merged-by-bors This PR was explicitly merged by bors. label Jan 9, 2021

bors merged commit c87ef0a into rust-lang:master Jan 9, 2021

rustbot added this to the 1.51.0 milestone Jan 9, 2021

tmiasko mentioned this pull request Mar 12, 2021

major performance regression between Rust 1.50 and beta when using target-cpu=native #83027

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make target-cpu=native detect individual features #80749

Make target-cpu=native detect individual features #80749

as-com commented Jan 6, 2021 •

edited

Loading

rust-highfive commented Jan 6, 2021

nagisa commented Jan 7, 2021

as-com commented Jan 7, 2021

nagisa commented Jan 7, 2021

as-com commented Jan 8, 2021 •

edited

Loading

nagisa commented Jan 8, 2021

bors commented Jan 8, 2021

bors commented Jan 8, 2021

bors commented Jan 8, 2021

rust-log-analyzer commented Jan 8, 2021

nagisa commented Jan 8, 2021

JohnTitor commented Jan 9, 2021

bors commented Jan 9, 2021

bors commented Jan 9, 2021

Make target-cpu=native detect individual features #80749

Make target-cpu=native detect individual features #80749

Conversation

as-com commented Jan 6, 2021 • edited Loading

rust-highfive commented Jan 6, 2021

nagisa commented Jan 7, 2021

as-com commented Jan 7, 2021

nagisa commented Jan 7, 2021

as-com commented Jan 8, 2021 • edited Loading

nagisa commented Jan 8, 2021

bors commented Jan 8, 2021

bors commented Jan 8, 2021

bors commented Jan 8, 2021

rust-log-analyzer commented Jan 8, 2021

nagisa commented Jan 8, 2021

JohnTitor commented Jan 9, 2021

bors commented Jan 9, 2021

bors commented Jan 9, 2021

as-com commented Jan 6, 2021 •

edited

Loading

as-com commented Jan 8, 2021 •

edited

Loading