Remove some unsound specializations #68358

matthewjasper · 2020-01-19T00:53:05Z

This removes the unsound and exploitable specializations in the standard library

The PartialEq and Hash implementations for RangeInclusive are changed to avoid specialization.
The PartialOrd specialization for slices now specializes on a limited set of concrete types.
Added some tests for the soundness problems.

rust-highfive · 2020-01-19T00:53:09Z

(rust_highfive has picked a reviewer for you, use r? to override)

matthewjasper · 2020-01-19T00:54:12Z

I'm running perf, but I suspect that this is more likely to affect other applications more than the compiler.
@bors try @rust-timer queue

rust-timer · 2020-01-19T00:54:14Z

Awaiting bors try build completion

bors · 2020-01-19T00:54:25Z

⌛ Trying commit 70e00844d4f12bf72c585d95a31b5402487efa85 with merge 7aaf7627e1102ae1adad316889196660d88879c8...

rust-highfive · 2020-01-19T02:01:28Z

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2020-01-19T00:53:40.5236094Z ========================== Starting Command Output ===========================
2020-01-19T00:53:40.5240323Z [command]/bin/bash --noprofile --norc /home/vsts/work/_temp/89fdb62c-760e-4425-a48e-2d0e9e6101eb.sh
2020-01-19T00:53:40.5680857Z 
2020-01-19T00:53:40.5762224Z ##[section]Finishing: Disable git automatic line ending conversion
2020-01-19T00:53:40.5769366Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/68358/merge to s
2020-01-19T00:53:40.5771186Z Task         : Get sources
2020-01-19T00:53:40.5771269Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-01-19T00:53:40.5771306Z Version      : 1.0.0
2020-01-19T00:53:40.5771339Z Author       : Microsoft
---
2020-01-19T00:53:46.2007720Z ##[command]git remote add origin https://github.com/rust-lang/rust
2020-01-19T00:53:46.2469771Z ##[command]git config gc.auto 0
2020-01-19T00:53:46.2539010Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2020-01-19T00:53:46.2595619Z ##[command]git config --get-all http.proxy
2020-01-19T00:53:46.2757410Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/68358/merge:refs/remotes/pull/68358/merge
---
2020-01-19T01:49:35.5149253Z .................................................................................................... 1700/9540
2020-01-19T01:49:42.4357499Z .................................................................................................... 1800/9540
2020-01-19T01:49:54.5865159Z ...................i................................................................................ 1900/9540
2020-01-19T01:50:01.9738941Z .................................................................................................... 2000/9540
2020-01-19T01:50:18.5317962Z .........iiiii...................................................................................... 2100/9540
2020-01-19T01:50:28.5163754Z .................................................................................................... 2300/9540
2020-01-19T01:50:30.9164233Z .................................................................................................... 2400/9540
2020-01-19T01:50:36.3936343Z .................................................................................................... 2500/9540
2020-01-19T01:50:57.8380704Z .................................................................................................... 2600/9540
---
2020-01-19T01:53:45.2492812Z .....................................................i...............i.............................. 4900/9540
2020-01-19T01:53:53.4996428Z .................................................................................................... 5000/9540
2020-01-19T01:54:01.5284469Z ................................................................................................i... 5100/9540
2020-01-19T01:54:06.8571910Z .................................................................................................... 5200/9540
2020-01-19T01:54:17.6304368Z ....................................................................ii.ii...........i............... 5300/9540
2020-01-19T01:54:26.5402163Z .....i.............................................................................................. 5500/9540
2020-01-19T01:54:36.4480560Z .................................................................................................... 5600/9540
2020-01-19T01:54:42.7806978Z ......................................................i............................................. 5700/9540
2020-01-19T01:54:49.5331266Z .................................................................................................... 5800/9540
2020-01-19T01:54:49.5331266Z .................................................................................................... 5800/9540
2020-01-19T01:54:59.4308700Z .................................................................................................... 5900/9540
2020-01-19T01:55:06.0772745Z ............................................ii...i..ii...........i.................................. 6000/9540
2020-01-19T01:55:27.2775928Z .................................................................................................... 6200/9540
2020-01-19T01:55:35.1912033Z .................................................................................................... 6300/9540
2020-01-19T01:55:35.1912033Z .................................................................................................... 6300/9540
2020-01-19T01:55:45.0614625Z ........................................................................i..ii....................... 6400/9540
2020-01-19T01:56:14.0288459Z .................................................................................................... 6600/9540
2020-01-19T01:56:16.9047381Z ................................................i................................................... 6700/9540
2020-01-19T01:56:18.9875178Z .................................................................................................... 6800/9540
2020-01-19T01:56:21.1827391Z ...............................................i.................................................... 6900/9540
---
2020-01-19T01:57:55.7721471Z .................................................................................................... 7500/9540
2020-01-19T01:58:00.2044904Z .................................................................................................... 7600/9540
2020-01-19T01:58:05.6208917Z .................................................................................................... 7700/9540
2020-01-19T01:58:11.9524722Z .................................................................................................... 7800/9540
2020-01-19T01:58:23.1610544Z ..................................................................................................ii 7900/9540
2020-01-19T01:58:29.8130513Z iiiii............................................................................................... 8000/9540
2020-01-19T01:58:44.7911472Z ..........................................................................................FF........ 8200/9540
2020-01-19T01:58:56.1954706Z .................................................................................................... 8300/9540
2020-01-19T01:59:08.3299847Z .................................................................................................... 8400/9540
2020-01-19T01:59:13.9401110Z .................................................................................................... 8500/9540
---
2020-01-19T02:01:06.1189990Z ---- [ui] ui/specialization/soundness/partial_eq_range_inclusive.rs stdout ----
2020-01-19T02:01:06.1190208Z 
2020-01-19T02:01:06.1190563Z error: test compilation failed although it shouldn't!
2020-01-19T02:01:06.1190766Z status: exit code: 1
2020-01-19T02:01:06.1191619Z command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/specialization/soundness/partial_eq_range_inclusive.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--error-format" "json" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-C" "prefer-dynamic" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/specialization/soundness/partial_eq_range_inclusive/a" "-Crpath" "-O" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/specialization/soundness/partial_eq_range_inclusive/auxiliary"
2020-01-19T02:01:06.1192197Z ------------------------------------------
2020-01-19T02:01:06.1192350Z 
2020-01-19T02:01:06.1194191Z ------------------------------------------
2020-01-19T02:01:06.1194508Z stderr:
2020-01-19T02:01:06.1194508Z stderr:
2020-01-19T02:01:06.1195080Z ------------------------------------------
2020-01-19T02:01:06.1195634Z error[E0369]: binary operation `==` cannot be applied to type `std::cell::Ref<'_, std::vec::Vec<&str>>`
2020-01-19T02:01:06.1196186Z   --> /checkout/src/test/ui/specialization/soundness/partial_eq_range_inclusive.rs:34:5
2020-01-19T02:01:06.1196729Z LL |     assert_eq!(values.borrow(), Vec::new());
2020-01-19T02:01:06.1196873Z    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2020-01-19T02:01:06.1197000Z    |     |
2020-01-19T02:01:06.1197335Z    |     std::cell::Ref<'_, std::vec::Vec<&str>>
2020-01-19T02:01:06.1197335Z    |     std::cell::Ref<'_, std::vec::Vec<&str>>
2020-01-19T02:01:06.1197537Z    |     std::vec::Vec<_>
2020-01-19T02:01:06.1197682Z    |
2020-01-19T02:01:06.1198088Z    = note: an implementation of `std::cmp::PartialEq` might be missing for `std::cell::Ref<'_, std::vec::Vec<&str>>`
2020-01-19T02:01:06.1198993Z 
2020-01-19T02:01:06.1199194Z error: aborting due to previous error
2020-01-19T02:01:06.1199314Z 
2020-01-19T02:01:06.1199755Z For more information about this error, try `rustc --explain E0369`.
---
2020-01-19T02:01:06.1205600Z 
2020-01-19T02:01:06.1205811Z warning: unused variable: `other`
2020-01-19T02:01:06.1206262Z   --> $DIR/partial_ord_slice.rs:25:19
2020-01-19T02:01:06.1206698Z    |
2020-01-19T02:01:06.1207256Z LL |     fn cmp(&self, other: &Self) -> Ordering {
2020-01-19T02:01:06.1207464Z    |                   ^^^^^ help: consider prefixing with an underscore: `_other`
2020-01-19T02:01:06.1207739Z warning: unused comparison that must be used
2020-01-19T02:01:06.1208247Z   --> $DIR/partial_ord_slice.rs:37:9
2020-01-19T02:01:06.1208420Z    |
2020-01-19T02:01:06.1208420Z    |
2020-01-19T02:01:06.1208547Z LL |         &y[..] <= &y[..];
2020-01-19T02:01:06.1209035Z    |
2020-01-19T02:01:06.1209198Z    = note: `#[warn(unused_must_use)]` on by default
2020-01-19T02:01:06.1209315Z 
2020-01-19T02:01:06.1209427Z 
2020-01-19T02:01:06.1209427Z 
2020-01-19T02:01:06.1209539Z 
2020-01-19T02:01:06.1209651Z 
2020-01-19T02:01:06.1209784Z The actual stderr differed from the expected stderr.
2020-01-19T02:01:06.1210457Z Actual stderr saved to /checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/specialization/soundness/partial_ord_slice/partial_ord_slice.stderr
2020-01-19T02:01:06.1210897Z To update references, rerun the tests and pass the `--bless` flag
2020-01-19T02:01:06.1211361Z To only update this specific test, also pass `--test-args specialization/soundness/partial_ord_slice.rs`
2020-01-19T02:01:06.1211684Z error: 1 errors occurred comparing output.
2020-01-19T02:01:06.1211854Z status: exit code: 0
2020-01-19T02:01:06.1211854Z status: exit code: 0
2020-01-19T02:01:06.1213119Z command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/specialization/soundness/partial_ord_slice.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--error-format" "json" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-C" "prefer-dynamic" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/specialization/soundness/partial_ord_slice/a" "-Crpath" "-O" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/specialization/soundness/partial_ord_slice/auxiliary"
2020-01-19T02:01:06.1227330Z ------------------------------------------
2020-01-19T02:01:06.1227725Z 
2020-01-19T02:01:06.1228097Z ------------------------------------------
2020-01-19T02:01:06.1228277Z stderr:
---
2020-01-19T02:01:06.1247439Z 
2020-01-19T02:01:06.1247569Z warning: unused variable: `other`
2020-01-19T02:01:06.1248021Z   --> /checkout/src/test/ui/specialization/soundness/partial_ord_slice.rs:25:19
2020-01-19T02:01:06.1248987Z    |
2020-01-19T02:01:06.1249494Z LL |     fn cmp(&self, other: &Self) -> Ordering {
2020-01-19T02:01:06.1249747Z    |                   ^^^^^ help: consider prefixing with an underscore: `_other`
2020-01-19T02:01:06.1249998Z warning: unused comparison that must be used
2020-01-19T02:01:06.1250396Z   --> /checkout/src/test/ui/specialization/soundness/partial_ord_slice.rs:37:9
2020-01-19T02:01:06.1250591Z    |
2020-01-19T02:01:06.1250591Z    |
2020-01-19T02:01:06.1250718Z LL |         &y[..] <= &y[..];
2020-01-19T02:01:06.1251010Z    |
2020-01-19T02:01:06.1251138Z    = note: `#[warn(unused_must_use)]` on by default
2020-01-19T02:01:06.1251245Z 
2020-01-19T02:01:06.1251351Z 
---
2020-01-19T02:01:06.1254814Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:387:22
2020-01-19T02:01:06.1255102Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2020-01-19T02:01:06.1255260Z 
2020-01-19T02:01:06.1255428Z 
2020-01-19T02:01:06.1257249Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "ui" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-7/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "7.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2020-01-19T02:01:06.1257855Z 
2020-01-19T02:01:06.1257963Z 
2020-01-19T02:01:06.1261144Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
2020-01-19T02:01:06.1261336Z Build completed unsuccessfully in 1:01:09
2020-01-19T02:01:06.1261336Z Build completed unsuccessfully in 1:01:09
2020-01-19T02:01:06.1321845Z == clock drift check ==
2020-01-19T02:01:06.1334280Z   local time: Sun Jan 19 02:01:06 UTC 2020
2020-01-19T02:01:06.6878784Z   network time: Sun, 19 Jan 2020 02:01:06 GMT
2020-01-19T02:01:06.6879509Z == end clock drift check ==
2020-01-19T02:01:07.1742311Z 
2020-01-19T02:01:07.1835227Z ##[error]Bash exited with code '1'.
2020-01-19T02:01:07.1848073Z ##[section]Finishing: Run build
2020-01-19T02:01:07.1869478Z ##[section]Starting: Checkout rust-lang/rust@refs/pull/68358/merge to s
2020-01-19T02:01:07.1871441Z Task         : Get sources
2020-01-19T02:01:07.1871479Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2020-01-19T02:01:07.1871536Z Version      : 1.0.0
2020-01-19T02:01:07.1871588Z Author       : Microsoft
2020-01-19T02:01:07.1871588Z Author       : Microsoft
2020-01-19T02:01:07.1871628Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
2020-01-19T02:01:07.1871670Z ==============================================================================
2020-01-19T02:01:07.5846374Z Cleaning any cached credential from repository: rust-lang/rust (GitHub)
2020-01-19T02:01:07.5887631Z ##[section]Finishing: Checkout rust-lang/rust@refs/pull/68358/merge to s
2020-01-19T02:01:07.6000979Z Cleaning up task key
2020-01-19T02:01:07.6001748Z Start cleaning up orphan processes.
2020-01-19T02:01:07.6107823Z Terminate orphan process: pid (3794) (python)
2020-01-19T02:01:07.6952760Z ##[section]Finishing: Finalize Job

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

bors · 2020-01-19T03:28:28Z

☀️ Try build successful - checks-azure
Build commit: 7aaf7627e1102ae1adad316889196660d88879c8 (7aaf7627e1102ae1adad316889196660d88879c8)

rust-timer · 2020-01-19T03:28:30Z

Queued 7aaf7627e1102ae1adad316889196660d88879c8 with parent 1ce08f9, future comparison URL.

src/libcore/iter/adapters/mod.rs

Centril · 2020-02-03T18:55:48Z

r? @nikomatsakis

nikomatsakis · 2020-02-07T20:20:56Z

@bors r+

bors · 2020-02-07T20:20:57Z

📌 Commit a81c59f has been approved by nikomatsakis

Remove some unsound specializations This removes the unsound and exploitable specializations in the standard library * The `PartialEq` and `Hash` implementations for `RangeInclusive` are changed to avoid specialization. * The `PartialOrd` specialization for slices now specializes on a limited set of concrete types. * Added some tests for the soundness problems.

bors · 2020-02-08T03:47:05Z

⌛ Testing commit a81c59f with merge 6cad754...

Remove some unsound specializations This removes the unsound and exploitable specializations in the standard library * The `PartialEq` and `Hash` implementations for `RangeInclusive` are changed to avoid specialization. * The `PartialOrd` specialization for slices now specializes on a limited set of concrete types. * Added some tests for the soundness problems.

bors · 2020-02-08T07:10:36Z

☀️ Test successful - checks-azure
Approved by: nikomatsakis
Pushing 6cad754 to master...

rust-highfive assigned KodrAus Jan 19, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jan 19, 2020

matthewjasper force-pushed the spec-fix branch from 70e0084 to 79e0098 Compare January 19, 2020 09:32

KamilaBorowska reviewed Jan 19, 2020

View reviewed changes

src/libcore/iter/adapters/mod.rs Outdated Show resolved Hide resolved

KamilaBorowska reviewed Jan 19, 2020

View reviewed changes

src/libcore/iter/adapters/mod.rs Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

JohnCSimon added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 26, 2020

pnkfelix mentioned this pull request Jan 30, 2020

[WIP] Commenting out unsound specialization. #68280

Closed

matthewjasper force-pushed the spec-fix branch from 79e0098 to 5665834 Compare January 30, 2020 20:09

This comment has been minimized.

Sign in to view

matthewjasper force-pushed the spec-fix branch from 5665834 to 4896837 Compare January 30, 2020 21:05

This comment has been minimized.

Sign in to view

matthewjasper changed the title ~~Remove some unsound or hard to verify specializations~~ Remove some unsound specializations Jan 31, 2020

matthewjasper force-pushed the spec-fix branch from 4896837 to 0998993 Compare January 31, 2020 22:28

This comment has been minimized.

Sign in to view

Remove some unsound specializations

a81c59f

matthewjasper force-pushed the spec-fix branch from 0998993 to a81c59f Compare February 1, 2020 09:20

rust-highfive assigned nikomatsakis and unassigned KodrAus Feb 3, 2020

matthewjasper removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Feb 5, 2020

matthewjasper added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Feb 5, 2020

nikomatsakis approved these changes Feb 7, 2020

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 7, 2020

Dylan-DPC-zz mentioned this pull request Feb 8, 2020

Rollup of 7 pull requests #68942

Closed

bors mentioned this pull request Feb 8, 2020

Invert control in struct_lint_level. #68725

Merged

bors added the merged-by-bors This PR was explicitly merged by bors. label Feb 8, 2020

bors merged commit a81c59f into rust-lang:master Feb 8, 2020

matthewjasper deleted the spec-fix branch February 8, 2020 07:17

bors mentioned this pull request Feb 8, 2020

Remove problematic specialization from RangeInclusive #68835

Merged

SimonSapin mentioned this pull request Feb 10, 2020

PartialEq implementation for RangeInclusive is unsound #67194

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove some unsound specializations #68358

Remove some unsound specializations #68358

matthewjasper commented Jan 19, 2020 •

edited

Loading

rust-highfive commented Jan 19, 2020

matthewjasper commented Jan 19, 2020

rust-timer commented Jan 19, 2020

bors commented Jan 19, 2020

rust-highfive commented Jan 19, 2020

bors commented Jan 19, 2020

rust-timer commented Jan 19, 2020

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Centril commented Feb 3, 2020

nikomatsakis commented Feb 7, 2020

bors commented Feb 7, 2020

bors commented Feb 8, 2020

bors commented Feb 8, 2020

Remove some unsound specializations #68358

Remove some unsound specializations #68358

Conversation

matthewjasper commented Jan 19, 2020 • edited Loading

rust-highfive commented Jan 19, 2020

matthewjasper commented Jan 19, 2020

rust-timer commented Jan 19, 2020

bors commented Jan 19, 2020

rust-highfive commented Jan 19, 2020

bors commented Jan 19, 2020

rust-timer commented Jan 19, 2020

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Centril commented Feb 3, 2020

nikomatsakis commented Feb 7, 2020

bors commented Feb 7, 2020

bors commented Feb 8, 2020

bors commented Feb 8, 2020

matthewjasper commented Jan 19, 2020 •

edited

Loading