Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

do not borrow non-Sync data in constants #54424

Closed
wants to merge 7 commits into from

Conversation

RalfJung
Copy link
Member

@RalfJung RalfJung commented Sep 21, 2018

We cannot share that data across threads. non-Sync is as bad as non-Freeze in that regard.

This is currently WIP because it ignores a test that is broken by #54419. But it is good enough to get crater going.

Fixes #49206.

Cc @eddyb @nikomatsakis

@RalfJung
Copy link
Member Author

@bors try

@bors
Copy link
Contributor

bors commented Sep 21, 2018

⌛ Trying commit f6a5c9c with merge e4686a6...

bors added a commit that referenced this pull request Sep 21, 2018
WIP: do not borrow non-Sync data in constants

We cannot share that data across threads. non-Sync is as bad as non-Freeze in that regard.

This is currently WIP because it ignores a test that is broken by #54419. But it is good enough ti get crater going.

Fixes #49206.

Cc @eddyb @nikomatsakis
@@ -9,6 +9,7 @@
// except according to those terms.

// error-pattern:reached recursion limit
// ignore-test FIXME #54419
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the WIP part.

@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-5.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
[00:25:49]    Compiling syntax_pos v0.0.0 (file:///checkout/src/libsyntax_pos)
[00:25:54]    Compiling rustc_errors v0.0.0 (file:///checkout/src/librustc_errors)
[00:27:27]    Compiling proc_macro v0.0.0 (file:///checkout/src/libproc_macro)
[00:27:38]    Compiling syntax_ext v0.0.0 (file:///checkout/src/libsyntax_ext)
[00:29:17] error[E0492]: cannot borrow a constant which may contain interior mutability or non-`Sync` data. If your data is `Sync`, create a static instead
[00:29:17]    --> librustc/middle/resolve_lifetime.rs:349:39
[00:29:17]     |
[00:29:17] 349 | const ROOT_SCOPE: ScopeRef<'static> = &Scope::Root;
[00:29:17] 
[00:29:31] error: aborting due to previous error
[00:29:31] 
[00:29:31] For more information about this error, try `rustc --explain E0492`.
[00:29:31] For more information about this error, try `rustc --explain E0492`.
[00:29:31] error: Could not compile `rustc`.
[00:29:31] 
[00:29:31] To learn more, run the command again with --verbose.
[00:29:31] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "--target" "x86_64-unknown-linux-gnu" "-j" "4" "--release" "--locked" "--color" "always" "--features" " jemalloc" "--manifest-path" "/checkout/src/rustc/Cargo.toml" "--message-format" "json"
[00:29:31] expected success, got: exit code: 101
[00:29:31] thread 'main' panicked at 'cargo must succeed', bootstrap/compile.rs:1135:9
[00:29:31] travis_fold:end:stage1-rustc

[00:29:31] travis_time:end:stage1-rustc:start=1537532308057024168,finish=1537532590662682214,duration=282605658046


[00:29:31] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap build
[00:29:31] Build completed unsuccessfully in 0:24:24
[00:29:31] make: *** [all] Error 1
[00:29:31] Makefile:28: recipe for target 'all' failed
91404 ./obj/build/x86_64-unknown-linux-gnu/stage1
91380 ./obj/build/x86_64-unknown-linux-gnu/stage1/lib
77952 ./obj/build/x86_64-unknown-linux-gnu/stage1-rustc
74188 ./obj/build/x86_64-unknown-linux-gnu/stage0-sysroot

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors
Copy link
Contributor

bors commented Sep 21, 2018

💔 Test failed - status-travis

@rust-highfive
Copy link
Collaborator

The job dist-x86_64-linux of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_fold:end:services

travis_fold:start:git.checkout
travis_time:start:0e2c65ae
$ git clone --depth=2 --branch=try https://github.com/rust-lang/rust.git rust-lang/rust
---
[00:54:54]    Compiling syntax_pos v0.0.0 (file:///checkout/src/libsyntax_pos)
[00:54:59]    Compiling rustc_errors v0.0.0 (file:///checkout/src/librustc_errors)
[00:56:35]    Compiling proc_macro v0.0.0 (file:///checkout/src/libproc_macro)
[00:56:46]    Compiling syntax_ext v0.0.0 (file:///checkout/src/libsyntax_ext)
[00:58:18] error[E0492]: cannot borrow a constant which may contain interior mutability or non-`Sync` data. If your data is `Sync`, create a static instead
[00:58:18]    --> librustc/middle/resolve_lifetime.rs:349:39
[00:58:18]     |
[00:58:18] 349 | const ROOT_SCOPE: ScopeRef<'static> = &Scope::Root;
[00:58:18] 
[00:58:30] error: aborting due to previous error
[00:58:30] 
[00:58:30] For more information about this error, try `rustc --explain E0492`.
[00:58:30] For more information about this error, try `rustc --explain E0492`.
[00:58:30] error: Could not compile `rustc`.
[00:58:30] 
[00:58:30] To learn more, run the command again with --verbose.
[00:58:30] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "--target" "x86_64-unknown-linux-gnu" "-j" "4" "--release" "--locked" "--color" "always" "--features" " jemalloc" "--manifest-path" "/checkout/src/rustc/Cargo.toml" "--message-format" "json"
[00:58:30] expected success, got: exit code: 101
[00:58:30] thread 'main' panicked at 'cargo must succeed', bootstrap/compile.rs:1135:9
[00:58:30] travis_fold:end:stage1-rustc

[00:58:30] travis_time:end:stage1-rustc:start=1537534055714089615,finish=1537534330162767347,duration=274448677732

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors bors added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Sep 21, 2018
@RalfJung
Copy link
Member Author

Oh great. Not even the full compiler bootstraps with this change.^^

@RalfJung
Copy link
Member Author

Seems like rustc::middle::resolve_lifetime::Scope is not Sync but we have a const with a reference to it?

Why is it not Sync?

@RalfJung
Copy link
Member Author

I hacked around this, so that we can get a crater run.

@bors try

@bors
Copy link
Contributor

bors commented Sep 21, 2018

⌛ Trying commit 5adae03 with merge 4056754...

bors added a commit that referenced this pull request Sep 21, 2018
WIP: do not borrow non-Sync data in constants

We cannot share that data across threads. non-Sync is as bad as non-Freeze in that regard.

This is currently WIP because it ignores a test that is broken by #54419. But it is good enough to get crater going.

Fixes #49206.

Cc @eddyb @nikomatsakis
@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-5.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
[00:52:49] ....................................................................................................
[00:52:52] .......................................................i............................................
[00:52:55] ....................................................................................................
[00:52:58] ....................................................................................................
[00:53:01] ...iiiiiiiii........................................................................................
[00:53:07] ....................................................................................................
[00:53:10] .......................................................................................i............
[00:53:12] ....................................................................................................
[00:53:15] ..........................................i.i..ii...................................................
---
[00:59:30] ---- [run-pass] run-pass/rvalue-static-promotion.rs stdout ----
[00:59:30] 
[00:59:30] error: compilation failed!
[00:59:30] status: exit code: 1
[00:59:30] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/run-pass/rvalue-static-promotion.rs" "--target=x86_64-unknown-linux-gnu" "-C" "prefer-dynamic" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass/rvalue-static-promotion/a" "-Crpath" "-O" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass/rvalue-static-promotion/auxiliary"
[00:59:30] ------------------------------------------
[00:59:30] 
[00:59:30] ------------------------------------------
[00:59:30] stderr:
[00:59:30] stderr:
[00:59:30] ------------------------------------------
[00:59:30] error[E0597]: borrowed value does not live long enough
[00:59:30]    |
[00:59:30]    |
[00:59:30] 25 |     let _: &'static Option<Cell<String>> = &NONE_CELL_STRING;
[00:59:30]    |                                             ^^^^^^^^^^^^^^^^ temporary value does not live long enough
[00:59:30] 26 |     let _: &'static Option<Box<()>> = &Foo::FOO;
[00:59:30] 27 | }
[00:59:30]    | - temporary value only lives until here
[00:59:30]    |
[00:59:30]    = note: borrowed value must be valid for the static lifetime...
[00:59:30] 
[00:59:30] error[E0597]: borrowed value does not live long enough
[00:59:30]    |
[00:59:30]    |
[00:59:30] 26 |     let _: &'static Option<Box<()>> = &Foo::FOO;
[00:59:30]    |                                        ^^^^^^^^ temporary value does not live long enough
[00:59:30] 27 | }
[00:59:30]    | - temporary value only lives until here
[00:59:30]    |
[00:59:30]    = note: borrowed value must be valid for the static lifetime...
[00:59:30] error: aborting due to 2 previous errors
[00:59:30] 
[00:59:30] For more information about this error, try `rustc --explain E0597`.
[00:59:30] 
---
[00:59:30] 
[00:59:30] thread 'main' panicked at 'Some tests failed', tools/compiletest/src/main.rs:496:22
[00:59:30] 
[00:59:30] 
[00:59:30] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/run-pass" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-pass" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "run-pass" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-5.0/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Zunstable-options " "--target-rustcflags" "-Crpath -O -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "5.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
[00:59:30] 
[00:59:30] 
[00:59:30] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
[00:59:30] Build completed unsuccessfully in 0:15:13
[00:59:30] Build completed unsuccessfully in 0:15:13
[00:59:30] make: *** [check] Error 1
[00:59:30] Makefile:58: recipe for target 'check' failed

The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:0ab30daa
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors
Copy link
Contributor

bors commented Sep 21, 2018

☀️ Test successful - status-travis
State: approved= try=True

@pietroalbini
Copy link
Member

@craterbot run start=master#1002e404e1ae6b8b907c8655edd41380d0c561cb end=try#4056754e28b7351841378254fef9dbe2f2357c48 mode=check-only

@craterbot
Copy link
Collaborator

👌 Experiment pr-54424 created and queued.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-crater Status: Waiting on a crater run to be completed. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 21, 2018
@craterbot
Copy link
Collaborator

🚧 Experiment pr-54424 is now running on agent aws-2.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@pietroalbini
Copy link
Member

@craterbot abort

@craterbot
Copy link
Collaborator

🗑️ Experiment pr-54424 deleted!

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-crater Status: Waiting on a crater run to be completed. labels Sep 22, 2018
@pietroalbini
Copy link
Member

@craterbot run start=master#1002e404e1ae6b8b907c8655edd41380d0c561cb end=try#4056754e28b7351841378254fef9dbe2f2357c48 mode=check-only

@craterbot
Copy link
Collaborator

👌 Experiment pr-54424 created and queued.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-crater Status: Waiting on a crater run to be completed. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 22, 2018
@craterbot
Copy link
Collaborator

🚧 Experiment pr-54424 is now running on agent aws-1.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot
Copy link
Collaborator

🎉 Experiment pr-54424-2 is completed!
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the blacklist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-crater Status: Waiting on a crater run to be completed. labels Oct 3, 2018
@RalfJung
Copy link
Member Author

RalfJung commented Oct 4, 2018

That didn't go as well as I had hoped. :/

The most common failure pattern is something like:

Oct 03 16:37:12.991 INFO kablam! 135 |   pub const FFI_RESULT_OK: &FfiResult = &FfiResult {
Oct 03 16:37:12.991 INFO kablam!     |  _______________________________________^
Oct 03 16:37:12.992 INFO kablam! 136 | |     error_code: 0,
Oct 03 16:37:12.993 INFO kablam! 137 | |     description: 0 as *const c_char,
Oct 03 16:37:12.993 INFO kablam! 138 | | };
Oct 03 16:37:12.994 INFO kablam!     | |_^

Value-based, this seems fine. However, this is still constructing an instance of a non-Sync type (because of the raw pointer), and there is no way to know that it's non-Sync for no good reason, at least for these values.

Basically, if I have a struct containing a non-Sync type, I can rely on it not being Sync. It doesn't matter that all values floating into the constructor are Sync, wrapping a newtype around them can change the game. This breaks ffi_utils, jobserver, libdwfl and is likely also the cause for yargb. Something similar can happen for enums (being inherently non-Sync because some variant is), and that breaks unison, sharing-list and sexpr-rust.

seq could be fixed by improving value-based reasoning for enums to ignore lifetime bounds.

rustfmt (indirectly) has a Span that it wants to share across thread in a const, which is not Sync.

@TimNN TimNN added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 16, 2018
@TimNN
Copy link
Contributor

TimNN commented Oct 16, 2018

Ping from triage @RalfJung: What is the status of this PR?

@RalfJung
Copy link
Member Author

Well, good question. I still think we need to do something for soundness, but I do not have a good counterexample (just the contrived one from #49206 (comment)) and I don't know how fix fix this and maintain compatibility -- how to obtain enough "value-based reasoning" that the existing code keeps working.

Would this be lang team or compiler team? I'm guessing lang.

@RalfJung RalfJung added the T-lang Relevant to the language team, which will review and decide on the PR/issue. label Oct 17, 2018
@TimNN
Copy link
Contributor

TimNN commented Oct 23, 2018

Ping from triage @rust-lang/lang: It looks like this PR could use your input.

If I understand things correctly, the current issue is that a specific pattern (consts containing references to other non-sync consts) would become an error with this PR, but is used by some crates. It is not immediately obvious that the this specific pattern is unsound except in some very contrived examples.

This PR also has some other effects I believe, but I'm not sure what exactly they are.

@RalfJung: Please verify that my summary is correct / add anything I have missed. It would then probably make sense to I-nominate this PR for lang-team discussion.

Copy link
Contributor

@oli-obk oli-obk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that these are weird cases, but the fact that these things can be constructed at compile-time does have some meaning (yes I read the issue).

I think we could reduce the breaking change to runtime promoteds only and keep allowing !Sync but Freeze data in constants.

/// Concretely, some type parameters get replaced by `()`. We assume that
/// if a type parameter has no bounds (except for `Sized`), and if the same
/// type is `Sync` for that type parameter replaced by `()`, then the constructor
/// valie is also `Sync` at the original type. This is a kind of parametricity
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

valie -> value

/// type is `Sync` for that type parameter replaced by `()`, then the constructor
/// valie is also `Sync` at the original type. This is a kind of parametricity
/// assumption.
/// For now, we only do anything if *no* type parameter has any bound other than
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

anything -> something

// So we can check for that instead of `Freeze`.
let freeze = Some(def.did) != self.tcx.lang_items().unsafe_cell_type();
// For Sync, we generally have to fall back on the type, but have a
// special exception for 0-argument ADT cosntructors.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cosntructors -> constructors

// special exception for 0-argument ADT cosntructors.
let sync = {
let generalized_ty = if args.len() == 0 {
// A type cosntructor with no arguments is directly a value.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cosntructor

@eddyb
Copy link
Member

eddyb commented Nov 5, 2018

@oli-obk Maybe we can emit lint warnings (from the MIR checking code) for consts?
Not necessarily saying we'll break it, but... I don't know.
I wish we had a proper model for value-based reasoning, because it's so nice.

@RalfJung
Copy link
Member Author

RalfJung commented Nov 8, 2018

I wish we had a proper model for value-based reasoning, because it's so nice.

The problem is mostly one of not enough information: When a type is not Sync, we do not know if that is an artifact arising from a non-Sync component (so if we can prove that we got a Sync value for that component, we can assume that a value of our original type is also Sync), or if this is a deliberate an necessary restriction because the type does something unsafe and non-Sync.

This would all be trivial if we had a notion of unsafe types...

@RalfJung
Copy link
Member Author

RalfJung commented Nov 8, 2018

@TimNN: Basically, yes. There is no grounds on which we can allow non-Sync references in constants: We are compiling code with behavior that would be impossible to obtain without these references. This is strictly expanding the power of safe code. To make things worse, it is entirely unclear what the proof obligation looks like that unsafe code has to satisfy to justify this extra power that safe code has.

Yet, with the current limitations of const, the counter-examples I can construct are rather contrived and based on comparing pointer addresses. But I am very worried that we might extend the scope of const to the extend that this soundness hole becomes easier to exploit.

I will try to be in the lang team meeting today to discuss this.

@RalfJung
Copy link
Member Author

RalfJung commented Nov 8, 2018

Some potential hacks I could imagine:

  • Only reject references to type that explicitly impl !Sync.
  • When checking for Sync here, assume that raw pointers are Sync. (That is a sound assumption, and from what I recall it would fix most of the bad examples. However, I am not sure if this is feasible.)

@RalfJung
Copy link
Member Author

RalfJung commented Nov 9, 2018

Summary of lang team discussion: Nobody objected to this being a soundness issue per se, but for lack of actually exhibited UB we also shouldn't just break code. I am going to try and make this a warning instead of an error, redirecting people to a tracking issue that explains what this is about. I think for promotion we can likely make it a hard error now.

I also noticed that almost all of the remaining regressions are due to raw pointers. They are by far the most common Freeze but non-Sync type -- and they could be Sync; the reason they are not is essentially just a lint. We cannot change that now, but I proposed that we could do the Sync check for references in const under a hypothetical logical context where raw pointers do implement Sync. That wouldn't be entirely sound either, but it would likely be better than the current situation...

@Mark-Simulacrum
Copy link
Member

hypothetical logical context where raw pointers do implement Sync. That wouldn't be entirely sound either, but it would likely be better than the current situation...

This seems unlikely to be ideal since unsafe code outside of the compiler/std is likely to use PhantomData with raw pointers to opt-out of sync intentionally, as negative impls are unstable.


More generally, will you have a chance to update this PR soon? Or maybe lang team needs to come to a decision on it?

@RalfJung
Copy link
Member Author

I hope to get to it next week.

@TimNN
Copy link
Contributor

TimNN commented Nov 27, 2018

Ping from triage @RalfJung: What is the status of this PR?

@RalfJung
Copy link
Member Author

Ah sorry, I didn't have time to get to this. :/ I still want to do this, but it could take a few more weeks.

@TimNN TimNN added A-allocators Area: Custom and system allocators and removed A-allocators Area: Custom and system allocators labels Dec 4, 2018
@Dylan-DPC-zz
Copy link

ping from triage @RalfJung any updates on this?

@RalfJung
Copy link
Member Author

RalfJung commented Jan 9, 2019

Not really, sorry -- the time machine in my basement is still broken.
I still want to do this, but don't know when I will have the time. If anybody else wants to give it a try, let me know!

@TimNN
Copy link
Contributor

TimNN commented Jan 22, 2019

Ping from triage! If I understand recent comments correctly, this PR won't make any progress for a while (I really wish there was that time-machine 🙄), so I'm closing it as inactive for now. Thanks for your contributions, @RalfJung!

@TimNN TimNN closed this Jan 22, 2019
@TimNN TimNN added S-inactive Status: Inactive and waiting on the author. This is often applied to closed PRs. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jan 22, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-inactive Status: Inactive and waiting on the author. This is often applied to closed PRs. T-lang Relevant to the language team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.