Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking issue for #![feature(const_precise_live_drops)] #73255

Open
1 task done
ecstatic-morse opened this issue Jun 11, 2020 · 52 comments
Open
1 task done

Tracking issue for #![feature(const_precise_live_drops)] #73255

ecstatic-morse opened this issue Jun 11, 2020 · 52 comments
Assignees
Labels
A-const-eval Area: Constant evaluation (MIR interpretation) A-destructors Area: Destructors (`Drop`, …) B-unstable Blocker: Implemented in the nightly compiler and unstable. C-tracking-issue Category: A tracking issue for an RFC or an unstable feature. S-tracking-needs-summary Status: It's hard to tell what's been done and what hasn't! Someone should do some investigation. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-lang Relevant to the language team, which will review and decide on the PR/issue.

Comments

@ecstatic-morse
Copy link
Contributor

ecstatic-morse commented Jun 11, 2020

This is a tracking issue for a more precise version of checking for drops in const contexts.

The feature gate for the issue is #![feature(const_precise_live_drops)].

With this feature enabled, drops are checked on slightly elaborated MIR. This makes the analysis that prevents dropping in const fn more accurate for code that does not actually drop anything: specifically, if the initial MIR contains an unnecessary call to drop that may be eliminated in elaboration, it can be accepted with const_precise_live_drops.

Internally, the library can also use this on a per-function basis with #[rustc_allow_const_fn_unstable(const_precise_live_drops)].

About tracking issues

Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.
Discussion comments will get marked as off-topic or deleted.
Repeated discussions on the tracking issue may lead to the tracking issue getting locked.

Steps

Blockers and Concerns:

@ecstatic-morse ecstatic-morse added C-tracking-issue Category: A tracking issue for an RFC or an unstable feature. A-const-eval Area: Constant evaluation (MIR interpretation) labels Jun 11, 2020
@jonas-schievink jonas-schievink added the T-lang Relevant to the language team, which will review and decide on the PR/issue. label Jun 11, 2020
@JohnTitor JohnTitor added the B-unstable Blocker: Implemented in the nightly compiler and unstable. label Jul 16, 2020
@RalfJung
Copy link
Member

Cc @rust-lang/wg-const-eval (which I think never happened for this feature or the PR)

@DoumanAsh
Copy link

Is there particular reason why this was moved to be a feature?
It is more or less bug fix to inaccurate drop detection.
So shouldn't it be already stable?
This would make it easier to create builders with generic parameters as we currently cannot mutate self

@oli-obk
Copy link
Contributor

oli-obk commented Mar 13, 2021

cc @rust-lang/lang I am nominating this feature for stabilization

citing @ecstatic-morse from the impl PR:

This isn't really a "feature" but a refinement to const-checking, and it's kind of non-obvious what the user opts in to with the feature gate, since drop elaboration is an implementation detail. A relevant precedent is #![feature(nll)], but that was much larger and more fundamental than this change would be.

This is also somewhat relevant to the stabilization of #![feature(const_if_match)]. Nothing in this PR is specifically related to branching in constants. For instance, it makes the example below compile as well. However, I expect users will commonly want to move out of Option::<T>::Some within a match, which won't work without this PR if T: !Copy

const _: Vec<i32> = {
    let vec_tuple = (Vec::new(),);
    vec_tuple.0
};

To clarify, this PR makes that code compile because previously we were dropping the vec_tuple which had type Option<(Vec,)>?

This code is functionally no different from the following, which currently works on stable because x is moved into the return place unconditionally.

const X: Option<Vec<i32>> = { let x = Vec::new(); x };

Const-checking only considers whole locals for const qualification, but drop elaboration is more precise: It works on projections as well. Because drop elaboration sees that all fields of the tuple have been moved from by the end of the initializer, no Drop terminator is left in the MIR, despite the fact that the tuple itself is never moved from.

@RalfJung
Copy link
Member

@oli-obk how do things look like in implementation complexity and the risk of locking us into a scheme that might be hard to maintain or make compatible with other future extensions?

@DoumanAsh

This comment has been minimized.

@oli-obk

This comment has been minimized.

@RalfJung
Copy link
Member

RalfJung commented Mar 15, 2021

@DoumanAsh due to the halting problem, rustc will always report "incorrect errors". It is impossible to predict at compiletime if e.g. drop will be executed when a piece of code is being run. Similarly, the borrow checker will always reject some correct programs. That's just a fact of programming languages. Neither of these are a bug.

So there's always a trade-off between analysis complexity and "incorrect errors", but even the best analysis will sometimes show "incorrect errors". My impression after looking at the PR here is that the analysis complexity is totally reasonable (it mostly implicitly reuses the analysis performed by drop elaboration), but this is still a decision we should make explicitly, not implicitly.

More precise drop analysis in const fn is a new feature, not a bugfix. (Just like, for example, NLL was a new feature, not a bugfix.)

@oli-obk Thanks. :) As you probably saw, I also left some questions in the PR that implemented the analysis.

@oli-obk
Copy link
Contributor

oli-obk commented Mar 15, 2021

I did see these comments, just didn't have time to look more deeply yet.

Cross posting from the comments on that PR:

I am guessing the difference between the needs_drop analysis, and drop elaboration's use of MaybeInitializedPlaces and MaybeUninitializedPlaces is the reason that this feature gate exists at all. We could probably rewrite drop elaboration in terms of the NeedsDrop qualif, which would (afaict) allow post_drop_elaboration to not do any checks except for looking for Drop terminators.

Of course such a change would insta-stabilize the feature gate from this PR without any reasonable way to avoid said stabilization. So I'm tempted to stabilize the feature but leave a FIXME on this function to merge its qualif checks into elaborate_drops

@RalfJung
Copy link
Member

We could probably rewrite drop elaboration in terms of the NeedsDrop qualif, which would (afaict) allow post_drop_elaboration to not do any checks except for looking for Drop terminators.
Of course such a change would insta-stabilize the feature gate from this PR without any reasonable way to avoid said stabilization.

Even worse, it would change drop elaboration, which is a rather tricky part of the compiler.^^ That should be done with utmost care.

@nikomatsakis
Copy link
Contributor

r? @nikomatsakis

@Mark-Simulacrum
Copy link
Member

Dropping the nomination here as it's waiting on Niko; that's tracked in the language team action item list.

@inquisitivecrystal
Copy link
Contributor

What remains to be done before this can be stabilized?

@ecstatic-morse
Copy link
Contributor Author

ecstatic-morse commented Jul 29, 2021

Hey y'all. I'll try to summarize exactly what this feature gate does to help inform a decision on stabilization. Oli and Ralf probably already know this information, and can skip the next section.

Background

Currently, it is forbidden to run custom Drop implementations during const-eval. This is because Drop::drop is a trait method, and there's no (stable) way to declare that trait methods are const-safe. MIR uses a special terminator (Drop) to indicate that something is dropped. When building the MIR initially, a Drop terminator is inserted for every variable that has drop glue (ty.needs_drop()). Typically, const-checks are run on the newly built MIR. This is to prevent MIR optimizations from eliminating or transforming code that would fail const-checking into code that passes it, which would make compilation success dependent on MIR optimizations. In general, we try very hard to avoid this.

However, some Drop terminators that we generate during MIR building will never actually execute. For example,

  • A struct has a field with a custom Drop impl (e.g. Foo(Box<i32>)), but that field is always moved out of before the variable goes out of scope.
  • An enum has a variant with a custom Drop impl, but we are in a match arm for a different variant without a custom Drop impl.

We rely on a separate MIR pass, elaborate_drops, to remove these terminators prior to codegen (it also does some extra stuff, like adding dynamic drop flags). This happens pretty early in the optimization pipeline, but it is still an optimization, so it runs after const-checking. As a result, the stable version of const-checking sees some Drop terminators that will never actually run, and raises an error.

Current Implementation

#![feature(const_precise_live_drops)] defers the Drop terminator const-checks until after drop elaboration is run. As I mention above, this is unusual, since it makes compilation success dependent on the result of an optimization. On the other hand,

  • Drop elaboration doesn't change very often. Unlike some other optimizations, it is conceptually pretty simple to determine whether a Drop terminator is frivolous along a given code path. It depends solely on whether some set of move paths are initialized at that point in the CFG.
  • It is always profitable to remove drop terminators from the MIR when we can. This makes it unlikely that we will ever want to "dial back" drop-elaboration.
  • Despite being conceptually straightforward, drop elaboration is not free to compute and having two separate implementations of it (one in const-checking and one in the optimization pipeline) is both inefficient and bad for maintainability.
  • If there is bug in drop elaboration that causes us to wrongly eliminate Drops, wrongly accepting some const fns will be the least of our worries, relatively speaking.

For these reasons, I chose the current implementation strategy. I realize that none of these arguments are dispositive, and I don't think it's unreasonable to gate stabilization of this feature on reimplementing the relevant bits of drop elaboration inside const-checking, although I still think it's overly conservative.

Besides that big question, I think there were also some concerns from const-eval members around the use of the NeedsDrop qualif and test coverage (e.g. #71824 (comment)). I'll try to answer those so they can provide a recommendation.

@inquisitivecrystal
Copy link
Contributor

I'm not educated on the details, but it would be super nice to see this stabilized in some form. There are a comparatively large number of new APIs that rely on this. For examples, see issues #76654, #82814, and PR #84087, the last of which is an approved stabilization PR that can't be merged until this is stabilized.

That's why I was checking in on the progress towards stabilization a few days ago. I'm sorry about that, by the way. I know that that sort of message can be annoying, but I wanted to know there if there was anything I could do to help move this along.

@inquisitivecrystal inquisitivecrystal added the A-destructors Area: Destructors (`Drop`, …) label Jul 29, 2021
@ecstatic-morse
Copy link
Contributor Author

ecstatic-morse commented Jul 29, 2021

The fact that the main blocker (and why this was feature gated in the first place) is the implementation makes it somewhat unusual (see #71824 (comment)). That makes it more the domain of the compiler-team rather than the lang-team. Niko reviewed #71824 and is assigned to this issue, but I'm hesitant to ping them specifically unless their expertise is required.

So, if you want to see this stabilized I would figure out some process for getting consent from the compiler team. I think they use the MCP process for this exclusively? Oli is a member, so there's at least one potential sponsor. The team might require documenting the drop-elaboration/const-checking dependency in the dev-guide and maybe the module itself, which I'm happy to do if asked. After that, I can write a stabilization report with some examples and we can do lang-team FCP (assuming any lingering concerns from @rust-lang/wg-const-eval have been addressed).

I'm, uh, not great at navigating bureaucratic systems with many veto points, so if you want to drive this forward your help would be greatly appreciated. However, unless we end up reimplementing drop-elaboration in const-checking like I mention above, I don't think much of the remaining work is technical.

@oli-obk
Copy link
Contributor

oli-obk commented Jul 30, 2021

I think your summary post contains most of what we need for a stabilization report (full instructions here). We can just mark this issue as requiring sign-off from both T-compiler and T-lang and do all of this at once.

@clarfonthey
Copy link
Contributor

Is this still blocked? It appears that #91009 has been closed.

The ability to make unwrap work in const fn would help a lot with simplifying a lot of existing code, since it can help replace a lot of manual match statements with unwraps.

@RalfJung
Copy link
Member

RalfJung commented Jan 14, 2023 via email

@clarfonthey
Copy link
Contributor

Ah! Is there a dedicated issue open for that that can be linked in the description, or is that just a known issue at the moment?

@RalfJung
Copy link
Member

It's not a very concrete issue, and I don't think is tracked anywhere explicitly outside of this tracking issue.

@clarfonthey
Copy link
Contributor

That's fair. As is expected with all these const features, something subtle and complicated lurks in the depths that makes it hard to finish up.

I was kinda hopeful that this was mostly done, but alas.

@RalfJung
Copy link
Member

It might be, I am honestly not familiar enough with drop elaboration to really evaluate the trade-offs here.

I hope someone else reading along has some good ideas for what can be done before we ask the lang team to discuss this again.

@RalfJung
Copy link
Member

RalfJung commented Mar 18, 2024

One thing I did miss when writing the comments above is #91410: this feature no longer relies on full drop elaboration, just a lightweight version of dead drop removal.

Still, doing this somewhere in the middle of our MIR pipeline does not feel great. Instead maybe it is possible to run the analysis that remove_uninit_drops performs during const-checking so that we can just ask "can this drop ever happen" rather than manifesting the result as explicit MIR? Then this could just happen during regular const checking.

(Borrowck already does something like that, doesn't it? If someone knows a borrowck expert, please point them to this thread, maybe they can help :)

Also see Zulip

matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Apr 6, 2024
…lstrieb

Stabilize const Atomic*::into_inner

Partial stabilization for rust-lang#78729, for which the FCP has already completed.

The other `into_inner` functions in that tracking issue (`UnsafeCell`, `Cell`, `RefCell`) are blocked on rust-lang#73255 for now.

```console
error[E0493]: destructor of `UnsafeCell<T>` cannot be evaluated at compile-time
    --> library/core/src/cell.rs:2076:29
     |
2076 |     pub const fn into_inner(self) -> T {
     |                             ^^^^ the destructor for this type cannot be evaluated in constant functions
2077 |         self.value
2078 |     }
     |     - value is dropped here
```
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Apr 7, 2024
Rollup merge of rust-lang#123522 - dtolnay:constatomicintoinner, r=Nilstrieb

Stabilize const Atomic*::into_inner

Partial stabilization for rust-lang#78729, for which the FCP has already completed.

The other `into_inner` functions in that tracking issue (`UnsafeCell`, `Cell`, `RefCell`) are blocked on rust-lang#73255 for now.

```console
error[E0493]: destructor of `UnsafeCell<T>` cannot be evaluated at compile-time
    --> library/core/src/cell.rs:2076:29
     |
2076 |     pub const fn into_inner(self) -> T {
     |                             ^^^^ the destructor for this type cannot be evaluated in constant functions
2077 |         self.value
2078 |     }
     |     - value is dropped here
```
@nikomatsakis
Copy link
Contributor

nikomatsakis commented Apr 13, 2024 via email

matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Sep 15, 2024
Stabilize `&mut` (and `*mut`) as well as `&Cell` (and `*const Cell`) in const

This stabilizes `const_mut_refs` and `const_refs_to_cell`. That allows a bunch of new things in const contexts:
- Mentioning `&mut` types
- Creating `&mut` and `*mut` values
- Creating `&T` and `*const T` values where `T` contains interior mutability
- Dereferencing `&mut` and `*mut` values (both for reads and writes)

The same rules as at runtime apply: mutating immutable data is UB. This includes mutation through pointers derived from shared references; the following is diagnosed with a hard error:
```rust
#[allow(invalid_reference_casting)]
const _: () = {
    let mut val = 15;
    let ptr = &val as *const i32 as *mut i32;
    unsafe { *ptr = 16; }
};
```

The main limitation that is enforced is that the final value of a const (or non-`mut` static) may not contain `&mut` values nor interior mutable `&` values. This is necessary because the memory those references point to becomes *read-only* when the constant is done computing, so (interior) mutable references to such memory would be pretty dangerous. We take a multi-layered approach here to ensuring no mutable references escape the initializer expression:
- A static analysis rejects (interior) mutable references when the referee looks like it may outlive the current MIR body.
- To be extra sure, this static check is complemented by a "safety net" of dynamic checks. ("Dynamic" in the sense of "running during/after const-evaluation, e.g. at runtime of this code" -- in contrast to "static" which works entirely by looking at the MIR without evaluating it.)
  - After the final value is computed, we do a type-driven traversal of the entire value, and if we find any `&mut` or interior-mutable `&` we error out.
  - However, the type-driven traversal cannot traverse `union` or raw pointers, so there is a second dynamic check where if the final value of the const contains any pointer that was not derived from a shared reference, we complain. This is currently a future-compat lint, but will become an ICE in rust-lang#128543. On the off-chance that it's actually possible to trigger this lint on stable, I'd prefer if we could make it an ICE before stabilizing const_mut_refs, but it's not a hard blocker. This part of the "safety net" is only active for mutable references since with shared references, it has false positives.

Altogether this should prevent people from leaking (interior) mutable references out of the const initializer.

While updating the tests I learned that surprisingly, this code gets rejected:
```rust
const _: Vec<i32> = {
    let mut x = Vec::<i32>::new(); //~ ERROR destructor of `Vec<i32>` cannot be evaluated at compile-time
    let r = &mut x;
    let y = x;
    y
};
```
The analysis that rejects destructors in `const` is very conservative when it sees an `&mut` being created to `x`, and then considers `x` to be always live. See [here](rust-lang#65394 (comment)) for a longer explanation. `const_precise_live_drops` will solve this, so I consider this problem to be tracked by rust-lang#73255.

Cc `@rust-lang/wg-const-eval` `@rust-lang/lang`
Cc rust-lang#57349
Cc rust-lang#80384
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Sep 15, 2024
Rollup merge of rust-lang#129195 - RalfJung:const-mut-refs, r=fee1-dead

Stabilize `&mut` (and `*mut`) as well as `&Cell` (and `*const Cell`) in const

This stabilizes `const_mut_refs` and `const_refs_to_cell`. That allows a bunch of new things in const contexts:
- Mentioning `&mut` types
- Creating `&mut` and `*mut` values
- Creating `&T` and `*const T` values where `T` contains interior mutability
- Dereferencing `&mut` and `*mut` values (both for reads and writes)

The same rules as at runtime apply: mutating immutable data is UB. This includes mutation through pointers derived from shared references; the following is diagnosed with a hard error:
```rust
#[allow(invalid_reference_casting)]
const _: () = {
    let mut val = 15;
    let ptr = &val as *const i32 as *mut i32;
    unsafe { *ptr = 16; }
};
```

The main limitation that is enforced is that the final value of a const (or non-`mut` static) may not contain `&mut` values nor interior mutable `&` values. This is necessary because the memory those references point to becomes *read-only* when the constant is done computing, so (interior) mutable references to such memory would be pretty dangerous. We take a multi-layered approach here to ensuring no mutable references escape the initializer expression:
- A static analysis rejects (interior) mutable references when the referee looks like it may outlive the current MIR body.
- To be extra sure, this static check is complemented by a "safety net" of dynamic checks. ("Dynamic" in the sense of "running during/after const-evaluation, e.g. at runtime of this code" -- in contrast to "static" which works entirely by looking at the MIR without evaluating it.)
  - After the final value is computed, we do a type-driven traversal of the entire value, and if we find any `&mut` or interior-mutable `&` we error out.
  - However, the type-driven traversal cannot traverse `union` or raw pointers, so there is a second dynamic check where if the final value of the const contains any pointer that was not derived from a shared reference, we complain. This is currently a future-compat lint, but will become an ICE in rust-lang#128543. On the off-chance that it's actually possible to trigger this lint on stable, I'd prefer if we could make it an ICE before stabilizing const_mut_refs, but it's not a hard blocker. This part of the "safety net" is only active for mutable references since with shared references, it has false positives.

Altogether this should prevent people from leaking (interior) mutable references out of the const initializer.

While updating the tests I learned that surprisingly, this code gets rejected:
```rust
const _: Vec<i32> = {
    let mut x = Vec::<i32>::new(); //~ ERROR destructor of `Vec<i32>` cannot be evaluated at compile-time
    let r = &mut x;
    let y = x;
    y
};
```
The analysis that rejects destructors in `const` is very conservative when it sees an `&mut` being created to `x`, and then considers `x` to be always live. See [here](rust-lang#65394 (comment)) for a longer explanation. `const_precise_live_drops` will solve this, so I consider this problem to be tracked by rust-lang#73255.

Cc `@rust-lang/wg-const-eval` `@rust-lang/lang`
Cc rust-lang#57349
Cc rust-lang#80384
jhpratt added a commit to jhpratt/rust that referenced this issue Oct 18, 2024
Partially stabilize const_pin

Tracking issue rust-lang#76654.

Eight of these methods can be made const-stable. The remainder are blocked on rust-lang#73255.
jhpratt added a commit to jhpratt/rust that referenced this issue Oct 18, 2024
Partially stabilize const_pin

Tracking issue rust-lang#76654.

Eight of these methods can be made const-stable. The remainder are blocked on rust-lang#73255.
jhpratt added a commit to jhpratt/rust that referenced this issue Oct 18, 2024
Partially stabilize const_pin

Tracking issue rust-lang#76654.

Eight of these methods can be made const-stable. The remainder are blocked on rust-lang#73255.
jieyouxu added a commit to jieyouxu/rust that referenced this issue Oct 18, 2024
Partially stabilize const_pin

Tracking issue rust-lang#76654.

Eight of these methods can be made const-stable. The remainder are blocked on rust-lang#73255.
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Oct 18, 2024
Rollup merge of rust-lang#130136 - GKFX:stabilize-const-pin, r=dtolnay

Partially stabilize const_pin

Tracking issue rust-lang#76654.

Eight of these methods can be made const-stable. The remainder are blocked on rust-lang#73255.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-const-eval Area: Constant evaluation (MIR interpretation) A-destructors Area: Destructors (`Drop`, …) B-unstable Blocker: Implemented in the nightly compiler and unstable. C-tracking-issue Category: A tracking issue for an RFC or an unstable feature. S-tracking-needs-summary Status: It's hard to tell what's been done and what hasn't! Someone should do some investigation. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-lang Relevant to the language team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests