Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dead variant removal in #[repr(C)] enums #500

Closed
GoldsteinE opened this issue Mar 25, 2024 · 4 comments
Closed

Dead variant removal in #[repr(C)] enums #500

GoldsteinE opened this issue Mar 25, 2024 · 4 comments

Comments

@GoldsteinE
Copy link

GoldsteinE commented Mar 25, 2024

Consider this enum:

#[repr(C)]
enum Fun<T> {
    Fun(T),
    // maybe some other variants here
}

For unsafe code it could be useful to assume that Fun<T> is layout-compatible with Fun<MaybeUninit<T>> for any choice of T. That’s not currently the case: if T is uninhabited, Fun<T> would be zero-sized (or, more generally, the ::Fun variant will not affect enum size), but Fun<MaybeUninit<T>> wouldn’t.

Reference currently documents the above enum as being rough equivalent to something like

#[repr(C)]
enum FunDiscriminant {
    Fun,
}

#[repr(C)]
union FunData<T> {
    fun: T,
}

#[repr(C)]
struct Fun<T> {
    discriminant: FunDiscriminant,
    data: FunData<T>,
}

That doesn’t hold for uninhabited types: this “desugaring” preserves the invariant above.

I think we should do one of the following:

  1. Either dead variant removal should be disabled for #[repr(C)] enums;
  2. or it should be documented in the reference (I can write the patch if this option is selected).

Related: #443
Playground demonstrating the issue: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=d2995ff99cea9c2e60d28cbf0f970176

@RalfJung
Copy link
Member

It's interesting what happens when you remove the repr(C) from FunDiscriminant:

[src/main.rs:29:5] std::mem::size_of::<Fun<Void>>() = 0
[src/main.rs:30:5] std::mem::size_of::<Fun<std::mem::MaybeUninit<Void>>>() = 4
[src/main.rs:31:5] std::mem::size_of::<desugared::Fun<Void>>() = 0
[src/main.rs:32:5] std::mem::size_of::<desugared::Fun<std::mem::MaybeUninit<Void>>>() = 0

That's more what I expected: single-variant enums can have a zero-sized discriminant.

Also interestingly, zero-variant enums with repr(C) are rejected.


My first reaction is that this is a bug in rustc, we shouldn't do dead variant removal on repr(C) enums.

@chorman0773
Copy link
Contributor

chorman0773 commented Mar 25, 2024

I agree with that. Even if the variant is uninhabited, we guarantee that there's a target-dependant discriminant field in the layout of a repr(C) enum. The same will be true of repr(Int), and repr(C,Int) (where the discriminant field is fixed to Int).

@GoldsteinE
Copy link
Author

Assuming that’s a rustc bug, I sent a patch that fixes it.

matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Jul 4, 2024
…s, r=oli-obk

Disable dead variant removal for `#[repr(C)]` enums.

This prevents removing dead branches from a `#[repr(C)]` enum (they now get discriminants allocated as if they were inhabited).

Implementation notes: ABI of something like

```rust
#[repr(C)]
enum Foo {
    Foo(!),
}
```

is still `Uninhabited`, but its layout is now computed as if all the branches were inhabited.
This seemed to me like a proper way to do it, especially given that ABI sanity check explicitly asserts that type-level uninhabitedness implies ABI uninhabitedness.

This probably needs some sort of FCP (given that it changes `#[repr(C)]` layout, which is a stable guarantee), but I’m not sure how to call for one or which team is the most relevant.

See rust-lang/unsafe-code-guidelines#500.
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Jul 4, 2024
Rollup merge of rust-lang#123043 - GoldsteinE:fix/repr-c-dead-branches, r=oli-obk

Disable dead variant removal for `#[repr(C)]` enums.

This prevents removing dead branches from a `#[repr(C)]` enum (they now get discriminants allocated as if they were inhabited).

Implementation notes: ABI of something like

```rust
#[repr(C)]
enum Foo {
    Foo(!),
}
```

is still `Uninhabited`, but its layout is now computed as if all the branches were inhabited.
This seemed to me like a proper way to do it, especially given that ABI sanity check explicitly asserts that type-level uninhabitedness implies ABI uninhabitedness.

This probably needs some sort of FCP (given that it changes `#[repr(C)]` layout, which is a stable guarantee), but I’m not sure how to call for one or which team is the most relevant.

See rust-lang/unsafe-code-guidelines#500.
@GoldsteinE
Copy link
Author

The fix is stable now, so I think this could be closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants