Consider improving layout of this enum

[carllerche]

I think the following snippet should be possible (but it fails currently). Since &'static MyTrait can't be 0, if the first half of the layout is 0, then the enum must be the Boxed variant.

use std::mem;

pub trait MyTrait {
    fn zomg(&self);
}

enum Foo {
    Inline(&'static MyTrait, [usize; 2]),
    Boxed(Box<MyTrait>),
}

pub fn main() {
    assert_eq!(2 * mem::size_of::<&'static MyTrait>(), mem::size_of::<Foo>());
}

[abonander]

I'm not sure I follow. &MyTrait and Box<MyTrait> are the same width, neither can be zero, and the second half of the enum could be [0us; 2] or empty space. I don't see a way to deduce the variant without a discriminator. Maybe I'm missing something.

[Diggsey]

The layout of the Boxed variant could theoretically be altered such that Box<MyTrait> occupies the same space as the [usize; 2] from the Inline variant. The first usize bytes could then be used as the discriminator, where 0 => Boxed, _ => Inline.

I'm not sure how worthwhile this is, since it's very specific, and the algorithm to calculate the best layout for every possible scenario is at the very least non-obvious. If there is ever a rust ABI, then it would be preferable to have layouts controlled by predictable, easy to describe rules.

[dotdash]

If you put the box in the same space as the array, i.e. in the second half,
then you can identify the Boxed variant by the first half being set to zero.
Am 12.04.2015 23:17 schrieb "Austin Bonander" notifications@github.com:

I'm not sure I follow. &MyTrait and Box are the same width,
neither can be zero, and the second half of the enum could be [0us; 2] or
empty space. I don't see a way to deduce the variant without a
discriminator. Maybe I'm missing something.

—
Reply to this email directly or view it on GitHub
#24041 (comment).

[Marwes]

Its not necessarily a strictly better idea to put the box in the same space as the array though.

fn zomg(foo: &Foo)
    match *foo {
        Inline(ref t, _) => t.zomg(),
        Boxed(ref t) => t.zomg()
}

In a function such as above, since the the data and vtable are in the same position it could be optimized to just load those positions and call the function, avoiding the discriminant test completely. I don't know if llvm performs that sort of optimization though.

[steveklabnik]

Triage: thread 'main' panicked at 'assertion failed: (left == right) (left: 32, right: 40)',

[nox]

This could be done with @eddyb's niche-filling optimisation. It currently only applies to enums with a single dataful variant, but we could make it apply to the largest variant of the enum instead. I am pretty sure this would also improve the layout of a bunch of enums in Stylo.

I've thought about this a bit and I think the niche-filling optimisation should apply to the largest variant of any repr(Rust) enum, if there is no other variant that is as large.

Cc @rust-lang/wg-codegen

[eddyb]

This seems to (now) be a duplicate of #46213.

[dtolnay]

Closing in favor of the more general #46213.