inventory::submit!() doesn't work unless code from downstream crate is actually used

[billyrieger]

Here's my setup: I have a workspace with two crates, primary and secondary. primary defines a struct Foo that's registered with inventory::collect!(). secondary registers an instance of Foo using inventory::submit!(). In secondary/tests/test.rs, I attempt to iterate over all registered Foo instances.

$ exa -T

.
├── Cargo.lock
├── Cargo.toml
├── primary
│  ├── Cargo.toml
│  └── src
│     └── lib.rs
└── secondary
   ├── Cargo.toml
   ├── src
   │  └── lib.rs
   └── tests
      └── test.rs

primary/src/lib.rs

#[derive(Debug)]
pub struct Foo {
    pub name: &'static str,
}

inventory::collect!(Foo);

secondary/src/lib.rs

inventory::submit!(primary::Foo { name: "asdf" });

secondary/tests/test.rs

#[test]
fn test() {
    for foo in inventory::iter::<primary::Foo> {
        println!("{:?}", foo);
    }
}

When running cargo test -- --nocapture, I expect to see Foo { name: "asdf" } in the output. However, the test has no output. I think the problem is that the test is compiled as a separate crate and since it doesn't refer to secondary at all, the code to register the Foo instance is not run.

From this forum post, I discovered a workaround: by adding a dummy function pub fn dummy() {} to the secondary crate, and calling that function from the test, the output is what I expect.

I was wondering if there are any other ways around this issue or if you could shed some light on why it occurs. Thanks!

[dtolnay]

This minimizes to:

#!/bin/bash

rustc <<<'
    #[used]
    #[link_section = ".ctors"]
    static STATIC: unsafe extern "C" fn() = print_msg;
    unsafe extern "C" fn print_msg() {
        use core::ffi::c_void;
        extern {
            fn write(fd: i32, buf: *const c_void, count: usize) -> isize;
        }
        write(
            1i32,
            "CTOR CALLED\n".as_ptr() as *const c_void,
            12usize,
        );
    }
    pub fn f() {}
    ' \
    --edition=2018 \
    --crate-name dep \
    --crate-type lib \
    --emit=link \
    /dev/stdin

rustc <<<'
    fn main() {
        dep::f();
    }
    ' \
    --edition=2018 \
    --crate-name repro \
    --emit=link \
    --extern dep=libdep.rlib \
    /dev/stdin

./repro

If you comment out dep::f() then print_msg does not end up in the binary. I think this is a rustc bug. Anything with #[used] and #[link_section = "..."] needs to go in the binary.

[billyrieger]

This seems to be the same issue: rust-lang/rust#47384.

I checked that STATIC does actually end up in libdep.rlib:

$ nm -C libdep.rlib   

dep.dep.3a1fbbbh-cgu.0.rcgu.o:
0000000000000000 V DW.ref.rust_eh_personality
0000000000000000 r GCC_except_table0
                 U rust_eh_personality
                 U write
0000000000000000 T dep::f
0000000000000000 d dep::STATIC
0000000000000000 t dep::print_msg
                 U core::str::<impl str>::as_ptr

so it appears to be a linker problem.

One suggested solution is from rust-lang/rust#47384 (comment):

The fix is to add an EXTERN(STATIC); declaration to the linker script, which tells the linker to keep looking for a STATIC symbol, even if all other symbols are already resolved.

However I'm not sure that is applicable in this case as no linker script is being used.

[idubrov]

Seeing the same issue. Is any workaround available?

Seems like changing crate-type to "dylib" helps my case (presumably, because linker would keep all symbols for the dynamically loadable library), but are there any other workaround?

[mathstuf]

What platform is this on? macOS has some odd behaviors in that static initializers are not guaranteed to run until something in the object file is actually needed. If nothing is triggering the object file (crate) with the initializer to actually get resolved/looked at by the linker, it may not run.

I don't know if there's a plausibly implementable solution for this in Rust. We do this reliably in a C++ codebase at work, but it involves code generation to basically inject a trigger for the static initializer into any TUs including a static initializer-needing header. I don't think there's an equivalent in Rust.

[pchickey]

Just wanted to note- I just ran into this issue, and it appears that (at least with rustc 1.43 on linux) the restriction is even narrower - the module in the downstream crate must be actually used, not just the crate itself:

#!/bin/bash

rustc <<<'
pub mod inner {
    #[used]
    #[link_section = ".ctors"]
    static STATIC: unsafe extern "C" fn() = print_msg;
    unsafe extern "C" fn print_msg() {
        use core::ffi::c_void;
        extern {
            fn write(fd: i32, buf: *const c_void, count: usize) -> isize;
        }
        write(
            1i32,
            "CTOR CALLED\n".as_ptr() as *const c_void,
            12usize,
        );
    }
    pub fn g() {}
}
pub fn f() {}
    ' \
    --edition=2018 \
    --crate-name dep \
    --crate-type lib \
    --emit=link \
    /dev/stdin

rustc <<<'
    fn main() {
        // comment out this call and the constructor won't be called:
        dep::inner::g();
        dep::f();
    }
    ' \
    --edition=2018 \
    --crate-name repro \
    --emit=link \
    --extern dep=libdep.rlib \
    /dev/stdin

./repro

[mathstuf]

Is it the linker or the compiler culling the symbols from the executable?

[jgarvin]

My experience in C++ is that build tools like bazel usually expose an option you can set on libraries to link even if symbols are unused. In bazel the option is called always_link=1. See the gcc option discussed here: https://stackoverflow.com/q/14116420/50385

Presumably there is some way to get rustc to give the same option to the linker?

[dtolnay]

This is fixed in rustc 1.62.0 / inventory 0.3.0.