statics in closures returned using impl Trait: undefined symbols

[TimNN]

I have Code like the following:

impl Docker {
    // #[inline] // marking this as inline fixes the problem
    pub fn new(handle: &Handle) -> impl Future<Item = Docker, Error = io::Error> {
        let client = hyper::Client::new(&handle);

        let docker = Docker {
            client: client,
        };

        // docker.version() returns a `impl Future`
        docker.version().map(|version| {
            println!("connected to docker {:?}", version);
            docker
        })
    }
}

When I'm calling Docker::new from another crate I get the following linker error:

  = note: Undefined symbols for architecture x86_64:
            "docker_run::Docker::new::_$u7b$$u7b$closure$u7d$$u7d$::__STATIC_FMTSTR::h39c0733cea565f9a", referenced from:
                docker_run::Docker::new::_$u7b$$u7b$closure$u7d$$u7d$::h260aa00d780d89c7 in docker_run-8e8c301168781829.0.o

Related symbols from the linked rlib:

$ objdump -t .../libdocker_run-b777eed0df4fdfc5.rlib | grep -F '6Docker3new28_$u7b$$u7b$closure$'
00000000000d5950 l       0e SECT   01 0000 [.text] __ZN10docker_run6Docker3new28_$u7b$$u7b$closure$u7d$$u7d$17h2a2bb6ebe6df45e4E
00000000000f96a0 l       0e SECT   05 0000 [.const_data] __ZN10docker_run6Docker3new28_$u7b$$u7b$closure$u7d$$u7d$15__STATIC_FMTSTR17h39c0733cea565f9aE

And from the object file in question:

objdump -t .../docker_run-8e8c301168781829.0.o  | grep -F '6Docker3new28_$u7b$$u7b$closure$'
00000000000033b0 l       0e SECT   01 0000 [.text] __ZN10docker_run6Docker3new28_$u7b$$u7b$closure$u7d$$u7d$17h260aa00d780d89c7E
0000000000000000 g       01 UND    00 0000 __ZN10docker_run6Docker3new28_$u7b$$u7b$closure$u7d$$u7d$15__STATIC_FMTSTR17h39c0733cea565f9aE

I hope this is enough for someone to get an idea, where the problem might be, other wise I can see about publishing a smaller testcase.

cc @michaelwoerister I guess this is related to the collector, which I believe is your area of expertise.

[michaelwoerister]

Yes, this could well be the collector not finding the instance.

Nominated for priority assignment.

[pnkfelix]

@TimNN marking as P-medium under the assumption that this is relatively easy to work-around. Let us know if I misunderstand

[Mark-Simulacrum]

@eddyb, perhaps you could comment here -- is this the same issue as #35870? It looks similar, at least.

[eddyb]

@Mark-Simulacrum Yes, they're both statics.

[Nemo157]

This also appears to affect using panic!() inside a closure in an impl Trait because of the implicit statics for the file location:

= note: Undefined symbols for architecture x86_64:
    ...::_$u7b$$u7b$closure$u7d$$u7d$::_FILE_LINE_COL::h989fc96fbef4eca7

[michaelwoerister]

I have not yet managed to reproduce this. I must be doing something wrong.

[Nemo157]

@michaelwoerister try https://github.com/Nemo157/impl-trait-across-crates

[michaelwoerister]

@Nemo157 Thanks!

[Nemo157]

It seems at some point between nightly-2017-07-08 and nightly-2017-08-09 this error stopped occurring for my testcase in https://github.com/Nemo157/impl-trait-across-crates, now both testcases there produce the error in #43135.

[michaelwoerister]

The root cause here some that compiler does not properly handle closures that are returned from a function via -> impl Fn(). The MIR of the closure will be exported to crate metadata (because that is done for all closures) but things referenced by the closure are not. When the compiler then tries to instantiate the closure's TransItem in a downstream crate, it fails because it can neither link to nor create a local copy of these referenced things.

A straightforward solution to this problem would be to fix the privacy pass, as @eddyb pointed out.

[eddyb]

TyClosure is missing in the list of types to mark as reached I believe.

[mhristache]

Using #[inline] does not seem to always help. See #43135 (comment) for details.

Could this issue be prioritized, please?

Thank you

[mhristache]

I tried the latest nightly (which I guess includes #43857) but compilation never ends. It seem to get stuck in :

INFO:rustc_metadata::cstore: topological ordering: [CrateNum(2), CrateNum(3), CrateNum(5), CrateNum(4), CrateNum(7), CrateNum(6), CrateNum(8), CrateNum(9), CrateNum(10), CrateNum(11), CrateNum(1), CrateNum(61), CrateNum(56), CrateNum(20), CrateNum(15), CrateNum(16), CrateNum(14), CrateNum(19), CrateNum(18), CrateNum(17), CrateNum(22), CrateNum(21), CrateNum(23), CrateNum(24), CrateNum(25), CrateNum(27), CrateNum(26), CrateNum(28), CrateNum(29), CrateNum(32), CrateNum(34), CrateNum(33), CrateNum(35), CrateNum(31), CrateNum(36), CrateNum(37), CrateNum(30), CrateNum(39), CrateNum(40), CrateNum(41), CrateNum(42), CrateNum(38), CrateNum(13), CrateNum(59), CrateNum(60), CrateNum(62), CrateNum(58), CrateNum(57), CrateNum(55), CrateNum(49), CrateNum(43), CrateNum(68), CrateNum(51), CrateNum(52), CrateNum(50), CrateNum(45), CrateNum(46), CrateNum(47), CrateNum(44), CrateNum(65), CrateNum(66), CrateNum(67), CrateNum(64), CrateNum(63), CrateNum(53), CrateNum(48), CrateNum(54), CrateNum(12)]

[michaelwoerister]

@maximih Thanks for giving it a try! That's interesting. What test case did you use exactly?

[mhristache]

I have a test crate (crate A) which is using another crate (crate B). Crate B has several methods with return type impl Future<>. I can build crate B just fine but I cannot build crate A (it gets stuck and never ends).

Before #43857, I used to get the error in #43135 when building crate A.

[mhristache]

@michaelwoerister I cannot reproduce it after a cargo clean && cargo update && cargo build. I will open a new issue if I see it again. I think this and the related issues should be closed.

Thanks for your time!

[Enselic]

I think this and the related issues should be closed.

Triage: All related issues already seems closed, so let's close this one too.