Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Static linking needs to include openssl somehow. #25

Closed
emk opened this issue Apr 19, 2019 · 6 comments
Closed

Static linking needs to include openssl somehow. #25

emk opened this issue Apr 19, 2019 · 6 comments

Comments

@emk
Copy link

emk commented Apr 19, 2019

Hello! Thank you for maintaining pq-sys. I've used it in a lot of projects, and it works very nicely.

I'm the maintainer of rust-musl-builder, which makes it possible to statically link Rust applications using the postgres, diesel and openssl crates. (It's apparently pretty popular, with over 450 thousand downloads on Docker Hub.)

Recently, some users of rust-musl-builder have been encountering a linking problem with pq-sys. This also seems to affect users of muslrust, the other cross-building library:

  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-Wl,--eh-frame-hdr" "-m64" "-nostdlib" "/home/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/crt1.o" "/home/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/crti.o" "-L" "/home/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/using_diesel-94aa5cadffcea30e.using_diesel.dqcexic7-cgu.0.rcgu.o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/using_diesel-94aa5cadffcea30e.using_diesel.dqcexic7-cgu.1.rcgu.o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/using_diesel-94aa5cadffcea30e.using_diesel.dqcexic7-cgu.10.rcgu.o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/using_diesel-94aa5cadffcea30e.using_diesel.dqcexic7-cgu.11.rcgu.o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/using_diesel-94aa5cadffcea30e.using_diesel.dqcexic7-cgu.12.rcgu.o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/using_diesel-94aa5cadffcea30e.using_diesel.dqcexic7-cgu.13.rcgu.o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/using_diesel-94aa5cadffcea30e.using_diesel.dqcexic7-cgu.14.rcgu.o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/using_diesel-94aa5cadffcea30e.using_diesel.dqcexic7-cgu.15.rcgu.o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/using_diesel-94aa5cadffcea30e.using_diesel.dqcexic7-cgu.2.rcgu.o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/using_diesel-94aa5cadffcea30e.using_diesel.dqcexic7-cgu.3.rcgu.o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/using_diesel-94aa5cadffcea30e.using_diesel.dqcexic7-cgu.4.rcgu.o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/using_diesel-94aa5cadffcea30e.using_diesel.dqcexic7-cgu.5.rcgu.o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/using_diesel-94aa5cadffcea30e.using_diesel.dqcexic7-cgu.6.rcgu.o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/using_diesel-94aa5cadffcea30e.using_diesel.dqcexic7-cgu.7.rcgu.o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/using_diesel-94aa5cadffcea30e.using_diesel.dqcexic7-cgu.8.rcgu.o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/using_diesel-94aa5cadffcea30e.using_diesel.dqcexic7-cgu.9.rcgu.o" "-o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/using_diesel-94aa5cadffcea30e" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/using_diesel-94aa5cadffcea30e.3n7t9phqceru8m8i.rcgu.o" "-Wl,--gc-sections" "-no-pie" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps" "-L" "/home/rust/src/target/release/deps" "-L" "/home/rust/src/target/x86_64-unknown-linux-musl/release/build/libsqlite3-sys-ac3cf7454f406832/out" "-L" "/usr/local/musl/lib" "-L" "/usr/local/musl/lib/" "-L" "/home/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib" "-Wl,-Bstatic" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/libopenssl-0de50740ee181784.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/libopenssl_sys-89c6410693b927d6.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/liblibc-6265db6daf6f0d94.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/liblazy_static-5cac84b776fcd98b.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/libforeign_types-23f459ebafab2c61.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/libforeign_types_shared-1eba4aeaa81c6d19.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/libdiesel-c89a5a6bebad19a9.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/liblibsqlite3_sys-818909750b3afa36.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/libpq_sys-dc85b10baeb00fc3.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/libbyteorder-9272cb6794a9382c.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/libbitflags-b8224b03590c6c21.rlib" "/home/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libstd-af9362ed5d81a840.rlib" "/home/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libpanic_unwind-4d55a38564aae54a.rlib" "/home/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libbacktrace_sys-f8521075e248b627.rlib" "/home/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libunwind-7c91ffdc8da860d3.rlib" "/home/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/librustc_demangle-0ad27b9879d551d3.rlib" "/home/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/liblibc-588f18eae3ea58be.rlib" "/home/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/liballoc-4ebf5caee903d98f.rlib" "/home/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/librustc_std_workspace_core-8895b32baedb08c6.rlib" "/home/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libcore-6a9d233d01acc350.rlib" "/home/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libcompiler_builtins-851bb3b5f6c4db49.rlib" "-Wl,-Bdynamic" "-lpq" "-static" "/home/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/crtn.o"
  = note: /usr/local/musl/lib/libpq.a(fe-connect.o): In function `connectFailureMessage':
          fe-connect.c:(.text+0xa12): undefined reference to `strlcpy'
...

Investigation suggests that the problem is that these libraries:

"/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/libopenssl-0de50740ee181784.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/libopenssl_sys-89c6410693b927d6.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/liblibc-6265db6daf6f0d94.rlib"

...appear before:

"/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/libpq_sys-dc85b10baeb00fc3.rlib"

Manually reordering these will cause the linker to succeed.

The underlying problem sees to be that a statically-linked libpq doesn't actually pull in a statically linked libopenssl or the relevant portions of musl-libc's libc.

The easiest fix seems to be including the following in main.rs:

// The order of these two matters!
//
// If you reverse them, the build will fail.
extern crate openssl;
#[macro_use]
extern crate diesel;

But this is fragile, and the exact details seem to change from one release of rustc to the next. You can find a working example here, and it should break if you reorder the extern crate declarations.

Probably the easiest fix (kludge?) for this would be to include extern crate openssl_sys; somewhere in pq-sys, at least when PQ_LIB_STATIC_X86_64_UNKNOWN_LINUX_MUSL=1 is set. That feels slightly hackish, but it would delegate responsibility for correctly finding and linking libopenssl to the openssl_sys crate, which has a fairly fancy cross-platform build system.

Once again, thank you so much for your crate, and for maintaining it over all these years! If there's anything I can do to help sort this out, please let me know! I'd be happy to prepare a PR for you to review, or whatever else is needed.

@sgrif
Copy link
Owner

sgrif commented Apr 24, 2019

PR sounds great

@golddranks
Copy link
Contributor

Hi, I'm the maintainer of https://gitlab.com/rust_musl_docker/image and I recently ran into this problem also. Is there any movement fixing this?

@golddranks
Copy link
Contributor

@emk Apparently there hasn't been a PR fixing this. Could you elaborate why the order of the linker flags matter? Is the problem that libpq pulls in a dependency to OpenSSL? ( https://github.com/postgres/postgres/blame/master/src/interfaces/libpq/fe-secure-openssl.c#L702-L715 )

@golddranks
Copy link
Contributor

From clux/muslrust#49 , I got the impression libpq can be built with support for SSL or not, and depending on that the linking might fail, if it doesn't find the object. I wonder what's the correct way for this crate to detect that – is there a way to detect that and insert a dependence, or should the build just be configurable via an env var (hoping that the users will find out about that and set it if they need to.)

@emk
Copy link
Author

emk commented Jan 27, 2020

I'm still getting reports of similar errors occasionally, and we have several long bug reports over at emk/rust-musl-builder#69 and emk/rust-musl-builder#64. Unfortunately, I've been busy lately, and I've forgotten the details. But if anybody wants to take a look, please feel welcome.

@emk emk closed this as completed Jan 27, 2020
Raniz85 pushed a commit to Raniz85/pq-sys that referenced this issue Apr 17, 2020
Added a new feature (openssl-static) that includes the openssl-sys crate
to fix static builds.

This fixes sgrif#25
Raniz85 pushed a commit to Raniz85/pq-sys that referenced this issue Apr 17, 2020
Added a new feature (openssl-static) that includes the openssl-sys crate
to fix static builds.

Renamed src/lib.rs to src/bindgen.rs and added a new lib.rs that
re-exports everything from bindgen.rs to prevent future bindgen runs
from reverting this change.

This fixes sgrif#25
@golddranks
Copy link
Contributor

I managed to consistently reproduce this. Here's a Dockerized repro: https://github.com/golddranks/pq_link_test

davepacheco pushed a commit to oxidecomputer/pq-sys that referenced this issue Sep 11, 2021
Added a new feature (openssl-static) that includes the openssl-sys crate
to fix static builds.

Renamed src/lib.rs to src/bindgen.rs and added a new lib.rs that
re-exports everything from bindgen.rs to prevent future bindgen runs
from reverting this change.

This fixes sgrif#25
ruir6o pushed a commit to ruir6o/pq-sys that referenced this issue Dec 14, 2022
Added a new feature (openssl-static) that includes the openssl-sys crate
to fix static builds.

Renamed src/lib.rs to src/bindgen.rs and added a new lib.rs that
re-exports everything from bindgen.rs to prevent future bindgen runs
from reverting this change.

This fixes sgrif#25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants