Add support for setting the soname of generated dynamic libraries

[ghost]

I know that Rust has no ABI compatibility, but some distributions, like Fedora, require each dynamic library to have an soname.

This patch adds the soname codegen flag to rustc and the --enable-soname option to the configure script.

Since the stage0 compiler does not support the soname flag yet, my modified target.mk file never sets the soname for libraries build in stage0 (which are not installed on the system anyways). Maybe should we update target.mk when a new bootstrap compiler will be build.

[rust-highfive]

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @alexcrichton (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

[alexcrichton]

Adding a flag to the compiler currently is something that requires an RFC because it'll be instantly stable and we don't have an unstable period to flesh it out. Would it be possible to perhaps do some sort of post-processing as part of the build system instead of affecting how the linker is called? That would certainly be much easier to land in the meantime at least.

[huonw]

There is -Z unstable-options, although I don't think this works with subflags of -C at the moment, so this would have to be added as a top-level argument to be (that sort of) unstable.

[ghost]

It would maybe be possible to use PatchELF for post-processing, since this program now supports --set-soname. However, this seems to be a recent feature that is not yet available in stable releases. Also, PatchELF is not installed by default in most distributions, so it would be an additional build dependency.

If the soname is added through post-processing, it would be possible do not introduce any change at all in the build system and to let the packager handle this (an soname is only required for libraries installed by the system's package manager, not for user-installed ones).

In the long term, I still think we should support soname in rustc itself, because other projects written in Rust will need to be packaged too. Should I open an RFC ?

Maybe rustc should add an soname by default when linking dynamic libraries (but I think this would also need an RFC, and we should probably add a flag to cancel or customize this behaviour).

[alexcrichton]

Perhaps for now this can be passed as -C link-args through the build system rather than through a first-class option?

[ghost]

Thank you, this seems to work.

At least two things should be taken into account before merging this pull request:

• I think that ./configure --enable-soname; make will fail on Windows when building with MSVC, since MSCV's linker will not recognize the -Wl,-soname option. On the other hand, using the --enable-soname option doesn't make sense on Windows.
• I absolutely don't know what will happen if, for some reason, the -C link-args flag is passed multiple times to rustc by the build system.

[brson]

@gmbonnet This patch looks like it makes the soname the same as the library name, but when I see sonames in the wild they are always like "library.so.1" with a version at the end. What does it mean to not include the version in the soname?

Since Rust provides no binary compatibility at all and the rust compiler uses its own versioning scheme that doesn't care about sonames, I guess it doesn't matter at all what the soname is - subsequent releases of std are going to have entirely different file names.

Is the purpose of this patch just to obey the letter of the Fedora policy? Is there any practical impact? Do you have links to details about Fedora's policy on sonames (cursory Googling failed)?

[bors]

☔ The latest upstream changes (presumably #31120) made this pull request unmergeable. Please resolve the merge conflicts.

[ghost]

@brson It is true that soname is related to library versioning. Fedora packaging guidelines for shared libraries seem to be assume that packaged libraries have binary compatibility, so the package for Rust would perhaps need an exception to these guidelines.

For a standard, binary-compatible library, at least two identical versions of the library are installed to /usr/lib, respectively named library.so and library.so.0, where 0 is the major version of the library. library.so is often a symlink to library.so.0. Developers link against library.so, but because library.so has the soname library.so.0, the resulting binary is effectively linked against library.so.0, not library.so (without suffix). So, if the library gets upgraded, library.so.0 is replaced by library.so.1 and programs that were linked against library.so.0 fail to run, instead of producing obscure errors due to ABI incompatibility.

There is an alternative library naming scheme where the version suffix is before the .so suffix, like in library-0.so. This is why I was thinking that CFG_FILENAME_EXTRA could be seen as some sort of hexadecimal version number (since library version numbers do not need to be ordered). However, technically, it is true that the soname is not really needed if developers do not link directly against Rust libraries, so we should perhaps better request an exception in Fedora guidelines.

Since my package is mostly an experiment, I'll try to get a source package that barely works, without support for soname yet, and then publish it and request feedback in the ticket on Red Hat Bugzilla about packaging Rust, where there are more experienced packagers. I'll add a comment to this pull request when there will be an official point of view about sonames for Rust libraries.

Some informations about libraries installed by Rust would be helpful:

• When are /usr/lib/lib<name>-<filename_extra>.so libraries needed ? If I understand well, rustc and rustdoc link against them. Are they also needed by compiler plugins ?
• When are /usr/lib/rustlib/<target>/lib/lib<name>-<filename_extra>.so libraries needed ? Why are not *.rlib libraries sufficient ?
• Assuming that Rust has been configured with identical host and target triples, would it work to replace /usr/lib/lib<name>-<filename_extra>.so libraries by symlinks to rustlib/<target>/lib/lib<name>-<filename_extra>.so (are they interchangeable) ?

[alexcrichton]

Thanks for the info @gmbonnet! Sounds like we may want to hold off on this temporarily while you hear back about packaging? In the meantime, though, I believe I can answer your questions:

When are /usr/lib/lib<name>-<filename_extra>.so libraries needed ? If I understand well, rustc and rustdoc link against them. Are they also needed by compiler plugins ?

Pedantically speaking, the rustc and rustdoc executables are linked against these and therefore need them at runtime. Compiler plugins are actually literally linked against the dynamic libraries in rustlib, but they then actually use the /usr/lib/libfoo.so copies at runtime. Which is to say that we link compiler plugins against one copy, and then run them with another.

The two copies of the *.so libraries are in theory byte-for-byte the same between /usr/lib and /usr/lib/rustlib.

We have an issue or two open on resolving this discrepancy I believe.

When are /usr/lib/rustlib/<target>/lib/lib<name>-<filename_extra>.so libraries needed ? Why are not *.rlib libraries sufficient ?

Ah yeah, as mentioned above we link against these libraries. These are in theory what should be loaded at runtime, but currently aren't.

Assuming that Rust has been configured with identical host and target triples, would it work to replace /usr/lib/lib<name>-<filename_extra>.so libraries by symlinks to rustlib/<target>/lib/lib<name>-<filename_extra>.so (are they interchangeable) ?

I believe so, yes. And I also believe we have to always maintain this invariant while we have compiler plugins.

[ghost]

Thanks. I'm currently verifying that my package compiles, then I'll contact Fedora developers on the official ticket.

I also needed to make a new pull request (#31388). I was just wondering if I should set a particular assignee in such cases.

[ghost]

Here is the link to my comment on Bugzilla:
https://bugzilla.redhat.com/show_bug.cgi?id=915043#c41

[alexcrichton]

Closing due to inactivity for now, but feel free to resubmit if something changes!