Implement dynamic linking feature

[zyansheep]

The Bevy game engine has a nice dynamic linking features which, coupled with the lld linker, significantly improves compile times.
I have added this to iced for faster compiling iterations (down to about 2.3 from 3.8 for a personal project) . What do you think?
Relevant issue: #849

[13r0ck]

Awesome!
Just out of curiosity, does this have any runtime performance degradation?
This just curious as how to best use this, should this feature only be enabled in the project's Cargo.toml for development compile to speed up dev speed, and then static compile for prod?

[zyansheep]

Just out of curiosity, does this have any runtime performance degradation?
This just curious as how to best use this, should this feature only be enabled in the project's Cargo.toml for development compile to speed up dev speed, and then static compile for prod?

Probably not? Bevy says it's better to be used for development, but I don't think there should be much runtime overhead. The only thing it might effect is the program's start-up time while it tries to search for one extra shared library (on top of all the other shared libraries it already has to find). Either way, if the dynamic feature is used for release builds, the shared binary would have to be packaged with the executable so it's probably not desirable.
I usually just use it in the cli as cargo run --features iced/dynamic

[zyansheep]

I copied the tour of iced example to a separate folder and pointed the iced dependency to my fork to do some testing.
I am getting consistently about 1-2 second higher for static linking.

// Static Compilation:			3.02s, 3.48s, 2.69s, 2.33s, 2.74, 2.80s, 2.17s, 2.15s, 2.17s, 2.68s, 2.81s, 3.09s
// Dynamic Compilation (no lld):	1.50s, 1.47s, 2.06s, 1.38s, 1.97s, 1.40s, 1.45, 1.95s, 1.99s, 1.36s, 1.51s, 1.99s
// Dynamic Compilation (with lld):	1.47s, 1.40s, 1.48s, 1.39s, 1.42s, 1.42s, 1.43s, 1.98s, 1.41s, 2.02s, 1.42s, 1.39s

I kinda expected lld to improve times a bit more, but there could be other reasons that the times are similar.

[13r0ck]

How are you timing the compile times, (and what OS)?
I am interested in testing the compile times with my project that is much larger than the tour example.

[zyansheep]

How are you timing the compile times, (and what OS)?
I am interested in testing the compile times with my project that is much larger than the tour example.

I'm using NixOS (with this flake.nix). To run each test, I modify the main.rs file a bit to trigger an iterative recompile and then note down the output time of cargo run. I'm not sure if there is a built-in way to do this though...

[13r0ck]

Ok cool!
I'll give it a try late today

[13r0ck]

Oh wow. The effects of dynamic linking seems to have a greater effect on bigger projects.
I have a ~9k line project that I tested your PR on

Static Linking	9.25s
Dynamic Linking	5.32s

👍 👍 Great job!

[hecrj]

Interesting!

Could we leave the iced crate as-is and build iced_dynamic on top? I think the dynamic linking could be setup by users directly, instead of offering a feature out of the box which forces us to further nest the dependencies.

If I understand correctly, a user could simply introduce iced_dynamic as an optional dependency to their Cargo.toml and then add to their main.rs:

#[cfg(feature = "iced_dynamic")]
use iced_dynamic;

Then, they should be able to dynamically link with cargo by using the flag --features iced_dynamic. Is that correct?

[zyansheep]

To do that wouldn't you need to duplicate use statements for each file that uses iced? Also, iced would still need a folder with a Cargo.toml enabling dylib for the user to include (which would itself include iced as a dependency).

Also, having people configure the dynamic linking themselves might put off people who are new to rust. The first time I used iced, I was annoyed by how long it took to recompile small changes. (>10 seconds sometimes!). If you are worried about dependency messiness, iced could do something like what Bevy does and move all the modules into a crates folder which would shorten the workspace list:

[workspace]
members = ["crates/*", "examples/*"]

[13r0ck]

I agree with zyan, if possible manually configuring dynamic linking should be avoided, for both newbies and lazy people 😀 👈

[hecrj]

Why can we keep iced as-is and build everything else on top? In other words, why is renaming iced to iced_internal necessary?

If a user wants dynamic linking, they could use a different entrypoint instead of iced at the Cargo.toml level.

[13r0ck]

I'm not sure if that question was directed at me. Either way, I don't understand Rust nor Iced's under belly enough to be helpful to this conversation.
Whatever the solution is I appreciate both of you 😃

[zyansheep]

If a user wants dynamic linking, they could use a different entrypoint instead of iced at the Cargo.toml level.

That might work, but it will be annoying to the user if they have to constantly change their dependency from iced to iced_dynamic every time they commit new changes (assuming they want static compilation by default). Simply adding a --features iced/dynamic when you want faster compilation is a lot easier.

Why [can't] we keep iced as-is and build everything else on top? In other words, why is renaming iced to iced_internal necessary?

It's necessary because the iced crate needs to be able to easily switch between the static version of a Cargo.toml and a dynamic version of a Cargo.toml (with crate-type = ["dylib"]) features don't allow for this kind of thing AFAIK so the only way to do it is to have an internal crate and a dynamic crate that aliases the internal crate but with dylib compilation.

bevy does the same thing with the main Cargo.toml (and src/lib.rs) simply exporting internal crates (including bevy_internal and bevy_dynamic) based on feature flags.

[kaimast]

It's necessary because the iced crate needs to be able to easily switch between the static version of a Cargo.toml and a dynamic version of a Cargo.toml (with crate-type = ["dylib"]) features don't allow for this kind of thing AFAIK so the only way to do it is to have an internal crate and a dynamic crate that aliases the internal crate but with dylib compilation.

bevy does the same thing with the main Cargo.toml (and src/lib.rs) simply exporting internal crates (including bevy_internal and bevy_dynamic) based on feature flags.

Is there a rust issue for this? This seems like something Cargo should support.

[kirawi]

When I investigated how Bevy's dynamic linking worked, it seems like it was a neat trick based on how Cargo handles dependency linkage.

Rust Reference

3. If an executable is being produced and the -C prefer-dynamic flag is not specified, then dependencies are first attempted to be found in the rlib format. If some dependencies are not available in an rlib format, then dynamic linking is attempted (see below).

4. If a dynamic library or an executable that is being dynamically linked is being produced, then the compiler will attempt to reconcile the available dependencies in either the rlib or dylib format to create a final product.

A major goal of the compiler is to ensure that a library never appears more than once in any artifact. For example, if dynamic libraries B and C were each statically linked to library A, then a crate could not link to B and C together because there would be two copies of A. The compiler allows mixing the rlib and dylib formats, but this restriction must be satisfied.

The compiler currently implements no method of hinting what format a library should be linked with. When dynamically linking, the compiler will attempt to maximize dynamic dependencies while still allowing some dependencies to be linked in via an rlib.

For most situations, having all libraries available as a dylib is recommended if dynamically linking. For other situations, the compiler will emit a warning if it is unable to determine which formats to link each library with.

By these rules, there needs to be wrapper over an internal crate to force Iced to compile as a dylib when the feature is toggled.

[zyansheep]

Dynamic linking will probably be a lot easier once the Rust ABI gets stabilized, but who knows when that will happen...

[hecrj]

I am getting a lot of linking errors when trying to compile a project of mine using this new feature:

[...]

 "core::fmt::ArgumentV1::new::h99acd15b0d810580", referenced from:
                hyper::proto::h2::client::handshake::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h66050cc4f757c305 in libreqwest-0cd9e7c87c592796.rlib(reqwest-0cd9e7c87c592796.reqwest.239vdzex-cgu.0.rcgu.o)
                _$LT$hyper..proto..h2..client..ClientTask$LT$B$GT$$u20$as$u20$core..future..future..Future$GT$::poll::hd336b96c90d51f4d in libreqwest-0cd9e7c87c592796.rlib(reqwest-0cd9e7c87c592796.reqwest.239vdzex-cgu.0.rcgu.o)
                _$LT$hyper..proto..h2..client..ClientTask$LT$B$GT$$u20$as$u20$core..future..future..Future$GT$::poll::_$u7b$$u7b$closure$u7d$$u7d$::h1a51a433330ee9cc in libreqwest-0cd9e7c87c592796.rlib(reqwest-0cd9e7c87c592796.reqwest.239vdzex-cgu.0.rcgu.o)
            "_$LT$hyper..proto..h1..dispatch..Client$LT$B$GT$$u20$as$u20$hyper..proto..h1..dispatch..Dispatch$GT$::poll_msg::CALLSITE::hc47fe63017ec44b1", referenced from:
                _$LT$hyper..proto..h1..dispatch..Client$LT$B$GT$$u20$as$u20$hyper..proto..h1..dispatch..Dispatch$GT$::poll_msg::h317a453411800ae0 in libreqwest-0cd9e7c87c592796.rlib(reqwest-0cd9e7c87c592796.reqwest.239vdzex-cgu.6.rcgu.o)

ld: symbol(s) not found for architecture x86_64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

[hecrj]

Also, I think I've been able to implement the dynamic linking feature completely on top of the current iced crate, as I explained in #867 (comment). Check out the feature/dynamic-linking branch. You can see the changes here: master...feature/dynamic-linking

Unfortunately, it produces the same linking errors I mentioned in #867 (comment).

[yusdacra]

I am getting a lot of linking errors when trying to compile a project of mine using this new feature:

[...]

 "core::fmt::ArgumentV1::new::h99acd15b0d810580", referenced from:
                hyper::proto::h2::client::handshake::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h66050cc4f757c305 in libreqwest-0cd9e7c87c592796.rlib(reqwest-0cd9e7c87c592796.reqwest.239vdzex-cgu.0.rcgu.o)
                _$LT$hyper..proto..h2..client..ClientTask$LT$B$GT$$u20$as$u20$core..future..future..Future$GT$::poll::hd336b96c90d51f4d in libreqwest-0cd9e7c87c592796.rlib(reqwest-0cd9e7c87c592796.reqwest.239vdzex-cgu.0.rcgu.o)
                _$LT$hyper..proto..h2..client..ClientTask$LT$B$GT$$u20$as$u20$core..future..future..Future$GT$::poll::_$u7b$$u7b$closure$u7d$$u7d$::h1a51a433330ee9cc in libreqwest-0cd9e7c87c592796.rlib(reqwest-0cd9e7c87c592796.reqwest.239vdzex-cgu.0.rcgu.o)
            "_$LT$hyper..proto..h1..dispatch..Client$LT$B$GT$$u20$as$u20$hyper..proto..h1..dispatch..Dispatch$GT$::poll_msg::CALLSITE::hc47fe63017ec44b1", referenced from:
                _$LT$hyper..proto..h1..dispatch..Client$LT$B$GT$$u20$as$u20$hyper..proto..h1..dispatch..Dispatch$GT$::poll_msg::h317a453411800ae0 in libreqwest-0cd9e7c87c592796.rlib(reqwest-0cd9e7c87c592796.reqwest.239vdzex-cgu.6.rcgu.o)

ld: symbol(s) not found for architecture x86_64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

I am also getting these linker errors... I tried using Rust's linker, gold, lld and all of them give linker errors.

[zyansheep]

Also, I think I've been able to implement the dynamic linking feature completely on top of the current iced crate, as I explained in #867 (comment). Check out the feature/dynamic-linking branch. You can see the changes here: master...feature/dynamic-linking

I got your pull request working with one of the examples by changing

iced = { path = "../.." features = ["canvas", "tokio", "debug"] }

to

iced = { path = "../../dynamic", package="iced_dynamic", features = ["canvas", "tokio", "debug"] }

which seems to be a fine way to do it (you could put one in dependencies and one in dev-dependencies and it should work fine)

Unfortunately, it produces the same linking errors I mentioned in #867 (comment).

I'm using rust nightly on NixOS with a custom wrapped lld linker so maybe my linker is configured differently? What OS/setup do you have?
my flake.nix file: https://github.com/zyansheep/Grasslandia/blob/main/flake.nix
Edit: It seems like you use the ld linker, maybe it only works with the lld linker?

[zyansheep]

I am getting a lot of linking errors when trying to compile a project of mine using this new feature:
do you think you could try to compile bevy with the --features dynamic flag to see if you get the same error?

[hecrj]

@zyansheep I tried compiling Bevy with the dynamic feature and it compiled just fine.

I believe the issue here may be related to using async. CI is also failing with weird errors...

I'm closing this for now, but feel free to reopen it if you make any progress towards figuring out the cause of these issues.