Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#[link(kind="raw-dylib")] #2627

Merged
merged 14 commits into from
Feb 24, 2019
106 changes: 106 additions & 0 deletions text/0000-dll-kind.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
- Feature Name: dll_kind
Centril marked this conversation as resolved.
Show resolved Hide resolved
- Start Date: 2019-01-22
- RFC PR: (leave this empty)
- Rust Issue: (leave this empty)

# Summary
[summary]: #summary

Extend the `#[link]` attribute by adding a new kind `kind="dll"` for use on Windows which emits idata sections for the items in the attached `extern` block, so they may be linked against without linking against an import library. Also add a `#[link_ordinal]` attribute for specifying symbols that are actually ordinals.

# Motivation
[motivation]: #motivation

[dll]: https://en.wikipedia.org/wiki/Dynamic-link_library

Traditionally, to link against a [dll], the program must actually link against an import library. For example to depend on some symbols from `kernel32.dll` the program links to `kernel32.lib`. However, this requires that the correct import libraries be available to link against, and for third party libraries that are only distributed as a dll creating an import library can be quite difficult, especially given that `lib.exe` is incapable of creating an import library that links to `stdcall` symbols.

A real advantage of this feature, however, is the fact that symbols will be *guaranteed* to come from the specified dll. Currently, linking is a very finnicky process where if multiple libraries provide the same symbol the linker will choose one of them to provide the symbol and the user has little control over it. With `kind="dll"` the user is ensured that the symbol will come from the specified dll.

Sometimes, a crate may know exactly which dll it wants to link against, but which import library it ends up linking against is unknown. In particular the `d3dcompiler.lib` provided by the Windows SDK can link to several different versions of the d3dcompiler dll depending on which version of the Windows SDK the user has installed. `kind="dll"` would allow `winapi` to link to a specific version of that dll and ensure the symbols are correct for that version.

This would also allow `winapi` to not have to bundle import libraries for the `pc-windows-gnu` targets, saving on bandwidth and disk space for users.

# Guide-level explanation
[guide-level-explanation]: #guide-level-explanation

When trying to link to a Windows dll, the `dylib` kind may sometimes be unsuitable, and `kind="dll"` can be used instead. A central requirement of `kind="dll"` is that the dll has a stable ABI. Here are some examples of valid reasons to use `kind="dll"`:

* You've had it up to here with trying to create an import library for a dll that has `stdcall` functions.
* You're in linking hell with multiple import libraries providing the same symbol but from different dlls.
* You know exactly which dll you need a symbol from, but you don't know which version of the dll the import library is going to give you.
* You maintain `winapi`.

Here is an example of usage:

```rust
#[cfg(windows)]
#[link(name = "kernel32.dll", kind = "dll")]
#[allow(non_snake_case)]
extern "system" {
fn GetStdHandle(nStdHandle: u32) -> *mut u8;
}
```

Some symbols are only exported by ordinal from the dll in which case `#[link_ordinal(..)]` may be used:

```rust
#[cfg(windows)]
#[link(name = "ws2_32.dll", kind = "dll")]
#[allow(non_snake_case)]
extern "system" {
#[link_ordinal(116)]
fn WSACleanup() -> i32;
}
```

# Reference-level explanation
[reference-level-explanation]: #reference-level-explanation

Add a new attribute `#[link_ordinal]` taking a single unsuffixed integer value, such as `#[link_ordinal(116)]`. It can only be specified on symbols in an extern block using `kind="dll"`.

Add a new value `dll` to the `kind` property of the `link` attribute. When this kind is specified, the `name` must explicitly include the extension. In addition, for all items in the associated extern block, Rust will *keep* the symbol mangled, instead of having an unmangled symbol. Rust will emit an idata section that maps from the *mangled* symbol to a symbol in the specified dll. The symbol in the dll that the idata section maps to depends on which attributes are specified on the item in question:

* If `#[link_ordinal]` is specified the idata section will map from the mangled symbol to the ordinal specified in the dll.
* If `#[link_name]` is specified the idata section will map from the mangled symbol to the name specified in the dll, without any calling convention decorations added. If calling convention decorations are desired they must be specified explicitly in the value of the `#[link_name]` attribute.
* If both `#[link_ordinal]` and `#[link_name]` are specified, an error will be emitted.
* If neither `#[link_ordinal]` nor `#[link_name]` are specified, the idata section will map from the mangled symbol to its unmangled equivalent in the dll. The unmangled symbol will *not* have calling convention decorations.
* If `#[no_mangle]` is specified an error will be emitted.

The idata section that is produced is equivalent to the idata sections found in import libraries, and should result in identical code generation by the linker.

# Drawbacks
[drawbacks]: #drawbacks

Additional complexity in the language through a new `kind` and a new attribute for specifying ordinals.

# Rationale and alternatives
[alternatives]: #alternatives

The RFC as proposed would allow for full control over linking to symbols from dlls with syntax as close as possible to existing extern blocks.

No alternatives are currently known other than the status quo.

# Prior art
[prior-art]: #prior-art

Many non-native languages have the ability to import symbols from dlls, but this uses runtime loading by the language runtime and is not the same as what is being proposed here.

Delphi is a native language that has the ability to import symbols from dlls without import libraries.

# Unresolved questions
[unresolved]: #unresolved-questions

* Bikeshedding on attribute names.
Centril marked this conversation as resolved.
Show resolved Hide resolved
* Should this feature be extended to other platforms?
Centril marked this conversation as resolved.
Show resolved Hide resolved

# Future possibilities
[future-possibilities]: #future-possibilities

With the features described in this RFC, we would be one step closer towards a fully standalone pure Rust target for Windows that does not rely on any external libraries (aside from the obvious and unavoidable runtime dependence on system libraries), allowing for easy installation and easy cross compilation.

If that were to happen, we'd no longer need to pretend the pc-windows-gnu toolchain is standalone, and we'd be able to stop bundling MinGW bits entirely in favor of the user's own MinGW installation, thereby resolving a bunch of issues such as [rust-lang/rust#53454](https://github.com/rust-lang/rust/issues/53454).

A future extension of this feature would be the ability to optionally lazily load such external functions, since Rust would naturally have all the information required to do so.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you were to "speculate", what might that look like?

Copy link

@zunzster zunzster Feb 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that might be a reference to my delayed=true optional extension discussed above e.g.

#[link(name = "kernel32.dll", kind = "dll", delayed = true)]

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd personally prefer the option where the caller of the function chooses whether to do a lazy loaded call of it, and not having to choose at declaration time whether it is lazy loaded. I don't know what the syntax would look like for that.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@retep998 Can you say a bit more about that? I'm curious as to why that's useful. Going that way would seem to preclude (or at least complicate) the compiler from being able to optimize all the generated thunks so they are not doing any repeated LoadLibrary/GetProcAddress calls. It's likely you have a use case in mind that I've not come across before which warrants giving up that benefit.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The compiler could still ensure there is only one LoadLibrary for a given dll and one GetProcAddress for each symbol. Allowing the caller to choose whether they want to lazy load it does not prevent the compiler from ensuring GetProcAddress is only called once. If no dylib crates are involved, the generation of GetProcAddress thunks can be lazily deferred all the way until binary creation time. If dylib crates are involved, then either the dylib that contains the crate with the declarations would have to generate all the thunks, or there could potentially be some duplication across dylibs. Since nobody uses dylib except rustc and people who accidentally used it when they meant to use cdylib, it doesn't seem like too big of an issue.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As for why, it depends on whether the caller is able to deal with the function not existing. If the caller has a fallback, then it can use the lazy loaded versions and fallback if it fails to load. If the caller doesn't have any fallback, then it can use the more efficient static version.

Copy link

@Arnavion Arnavion Feb 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zunzster The user can also write such thunks themselves. It would be preferable for the user to write them since they can choose how to handle the LL/GPA failure if the DLL / function doesn't exist (crash / no-op / return a custom Err()) rather than have the compiler enforce a specific choice.

Copy link

@zunzster zunzster Feb 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Arnavion Oh, sure. I know users can write the LL/GPA thunks themselves. I've done that many a time myself. It's just tedious and hence sometimes error prone since writing robust error handling around each API call is annoying. Hence, it's nice if the compiler provides a 'pit of success' offering convenient lazy loading with robust error handling as the default.

@retep998 If you're making the lazy loading transparent to the user, you can't play with the return values since they're already a concrete type defined by some random API. So, yes, having a standard 'missing_API_handler' hook which the user can potentially override is nice. You can make the hook a no-op or your own custom handler (similar to panic handlers I suppose) but the default one should probably panic with a descriptive error and back trace.

Actually, I can't see the no-op change being especially easy though with regard to specifying what the return value (if any) should be in case of a missing API.

Some Windows API return BOOL (typedefed Integers) with 0 meaning failure.
Some APIs return HANDLE with -1 (aka INVALID_HANDLE_VALUE) for failure.

So, offering the no-op case would seem to require inelegant/complicated declarative support. I think if users want that kind of no-op behavior, they probably should have to do their own LL/GPA handling since the alternative isn't well-specified enough to be safe.

Of course, this is all just my opinion. Maybe there is a better solution I'm just not seeing since I'm used to the Delphi approach. When all you have is a hammer, every problem can start to look like a nail. :-)


Also users would stop complaining about having to install several gigabytes of VC++ just to link their Rust binaries.
retep998 marked this conversation as resolved.
Show resolved Hide resolved