-
Notifications
You must be signed in to change notification settings - Fork 12.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Resolve DLL imports at CRT startup, not on demand #81478
Conversation
Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @joshtriplett (or someone else) soon. If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes. Please see the contribution instructions for more information. |
Suggest @m-ou-se review this, since they've made changes in this area before. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like a great improvement!
Will this .CRT$XCU
section work also work when it is part of a Rust cdylib
/dll
that'll be loaded into another program, possibly written in another language?
18a0325
to
c799a97
Compare
My understanding was that we're removing the dynamic detection of these features in the course of removing Windows XP support. |
That already happened. The few APIs left that are loaded dynamically now are also not available on Windows 7. E.g. |
@sivadeilra This PR now adds a few files called |
Eeek! No, that's an accidental inclusion. I'll remove it. |
On Windows, libstd uses GetProcAddress to locate some DLL imports, so that libstd can run on older versions of Windows. If a given DLL import is not present, then libstd uses other behavior (such as fallback implementations). This commit uses a feature of the Windows CRT to do these DLL imports during module initialization, before main() (or DllMain()) is called. This is the ideal time to resolve imports, because the module is effectively single-threaded at that point; no other threads can touch the data or code of the module that is being initialized. This avoids several problems. First, it makes the cost of performing the DLL import lookups deterministic. Right now, the DLL imports are done on demand, which means that application threads _might_ have to do the DLL import during some time-sensitive operation. This is a small source of unpredictability. Since threads can race, it's even possible to have more than one thread running the same redundant DLL lookup. This commit also removes using the heap to allocate strings, during the DLL lookups.
c799a97
to
f4debc8
Compare
Not directly feedback on this PR, but just thinking a bit out loud about (the future of) these dynamic imports on Windows: Since support for XP/Vista was dropped recently, the only dynamic imports left are:
Each of these is only used in one place. The first one is used every time a thread is created. The fallback is a function that does nothing. That code calls this function unconditionally, but it could've avoided allocating the utf16 version of the name if it checked if this function was available first. (In that case, no fallback implementation is needed.) The second one has the exact same signature as the less precise The third and forth sets are alternatives for eachother: The third set (the 'futex' calls) is used if available, and otherwise the fourth set (keyed events) is used instead. (For the thread parker implementation.) Lookups for the fourth set can be skipped entirely if the third set is available. Also, the fallback implementations for the third set are never called. And for both sets, either all functions of none of the functions will be available. When This all makes me wonder what the best outcome could be, if we start making use of this It also makes me wonder if this system of Maybe a hand-written I'm not saying this is all something we should do right now, but maybe this PR could be a step towards something like that. (And it'd be good to get some experience in Rust with Would love to hear your thoughts about this, as you know more about this than I do. (And I suppose you probably already thought about next steps here as well.) |
These are great thoughts! There are several aspects of this: First, @rylev and I have been working on supporting different Windows API versions as different targets for Rust. The goal is that you would be able to copile Rust apps (and libstd) for Windows 7, Windows 10, Xbox, etc., and we can optimize the behavior of libstd for each of these targets. So for Windows 10, all of these imports would be normal DLL imports, with no delayed binding. (And they should also use API sets, which is a relatively recent concept added to Windows DLL loading, and has some advantages.) @rylev's pre-RFC for this is here: https://internals.rust-lang.org/t/pre-rfc-min-target-api-version/13339 Second, Windows has built-in support for handling delayed DLL imports. So if you set things up correctly, the import tables say "I always need X, Y, and Z, but I also want W. If you can find W, then use it, otherwise use this fallback function." It's basically exactly what Right now, when So ideally, if you compile for Win10, you get all normal imports, with no delayed binding. And once we land the Then, if you compile for a specific downlevel version, like Windows 7, then we can set up the import tables so that Windows handles all of the delayed binding for you. The binaries would contain both the "do the fancy new thing" code as well as the fallback code, but libstd won't have to do its own lookups to find the DLL imports. I wanted to submit this PR now, just to slightly improve things, while we're working on the better long-term solution. Also, a member of my team is currently working on extending |
Sounds good! Let's get started with this PR then. :) @bors r+ |
📌 Commit f4debc8 has been approved by |
@bors rollup=never Let's not roll this one up, to make sure it's easy to find if it causes any regressions. |
By the way, feel free to use the |
FWIW, Miri does not implement these magic init link sections, so the new init code will just not be called when running on Miri. From what I understood, this means the |
☀️ Test successful - checks-actions |
rustup; remove some no-longer-needed Windows shims libstd now calls these lock functions directly, and `GetModuleHandleW` isn't use either any more since rust-lang/rust#81478.
My PR rust-lang#81478 used the wrong calling convention for a set of functions that are called by the CRT. These functions need to use `extern "C"`. This would only affect x86, which is the only target (that I know of) that has multiple calling conventions.
…x86, r=m-ou-se Fix calling convention for CRT startup My PR rust-lang#81478 used the wrong calling convention for a set of functions that are called by the CRT. These functions need to use `extern "C"`. This would only affect x86, which is the only target (that I know of) that has multiple calling conventions. `@bors` r? `@m-ou-se`
Can I ask why the C runtime's init routine is used instead of Rust's native Also, while I'm here, a quick question about |
To be honest, I was simply unaware that the Rust About This may seem like a distinction without a difference, but there is a difference. The difference is that the Windows allocator state is separated from the Rust allocator state. It is safe to use |
Thanks, for the response!
That's fair enough. There have been others who want to be able to use "pure Rust" without the C runtime. Which admittedly isn't currently possible (well it's technically possible but not easy).
Ah, I understand. My other concern with using |
I don't see much value in "pure Rust", for it's own sake. I do see huge value in integrating Rust with the enormous existing software ecosystems, and at least today that means C/C++. If there is a reason to go with "pure Rust" in some situation, then I'd certainly be willing to understand that better, and possibly to support it. But purity for its own sake seems more like an aesthetic goal than one that has concrete benefits. |
On Windows, libstd uses GetProcAddress to locate some DLL imports, so
that libstd can run on older versions of Windows. If a given DLL import
is not present, then libstd uses other behavior (such as fallback
implementations).
This commit uses a feature of the Windows CRT to do these DLL imports
during module initialization, before main() (or DllMain()) is called.
This is the ideal time to resolve imports, because the module is
effectively single-threaded at that point; no other threads can
touch the data or code of the module that is being initialized.
This avoids several problems. First, it makes the cost of performing
the DLL import lookups deterministic. Right now, the DLL imports are
done on demand, which means that application threads might have to
do the DLL import during some time-sensitive operation. This is a
small source of unpredictability. Since threads can race, it's even
possible to have more than one thread running the same redundant
DLL lookup.
This commit also removes using the heap to allocate strings, during
the DLL lookups.