-
Notifications
You must be signed in to change notification settings - Fork 733
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SYCL][UR] Use Windows proxy loader for UR #15262
[SYCL][UR] Use Windows proxy loader for UR #15262
Conversation
7287d30
to
e04fd16
Compare
e04fd16
to
ae77ba8
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM - detail/bindless_images.cpp
On further investigation it seems like this was a change we made to |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm, just one nit.
@intel/llvm-gatekeepers Please merge when possible |
@callumfare I think this PR also caused this error too:
That's from my PR here and we also see it in Mac postcommit here. Any ideas? Should we revert this? Build is broken for everyone using HEAD. |
@sarnex That's unrelated to this change. It appears to be caused by this change to OpenCL-Headers - KhronosGroup/OpenCL-Headers@7258b9e - which just merged. We must be pulling the tip of the repo rather than a specific commit |
@callumfare Nice find, I didn't understand how this PR could cause the problem either. I'll try to fix it. FYI @EwanC |
This PR should fix it but then I have to bump the version here. |
These e2e tests started failing after PI removal and replacing it with UR ([PR](#14145)) However, they were not related to it. On Windows, dlls unloading is inconsistent and if we try to release these UR objects manually, inconsistent hangs happen due to a race between unloading the UR adapters dlls (in addition to their dependency dlls) and the releasing of these UR objects (The proxy loader have solved this problem partially [here](#15262)). So, we currently shutdown without releasing them and windows should handle the memory cleanup. This behaviour is the same old behaviour as before removing PI as on Investigations on this. This was only hidden before PI removal as it was calling the PI entry-point but doesn't make it to UR entry-point and Filecheck logs check for objects release would pass as it only check for the call to the PI entry-point without check that the call was a successful call. That was the PI call for `piContextRelease` before PI removal: ``` ---> piContextRelease( <unknown> : 0000023CC0EBA6C0 ) ---> API Called After Plugin Teardown, Functon Call ignored. ``` Fixes #14768 Fixes #14950 Fixes #14968
The issues with DLLs and teardown of global objects on Windows is well documented, and was the reason for the use of the
pi_win_proxy_loader
library in SYCL-RT. When we ported from PI to UR, we ported this library (it's now calledur_win_proxy_loader
), but it was not actually fully utilized. SYCL-RT still linked withur_loader.dll
and still experienced issues with race conditions in the teardown of SYCL-RT and Unified Runtime. See #14768.This PR reintroduces the proxy loader as it was previously used with PI. The UR loader (
ur_loader.dll
) is loaded viaLoadLibraryEx
at initialization, and is therefore not cleaned up too early for normal teardown to occur.This necessitates changing the signature of
Plugin->call
to look like it did with PI, taking an enum template argument to specify which UR entry point to call.On Windows, when each plugin (which is a wrapper over a UR adapter) is loaded, it populates a table of function pointers to each API entry point in the UR loader. When UR entry points are called, the function pointer is retrieved from the table. This is more or less equivalent to the previous PI implementation.
On Linux, the UR loader is dynamically linked as before. The
Plugin->call
methods just use the regular UR functions rather than programmatically looking up the symbols.For the unittest executables, the UR loader is still dynamically linked as before to avoid having to introduce noisy changes to the tests, and since we aren't concerned about teardown issues there.
The implementation of these changes in the runtime should avoid as much overhead as possible (and be no worse than PI), but suggestions on how to improve and tidy things are more than welcome.
Associated UR change: oneapi-src/unified-runtime#2045