-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replace libosxunwind wth LLVM libunwind #39127
Conversation
@staticfloat Any idea why CI isn't starting? |
I need to enable the LLVMLibUnwind stub only on macOS for now. The macOS failure is strange as it appears it's looking for |
Dropping the version number from |
I tried this locally on MacOS and got profiling working!
however, when using in conjunction with
Unfortunately when ProfileView does work, it doesn't immediately solve the dropped samples issue #38350 i.e. the profile view is still 80% blank Thanks! |
I'm fixing some issues with building Julia dependencies from source which should allow me to iterate faster on this PR. |
9704c9e
to
770b169
Compare
All tests are passing but the "Profile" tests have the following warnings when using the LLVM libunwind dependency built from source and using
The output above was generated with a patched version of LLVM libunwind which lets Julia force the use of DWARF during unwinding (adapted from JuliaLang/libosxunwind@bb95a5c). The BB version doesn't have this patch at the moment which why the I am uncertain what is required to change to address these warnings. Someone more familiar with libunwind should probably continue this work at this point. |
Alright, to summarize the situation here:
In an ideal world, compact unwind info would be async safe, so we could stop building both kinds of unwind info. There was a discussion of that at https://lists.llvm.org/pipermail/llvm-dev/2018-January/120741.html, but I don't believe that ever went anywhere. In the absence of that, and more achievable, we should be able to use DWARF unwind info everywhere. This is already true for code we built, but of course sometimes Apple-provided code is on the stack, which is generally not async-unwind safe. I'll follow up with Apple to see if something can be done here. I understand @IanButterworth filed FB8974841 to request this also. I've looked into the warnings posted by @omus above and they are generally just caused by these issues. However, they are part of a debug print and should be disabled. There is probably some release flag missing somewhere to turn these off. I think we can proceed with this PR as is and just east the unwind imprecision on aarch64. If it's easy to do, we can consider carrying JuliaLang/libosxunwind@ca57a5b in addition to the patch already in this PR, but I'd be ok with dropping it if we can work with Apple on a more longterm solution. |
You are correct that these are debug messages and they were disabled in libosxunwind. I discovered that using
I've generated a patch that works with LLVM libunwind and will post it soon. |
770b169
to
290bdb9
Compare
I've rebased these changes and have included the patch @Keno asked for. As I already need to update the BB build I'll update to use LLVM libunwind 11.0.1: JuliaPackaging/Yggdrasil#2557 |
829c5b5
to
8c1433b
Compare
That does sound like a bug. |
I found one more thing to take care of. After doing a
This is due to to not copying the the modified header files which include the |
I just need to update the |
name = "LibOSXUnwind_jll" | ||
uuid = "a83860b7-747b-57cf-bf1f-3e79990d037f" | ||
version = "0.0.6+1" | ||
name = "LLVMLibUnwind_jll" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have platform dependent stdlibs now? Or is this installed on all platforms?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just like LibOSXUnwind_jll
this is installed on all platforms but only used on macOS. I will mention that we should be able to use LLVMLibUnwind_jll
on all platforms in the future but that's beyond the scope of this PR
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right it's nothing that is changed in this PR. Who even uses/depends on this stdlib?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe only the Julia internals depend libunwind. I'm not sure but I suspect the BB infrastructure needs a stdlib in place for the build to work.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, seems bad to expose things as official stdlib packages just because BB needs it...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@staticfloat would know
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All binary dependencies must be represented by JLLs for a few reasons:
-
If client code wants to load the library, they should do so by expressing a dependency on a JLL. This is better than them blindly running
dlopen(libname)
as it won't get confused byLD_LIBRARY_PATH
, etc.... -
If a user package requires a particular version of a binary dependency, then this will prevent that package from loading on an older version of Julia if that binary dependency is locked to an incompatible versions by Julia itself. By having explicit JLLs that are stdlibs, we are able to inform the Pkg resolver about the versions of binaries that ship with Julia.
-
For executables that ship with Julia (like
7z
) we can provide the standard tools that users are being trained to use to avoid issues around environment variables, paths, etc...
I'm validating that these changes are working correctly on Apple Silicon which was the main motivation for me to get this switch done. Currently working through this issue when building LLVM libunwind from source on
Update: I sorted this out. It was just an issue on my local system where the x86 brew's |
Stacktraces for Apple Silicon are kind of working:
|
9c514d7
to
5f51d69
Compare
Thinking about it more the original motivation for replacing libosxunwind was to get off a 7 year old version of libunwind. LLVM libunwind does support Apple silicon but I think adding that support into Julia is beyond the scope of this PR. The PR is ready to merge from my end. I'll let someone else pull the trigger :) |
5f51d69
to
47bc0e8
Compare
Encountering a new failure after doing a rebase. I thought removing the |
I'm on board with merging this and I can look info the aarch64 unwind issue, but something broke here and this is now broken on mac CI. |
Thanks @Keno. I'll be looking into the CI failure |
a4549e1
to
5da3e7e
Compare
I believe the CI issue had to do with some bad caching resulting from accidentally committing the StdlibArtifacts.toml. I made some superficial corrections as well and did a final rebase. Should see a green CI 🤞 |
RTM |
Replaces libosxunwind with LLVM's libunwind (#30154). It appears that we can also replace the nongnu libunwind as well with this library in another PR.
make -C deps distclean-llvmunwind
when building from sourcelibunwind.h
which includesunw_init_local_dwarf