-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Vowpalwabbit python wrappers do not build or crashes on OSX with Anaconda #2100
Comments
For users of Homebrew having trouble installing VW from pip:
|
Having same issue with pip on OSX. When is the fixed version going to be available on pip?
|
Also if it will be more than a few days, are there instructions for installing package from source please? I tried to clone the repo and then ran the instructions from the issue with the fix but I still get following error:
|
Ah managed to get it to work by just running |
However, now both examples cause segementation fault:
|
@scgalois I was unable to reproduce your segfault on my setup. Can I get some information from you?
Thanks |
@peterychang thanks for following up. Here are the details: OSX Mojave 10.14.6 with no anaconda installation (when I type conda in command line says command not found)
|
@scgalois From your logs, it doesn't look like you're using brew's python. Since your version of python differs from whats on homebrew, its possible the boost python library from homebrew is not binary compatible with the version of python you're running. If this is the cause, then there are a couple of possible solutions. First, you can switch to python3 from homebrew. |
@peterychang Thanks for the feedback. assuming I clone and compile boost myself, how do I make sure the vw package will link my new version rather than the boost version? Are there any detailed instructions anywhere or do I need to dig into the config scripts to figure it out? |
@peterychang Actually I ended up just changing my version of python and it is working fine now. Thanks for your help. But just for future reference, if there are instructions on building and using older version of boost against pyvw would appreciate it. |
@scgalois If you have a build of boost you want to link against, the following command should force VW to link against it. In this case, you'll want to pay attention to the build logs since I've noticed that the BOOST_ROOT variable is ignored in some cases that I'm still investigating.
|
Any progress @peterychang ?
|
@elviraux Unfortunately this is not an issue we can resolve from our end. Conda needs to rebuild python on a newer version of OSX, linking against libc++ (instead of libstdc++) The only workaround at the moment is to avoid using anaconda on MacOS 10.14+ |
One planned task may actually resolve this issue (this is likely how other libraries work despite the missing library). Only the c++ library is incompatible, the underlying c library should be compatible. If we created a VW shared object with a pure C interface, and called those functions directly from the python wrappers, we should be able to resolve the crashes. NOTE This is a long-term goal thats currently being discussed. There is no timeline or estimate for this change, but we will take this issue into account as we work on it. |
@scgalois Dear Scgalois, how do you change your version of python? I encounter the same problem, but cannot fix by now. Thank you very much for sharing your experience. |
Dear peterychang, I encounter the same problem and cannot fix by now. My MacOS is Mojave 10.14.6 with Python 3.7.7 and Clang 11.0.0. Can you help me try to resolve this issue? Thank you very much! |
@bmdy The only solution I know of is to uninstall Anaconda and use a compatible version of python, Brew's version should will work. If that's not an option, you may be able to find a conda repo that has a version of python built against a newer version of OSX, but it seems doubtful since it would introduce incompatibilities with base Anaconda packages. |
Thank you very much for your response. I've uninstalled Anaconda and use the Brew's version of Python. But I still encounter the problem: Library not loaded: @rpath/libpython3.7m.dylib. I've tried several ways to resolve this, but in vain. Can you help me in more details? |
@bmdy Have you rebuilt VW after uninstalling Anaconda? You'll need to If that doesn't work, please paste any output you get from the install process as well as the output of |
Yeah. I've completely uninstall vowpalwabbit. My process is as follow: $ pip install vowpalwabbit $ brew install cmake The default 'which python' gives '/usr/bin/python' and 'which python3' gives '/usr/local/bin/python3'. Below is the output: |
I'm wondering if you got the cached version of the binary. Try running the following command
This will force pip to rebuild the VW binary. If it continues to crash, please |
For documentation's sake, the command |
Thank you very much! It works. I can use vw to write code now. I will come to test more codes tomorrow and come back to you if some new issues occur. |
I've encountered a new problem. Now I need to write code on a server with Linux version 4.9.93. The python is Python 3.6.8 (with [GCC 7.3.0] on linux) with Miniconda3. How can I install vowpalwabbit on this environment? I've tried several ways following the instructions on this page https://pypi.org/project/vowpalwabbit/, but all failed. Thank you very much! Maybe I should open a new problem. I opened the #2370. You can paste your answer there. |
I am currently investigating if it is feasible to get wheels for MacOS and have these preliminary artifacts - https://github.com/jackgerrits/vowpal_wabbit/actions/runs/254408712 if you could try installing the relevant wheel from there and see if it resolves the issue it would be appreciated |
We have wheels produced for each commit now. Once we release 8.9 they will make it to pip so you can simply pip install vowpalwabbit. In the meantime check the wiki for a link to the wheel artifacts: https://github.com/VowpalWabbit/vowpal_wabbit/wiki/Python#bleeding-edge-latest-commit-on-master |
Now that VW 8.9 is released, there are now wheels available. Using If anyone hits this in future please feel free to comment/reopen the issue. |
There is a known issue on OSX 10.14+ with the currently deployed version of python on Anaconda.
The problem is as follows:
As a result, VW will typically fail to compile or install on the latest version of OSX. Using compiler options to force the use of libc++ will cause python to crash.
At the moment, the only resolutions available to users are to either uninstall Anaconda and use Homebrew or to copy the missing libraries from an older version of xcode.
We are currently exploring ways to resolve these compatibility issues, but there does not appear to be a quick or easy solution to the problem.
If this is a critical issue for you, please feel free to reach out to us so we can see if there is an individual solution that may work.
We also welcome anybody who would like to contribute to the resolution of this issue.
The text was updated successfully, but these errors were encountered: