Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Throws an error when using Qwen2-vl On Mac #1273

Closed
chigkim opened this issue Dec 20, 2024 · 5 comments
Closed

Throws an error when using Qwen2-vl On Mac #1273

chigkim opened this issue Dec 20, 2024 · 5 comments

Comments

@chigkim
Copy link

chigkim commented Dec 20, 2024

There was a bug that prevents from Qwen2-vl to run on Mac. The fix has been merged to llama.cpp yesterday.
ggml-org#10896
Thanks!
@chigkim
Copy link
Author

chigkim commented Dec 20, 2024

Here's the error:

ggml_metal_encode_node: error: unsupported op 'ROPE'
ggml/src/ggml-metal/ggml-metal.m:1263: unsupported op

@LostRuins
Copy link
Owner

I don't think wholesale disabling the GPU is a good idea. I'll wait and see if upstream develops a better solution - otherwise I'll only run clip on CPU for this specific model on macOS.

@chigkim
Copy link
Author

chigkim commented Dec 20, 2024

Isn't that what the fix did for now? Only run clip on cpu on mac?

@LostRuins
Copy link
Owner

Fixed, please update

@chigkim
Copy link
Author

chigkim commented Dec 21, 2024

Thanks it works now.

@chigkim chigkim closed this as completed Dec 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@chigkim @LostRuins