Segmentation fault when running llava #3637
Labels
Comments
|
I do not know if it is your problem, but gguf_init_from_file() returns null silently if it cannot access to the file of the model defined by the "--mmproj" parameter After that clip_model_load() does not check the value of "ctx", which leads to the crash. Surprisingly, in the last line, "segmentation fault ./llava -m models/llava/ggml-model-q5_k.gguf --mmproj --image example.jpg", the mmproj parameter appears empty |
That could be the cause in the code, but would make the function unable to access the file? I tried I also tried feeding some random files as the parameter, it then gives a segmentation fault as well, but prints
If
Maybe that points to what the problem is? Edit: I tried adding a |
|
Fixed in #3674 |
Not fixed. Running |
|
@michaelbogdan can you try the q4_k or f16 model? |
|
Sure, same result for the |
|
Sorry, cannot reproduce it on M2 Pro on master. It might be a build issue, though. Can you run the |
|
Running |
|
same issue here. I tried to transfer llava 1.5 model with finetuned weights. On cpu version it shows a segment fault. On CUBLAS version it showed: Aborted (core dumped) I tried both q5_k version and f16 model, neither worked. but the main program works fine. |
|
@DietDietDiet which model are you using?
|
|
|
No, |
@monatis got it, I tried a second time and make sure that option was passed into the script, still cannot find the mm.0.weight at runtime. Any ways to check the weights in gguf file? |
|
You can set verbosity to 3 instead of 1 on this line --it'll print out tensor names until it fails with llama.cpp/examples/llava/llava.cpp Line 38 in 2833a6f But a better way to debug this is to check import torch
proj = torch.load("path/to/llava.projector")
print(proj.keys())It should contain 4 tensors starting with |
I see where the problem happens. Seems that surgery.py will delete corresponding keys in the original checkpoint. When I repeat the process using the same checkpoint, the projector weights has been removed. Thanks for the great help! @monatis |
|
@monatis @DietDietDiet Does this discussion here mean that the "ready-made" weights I pulled are bad somehow? |
This is needed because the Pytorch checkpoints should contain only the LLaMA weights when running |
|
@michaelbogdan No. From your outputs you seem to run ./llava -m models/llava/ggml-model-q5_k.gguf --mmproj example.jpg --imageIt Should be instead: ./llava -m models/llava/ggml-model-q5_k.gguf --mmproj models/llava/mmproj-model-f16.gguf --image path/to/an/image.jpg
```, |
|
I came across a similar problem here #4196 (comment) Anyone could help? |
|
This issue was closed because it has been inactive for 14 days since being marked as stale. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Prerequisites
Please answer the following questions for yourself before submitting an issue.
Expected Behavior
./llava <arguments>should not run into asegmentation faultor at least offer some context for thesegmentation fault.Current Behavior
I run
./llava -m models/llava/ggml-model-q5_k.gguf --mmproj models/llava/mmproj-model-f16.gguf --image example.jpgand it outputsEnvironment and Context
Please provide detailed information about your computer setup. This is important in case the issue is not reproducible except for under certain specific conditions.
Failure Information (for bugs)
Please help provide information about the failure if this is a bug. If it is not a bug, please remove the rest of this template.
Steps to Reproduce
Please provide detailed steps for reproducing the issue. We are not sitting in front of your screen, so the more detail the better.
Failure Logs
Plain run
Edit:
Output of
lldb:The text was updated successfully, but these errors were encountered: