[ISSUE] --gpu-layers in useless.

[skidd-level-100]

I am on linux with an nvidia card using CLBLAST (CUBLAST hates me) and --gpu-layers (any number) is no faster than without it,
I checked the nvidia-smi ouput and it is showing me only 80mib vram used (same as without -ngl)

pls help

[Green-Sky]

Please use the cuda/cublas variant over the OpenCL one. It is always better on nvidia hardware.

[JohannesGaessler]

The PR for OpenCL GPU acceleration #1459 hasn't been merged yet so setting --n-gpu-layers with LLAMA_CLBLAST does nothing.

[KerfuffleV2]

Would it be a good idea to have --n-gpu-layers fail if stuff isn't compiled in a way that enables actually putting layers on the GPU? Could probably just add some #ifdefs around the commandline option unless there's actually a reason to allow the user to use the argument even when there's no effect. (I guess an alternative is just to display a warning/notice in that case rather than raising an error.)

[JohannesGaessler]

I opened a PR that adds documentation and an error upon trying to use --n-gpu-layers without an implementation. I'll revisit it once my other PR for optimization options gets merged.