WSL CUDA Setup failed despite GPU being available, only 4bit working

[iChristGit]

Describe the bug

I am setting up Webui with WSL, followed all the steps, running normally or 8-bit doesnt work, 4bit does.

I dont know how to fix that.

Is there an existing issue for this?

• I have searched the existing issues

Reproduction

~fresh install using instructions

Screenshot

No response

Logs

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
CUDA SETUP: Loading binary /home/ultra/miniconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda116.dll...
/home/ultra/miniconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda116.dll: invalid ELF header
CUDA SETUP: Loading binary /home/ultra/miniconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda116.dll...
/home/ultra/miniconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda116.dll: invalid ELF header
CUDA SETUP: Loading binary /home/ultra/miniconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda116.dll...
/home/ultra/miniconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda116.dll: invalid ELF header
CUDA SETUP: Problem: The main issue seems to be that the main CUDA library was not detected.
CUDA SETUP: Solution 1): Your paths are probably not up-to-date. You can update them via: sudo ldconfig.
CUDA SETUP: Solution 2): If you do not have sudo rights, you can do the following:
CUDA SETUP: Solution 2a): Find the cuda library via: find / -name libcuda.so 2>/dev/null
CUDA SETUP: Solution 2b): Once the library is found add it to the LD_LIBRARY_PATH: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:FOUND_PATH_FROM_2a
CUDA SETUP: Solution 2c): For a permanent solution add the export from 2b into your .bashrc file, located at ~/.bashrc
Traceback (most recent call last):
  File "/home/ultra/text-generation-webui/server.py", line 241, in <module>
    shared.model, shared.tokenizer = load_model(shared.model_name)
  File "/home/ultra/text-generation-webui/modules/models.py", line 49, in load_model
    model = AutoModelForCausalLM.from_pretrained(Path(f"models/{shared.model_name}"), device_map='auto', load_in_8bit=True)
  File "/home/ultra/miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 471, in from_pretrained
    return model_class.from_pretrained(
  File "/home/ultra/miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2503, in from_pretrained
    from .utils.bitsandbytes import get_keys_to_not_convert, replace_8bit_linear
  File "/home/ultra/miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/utils/bitsandbytes.py", line 7, in <module>
    import bitsandbytes as bnb
  File "/home/ultra/miniconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/__init__.py", line 7, in <module>
    from .autograd._functions import (
  File "/home/ultra/miniconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/autograd/__init__.py", line 1, in <module>
    from ._functions import undo_layout, get_inverse_transform_indices
  File "/home/ultra/miniconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py", line 9, in <module>
    import bitsandbytes.functional as F
  File "/home/ultra/miniconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/functional.py", line 17, in <module>
    from .cextension import COMPILED_WITH_CUDA, lib
  File "/home/ultra/miniconda3/envs/textgen/lib/python3.10/site-packages/bitsandbytes/cextension.py", line 22, in <module>
    raise RuntimeError('''
RuntimeError:
        CUDA Setup failed despite GPU being available. Inspect the CUDA SETUP outputs above to fix your environment!
        If you cannot find any issues and suspect a bug, please open an issue with detals about your environment:
        https://github.com/TimDettmers/bitsandbytes/issues

System Info

Windows11, WSL Ubuntu install, followed install steps and fixes

[NolanMangalMM]

did you make the special changes required for windows bitsandbytes?

#147 (comment)

[github-actions]

This issue has been closed due to inactivity for 30 days. If you believe it is still relevant, please leave a comment below.