RuntimeError: The current installed version of g++ (12.2.1) is greater than the maximum required version by CUDA 11.7. Please make sure to use an adequate version of g++ (>=6.0.0, <12.0). #850

kaidohTips · 2023-04-06T23:21:10Z

Describe the bug

Hello everyone,

I'm facing a problem when trying to install the setup_cuda.py inside GPTQ-for-LLaMa.

I have a RuntimeError. It look like a problem due to my version of g++ (12.2.1). How can I make a downgrade to the 11 ?
I'm on Nobara 37.

Thanks

Is there an existing issue for this?

I have searched the existing issues

Reproduction

Just following the setup for 4 bit mode

mkdir repositories
cd repositories
git clone https://github.com/oobabooga/GPTQ-for-LLaMa.git -b cuda
cd GPTQ-for-LLaMa
python setup_cuda.py install    # ERROR AT THIS STEP

Screenshot

Logs

(textgen) ➜  GPTQ-for-LLaMa git:(cuda) ✗ python setup_cuda.py install                             
running install
/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
running bdist_egg
running egg_info
writing quant_cuda.egg-info/PKG-INFO
writing dependency_links to quant_cuda.egg-info/dependency_links.txt
writing top-level names to quant_cuda.egg-info/top_level.txt
reading manifest file 'quant_cuda.egg-info/SOURCES.txt'
writing manifest file 'quant_cuda.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
Traceback (most recent call last):
  File "/home/kaidoh/text-generation-webui/repositories/GPTQ-for-LLaMa/setup_cuda.py", line 4, in <module>
    setup(
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/dist.py", line 1208, in run_command
    super().run_command(command)
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/command/install.py", line 74, in run
    self.do_egg_install()
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/command/install.py", line 123, in do_egg_install
    self.run_command('bdist_egg')
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/dist.py", line 1208, in run_command
    super().run_command(command)
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/command/bdist_egg.py", line 165, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/command/bdist_egg.py", line 151, in call_command
    self.run_command(cmdname)
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/dist.py", line 1208, in run_command
    super().run_command(command)
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/_distutils/command/install_lib.py", line 112, in build
    self.run_command('build_ext')
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
    self.distribution.run_command(command)
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/dist.py", line 1208, in run_command
    super().run_command(command)
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 84, in run
    _build_ext.run(self)
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 346, in run
    self.build_extensions()
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 499, in build_extensions
    _check_cuda_version(compiler_name, compiler_version)
  File "/home/kaidoh/miniconda3/envs/textgen/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 416, in _check_cuda_version
    raise RuntimeError(
RuntimeError: The current installed version of g++ (12.2.1) is greater than the maximum required version by CUDA 11.7. Please make sure to use an adequate version of g++ (>=6.0.0, <12.0)

System Info

⠀⠀⢀⣤⣴⣶⣶⣶⣦⣤⡀⠀⣀⣠⣤⣴⣶⣶⣶⣶⣶⣶⣶⣶⣤⣤⣀⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀   --------- 
    ⠀⣰⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣶⣤⡀⠀⠀⠀⠀⠀⠀⠀⠀   OS: Nobara Linux 37 (Thirty Seve 
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣷⣄⠀⠀⠀⠀⠀⠀   Kernel: 6.2.8-200.fsync.fc37.x86 
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣷⣄⠀⠀⠀⠀   Uptime: 56 mins 
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣧⠀⠀⠀   Packages: 2807 (rpm), 19 (flatpa 
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠟⠋⠉⠁⠀⠀⠉⠉⠛⠿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣧⠀⠀   Shell: zsh 5.9 
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠟⠁⠀⠀⠀⢀⣀⣀⡀⠀⠀⠀⠈⢻⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡇⠀   Resolution: 1920x1080, 1080x1920 
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡏⠀⠀⠀⢠⣾⣿⣿⣿⣿⣷⡄⠀⠀⠀⠻⠿⢿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀   DE: GNOME 43.2 
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠁⠀⠀⠀⣿⣿⣿⣿⣿⣿⣿⡇⠀⠀⠀⠀⠀⣀⣀⣬⣽⣿⣿⣿⣿⣿⣿⠀   WM: Mutter 
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀⠀⠀⠀⠈⠻⢿⣿⣿⡿⠟⠁⠀⠀⠀⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀   WM Theme: CustomAccentColors 
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀   Theme: adw-gtk3-dark [GTK2/3] 
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣷⣤⣤⣄⣀⡀⠀⠀⠀⠀⠀⠀⠀⠀⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀   Icons: Papirus-Dark [GTK2/3] 
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣷⣄⠀⠀⠀⠀⠀⠀⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀   Terminal: gnome-terminal 
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣇⠀⠀⠀⠀⠀⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀   CPU: Intel i9-10900K (20) @ 5.30 
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡿⠟⠛⠉⠉⠛⠛⢿⣿⣿⠀⠀⠀⠀⠀⠸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡿⠀   GPU: NVIDIA GeForce RTX 3080 Ti 
    ⠀⠘⢿⣿⣿⣿⣿⣿⣿⣿⡿⠋⠀⠀⠀⠀⠀⠀⠀⠀⠈⢿⠀⠀⠀⠀⠀⠀⠙⢿⣿⣿⣿⣿⣿⣿⣿⠟⠁⠀   Memory: 5380MiB / 31955MiB 
    ⠀⠀⠀⠈⠙⠛⠛⠛⠋⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠉⠛⠛⠛⠛⠉⠁⠀⠀⠀

The text was updated successfully, but these errors were encountered:

practical-dreamer · 2023-04-07T00:46:09Z

I had some issue. Running gcc version 11.3.0 (Ubuntu 11.3.0-1ubuntu1~22.04)

Try this:

sudo apt install gcc-10 g++-10
export CC=/usr/bin/gcc-10
export CPP=/usr/bin/g++-10
python setup_cuda.py install

DFP- · 2023-04-07T01:40:23Z

I've been struggling with this for the past few days, it's ultimately a GPTQ-for-LLaMa problem so I'm not sure how much it belongs here, but the maintainer of that repo has switched to Triton development from cuda. When I try using gcc 10 (running on arch linux with cuda 11.1) I get the following error:

|g++-10 -shared -Wl,-rpath,/home/dfp/.conda/envs/textgen/lib -Wl,-rpath-link,/home/dfp/.conda/envs/textgen/lib -L/home/dfp/.conda/envs/textgen/lib -Wl,-rpath,/home/dfp/.conda/envs/textgen/lib -Wl,-rpath-link,/home/dfp/.conda/envs/textgen/lib -L/home/dfp/.conda/envs/textgen/lib /mnt/hynix/Code/text-generation-webui/repositories/GPTQ-for-LLaMa/build/temp.linux-x86_64-cpython-310/quant_cuda.o /mnt/hynix/Code/text-generation-webui/repositories/GPTQ-for-LLaMa/build/temp.linux-x86_64-cpython-310/quant_cuda_kernel.o -L/home/dfp/.conda/envs/textgen/lib/python3.10/site-packages/torch/lib -L/opt/cuda-11.1/lib64 -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-cpython-310/quant_cuda.cpython-310-x86_64-linux-gnu.so g++-10: error: /mnt/hynix/Code/text-generation-webui/repositories/GPTQ-for-LLaMa/build/temp.linux-x86_64-cpython-310/quant_cuda_kernel.o: No such file or directory
As best I can tell it simply doesn't make the quant_cuda_kernel.o, don't have a clue why and have mostly been trying to get through this by alternate routes.

allenhs · 2023-04-07T04:44:23Z

This is how I fixed from a user named plhosk (Thanks!):

pacman -S gcc11
export CC=gcc-11 CXX=g++-11
python setup_cuda.py install

DFP- · 2023-04-07T05:54:14Z

...that worked? What cuda do you have? I've tried that several times with cuda 12.1 before and nada

kaidohTips · 2023-04-07T13:10:33Z

This is how I fixed from a user named plhosk (Thanks!):

pacman -S gcc11 export CC=gcc-11 CXX=g++-11 python setup_cuda.py install

Can't install gcc11, even with the binaries

Yersi88 · 2023-04-07T14:57:11Z

This is how I fixed from a user named plhosk (Thanks!):

pacman -S gcc11 export CC=gcc-11 CXX=g++-11 python setup_cuda.py install

Doesn't work on Arch either:

[sudo] password for x: 
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
 multilib is up to date
error: target not found: gcc11

DFP- · 2023-04-07T16:47:38Z

Gcc11 is in thr AUR, should be fine installing it from there. I was lazy and just did yay gcc11

Not sure how to get that on nobara though

Yersi88 · 2023-04-08T18:18:08Z

Gcc11 is in thr AUR, should be fine installing it from there. I was lazy and just did yay gcc11

Not sure how to get that on nobara though

Unfortunately, yay gcc11 seems to get stuck for me at some point, never finishes. It's not a binary, so it builds it but fails.

ziontee113 · 2023-04-09T01:28:56Z

I was able to use downgrade (from AUR) to downgrade gcc to lower version.
This would requires you have to also downgrade gcc-libs, which is very important for most other programs.
Downgrade both gcc and gcc-libs worked made the thing compile for me.
Please remember to re-update both gcc and gcc-libs to the new versions, otherwise most of your programs WILL NOT WORK (spotify, firefox, etc...)

ghost · 2023-04-11T16:22:18Z

This is how I fixed from a user named plhosk (Thanks!):
pacman -S gcc11 export CC=gcc-11 CXX=g++-11 python setup_cuda.py install

Doesn't work on Arch either:
[sudo] password for x: 
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
 multilib is up to date
error: target not found: gcc11

try with yay

github-actions · 2023-11-05T23:16:14Z

This issue has been closed due to inactivity for 6 weeks. If you believe it is still relevant, please leave a comment below. You can tag a developer in your comment.

kaidohTips added the bug Something isn't working label Apr 6, 2023

da3dsoul mentioned this issue Apr 7, 2023

No module named 'llama_inference_offload' on Arch Linux #879

Closed

1 task

github-actions bot added the stale label Nov 5, 2023

github-actions bot closed this as completed Nov 5, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RuntimeError: The current installed version of g++ (12.2.1) is greater than the maximum required version by CUDA 11.7. Please make sure to use an adequate version of g++ (>=6.0.0, <12.0). #850

RuntimeError: The current installed version of g++ (12.2.1) is greater than the maximum required version by CUDA 11.7. Please make sure to use an adequate version of g++ (>=6.0.0, <12.0). #850

kaidohTips commented Apr 6, 2023 •

edited

Loading

practical-dreamer commented Apr 7, 2023 •

edited

Loading

DFP- commented Apr 7, 2023

allenhs commented Apr 7, 2023

DFP- commented Apr 7, 2023

kaidohTips commented Apr 7, 2023

Yersi88 commented Apr 7, 2023 •

edited

Loading

DFP- commented Apr 7, 2023 •

edited

Loading

Yersi88 commented Apr 8, 2023 •

edited

Loading

ziontee113 commented Apr 9, 2023 •

edited

Loading

ghost commented Apr 11, 2023

github-actions bot commented Nov 5, 2023

RuntimeError: The current installed version of g++ (12.2.1) is greater than the maximum required version by CUDA 11.7. Please make sure to use an adequate version of g++ (>=6.0.0, <12.0). #850

RuntimeError: The current installed version of g++ (12.2.1) is greater than the maximum required version by CUDA 11.7. Please make sure to use an adequate version of g++ (>=6.0.0, <12.0). #850

Comments

kaidohTips commented Apr 6, 2023 • edited Loading

Describe the bug

Is there an existing issue for this?

Reproduction

Screenshot

Logs

System Info

practical-dreamer commented Apr 7, 2023 • edited Loading

DFP- commented Apr 7, 2023

allenhs commented Apr 7, 2023

DFP- commented Apr 7, 2023

kaidohTips commented Apr 7, 2023

Yersi88 commented Apr 7, 2023 • edited Loading

DFP- commented Apr 7, 2023 • edited Loading

Yersi88 commented Apr 8, 2023 • edited Loading

ziontee113 commented Apr 9, 2023 • edited Loading

ghost commented Apr 11, 2023

github-actions bot commented Nov 5, 2023

kaidohTips commented Apr 6, 2023 •

edited

Loading

practical-dreamer commented Apr 7, 2023 •

edited

Loading

Yersi88 commented Apr 7, 2023 •

edited

Loading

DFP- commented Apr 7, 2023 •

edited

Loading

Yersi88 commented Apr 8, 2023 •

edited

Loading

ziontee113 commented Apr 9, 2023 •

edited

Loading