examples/optimal-performance/multi.py script not working. RuntimeError: Cannot re-initialize CUDA in forked subprocess.

[c4fun]

Environment

• ubuntu 22.04
• rtx 4090

Problem description

After typing in python optimal-performance/multi.py or python optimal-performance/single.py as per instruction, this problem occurs:

$ python optimal-performance/multi.py 
Loading pipeline components...: 100%|███████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 16.01it/s]
You have disabled the safety checker for <class 'diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion.StableDiffusionPipeline'> by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
Traceback (most recent call last):
  File "/home/c4fun/code/github.com/cumulo-autumn/StreamDiffusion/examples/optimal-performance/../../utils/wrapper.py", line 411, in _load_model
    pipe: StableDiffusionPipeline = StableDiffusionPipeline.from_pretrained(
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/diffusers-0.24.0-py3.10.egg/diffusers/pipelines/pipeline_utils.py", line 864, in to
    module.to(device, dtype)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/transformers-4.36.2-py3.10.egg/transformers/modeling_utils.py", line 2460, in to
    return super().to(*args, **kwargs)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1160, in to
    return self._apply(convert)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 810, in _apply
    module._apply(fn)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 810, in _apply
    module._apply(fn)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 810, in _apply
    module._apply(fn)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 833, in _apply
    param_applied = fn(param)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1158, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/cuda/__init__.py", line 284, in _lazy_init
    raise RuntimeError(
RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method
Model load has failed. Doesn't exist.

[Ezharjan]

Environment

* ubuntu 22.04

* rtx 4090

Problem description

After typing in python optimal-performance/multi.py or python optimal-performance/single.py as per instruction, this problem occurs:

$ python optimal-performance/multi.py 
Loading pipeline components...: 100%|███████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 16.01it/s]
You have disabled the safety checker for <class 'diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion.StableDiffusionPipeline'> by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
Traceback (most recent call last):
  File "/home/c4fun/code/github.com/cumulo-autumn/StreamDiffusion/examples/optimal-performance/../../utils/wrapper.py", line 411, in _load_model
    pipe: StableDiffusionPipeline = StableDiffusionPipeline.from_pretrained(
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/diffusers-0.24.0-py3.10.egg/diffusers/pipelines/pipeline_utils.py", line 864, in to
    module.to(device, dtype)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/transformers-4.36.2-py3.10.egg/transformers/modeling_utils.py", line 2460, in to
    return super().to(*args, **kwargs)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1160, in to
    return self._apply(convert)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 810, in _apply
    module._apply(fn)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 810, in _apply
    module._apply(fn)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 810, in _apply
    module._apply(fn)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 833, in _apply
    param_applied = fn(param)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1158, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/cuda/__init__.py", line 284, in _lazy_init
    raise RuntimeError(
RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method
Model load has failed. Doesn't exist.

I'm having the same issue on:

• CUDA 12.1
• GPU 3090
• Linux Ubuntu 20.04 LTS
• Anaconda Environment for StreamDiffusion

[Ezharjan]

Environment

* ubuntu 22.04

* rtx 4090

Problem description

After typing in python optimal-performance/multi.py or python optimal-performance/single.py as per instruction, this problem occurs:

$ python optimal-performance/multi.py 
Loading pipeline components...: 100%|███████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 16.01it/s]
You have disabled the safety checker for <class 'diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion.StableDiffusionPipeline'> by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
Traceback (most recent call last):
  File "/home/c4fun/code/github.com/cumulo-autumn/StreamDiffusion/examples/optimal-performance/../../utils/wrapper.py", line 411, in _load_model
    pipe: StableDiffusionPipeline = StableDiffusionPipeline.from_pretrained(
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/diffusers-0.24.0-py3.10.egg/diffusers/pipelines/pipeline_utils.py", line 864, in to
    module.to(device, dtype)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/transformers-4.36.2-py3.10.egg/transformers/modeling_utils.py", line 2460, in to
    return super().to(*args, **kwargs)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1160, in to
    return self._apply(convert)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 810, in _apply
    module._apply(fn)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 810, in _apply
    module._apply(fn)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 810, in _apply
    module._apply(fn)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 833, in _apply
    param_applied = fn(param)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1158, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/cuda/__init__.py", line 284, in _lazy_init
    raise RuntimeError(
RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method
Model load has failed. Doesn't exist.

I realized that I used a different version of CUDA (with CUDA 12.2) while the documentation recommends CUDA12.1 or CUDA 11.8.
I fixed the problem by running the commands below, maybe the following commands (especially the third line) would also be helpful for you:

conda create -n streamdiffusion python=3.10
conda activate streamdiffusion
conda install -c "nvidia/label/cuda-12.1.0" cuda # this is the crucial step!
pip3 install torch==2.1.0 torchvision==0.16.0 xformers --index-url https://download.pytorch.org/whl/cu121
git clone https://github.com/cumulo-autumn/StreamDiffusion.git
python setup.py develop easy_install streamdiffusion[tensorrt]
python -m streamdiffusion.tools.install-tensorrt
python txt2img/single.py --output output.png --prompt "A cat with a hat"

[samezi-but]

me too.
Windows WSL Ubuntu22.04
RTX A4500 NVLINK*2
CUDA 12.1

[teftef6220]

@Ezharjan

Thanks, you are right.

Please check if the CUDA version, torch version and cuDNN version match.

https://docs.nvidia.com/deeplearning/cudnn/support-matrix/index.html

[c4fun]

Environment

* ubuntu 22.04

* rtx 4090

Problem description

After typing in python optimal-performance/multi.py or python optimal-performance/single.py as per instruction, this problem occurs:

$ python optimal-performance/multi.py 
Loading pipeline components...: 100%|███████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 16.01it/s]
You have disabled the safety checker for <class 'diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion.StableDiffusionPipeline'> by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
Traceback (most recent call last):
  File "/home/c4fun/code/github.com/cumulo-autumn/StreamDiffusion/examples/optimal-performance/../../utils/wrapper.py", line 411, in _load_model
    pipe: StableDiffusionPipeline = StableDiffusionPipeline.from_pretrained(
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/diffusers-0.24.0-py3.10.egg/diffusers/pipelines/pipeline_utils.py", line 864, in to
    module.to(device, dtype)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/transformers-4.36.2-py3.10.egg/transformers/modeling_utils.py", line 2460, in to
    return super().to(*args, **kwargs)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1160, in to
    return self._apply(convert)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 810, in _apply
    module._apply(fn)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 810, in _apply
    module._apply(fn)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 810, in _apply
    module._apply(fn)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 833, in _apply
    param_applied = fn(param)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1158, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
  File "/opt/anaconda3/envs/streamdiffusion/lib/python3.10/site-packages/torch/cuda/__init__.py", line 284, in _lazy_init
    raise RuntimeError(
RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method
Model load has failed. Doesn't exist.

I realized that I used a different version of CUDA (with CUDA 12.2) while the documentation recommends CUDA12.1 or CUDA 11.8. I fixed the problem by running the commands below, maybe the following commands (especially the third line) would also be helpful for you:

conda create -n streamdiffusion python=3.10
conda activate streamdiffusion
conda install -c "nvidia/label/cuda-12.1.0" cuda # this is the crucial step!
pip3 install torch==2.1.0 torchvision==0.16.0 xformers --index-url https://download.pytorch.org/whl/cu121
git clone https://github.com/cumulo-autumn/StreamDiffusion.git
python setup.py develop easy_install streamdiffusion[tensorrt]
python -m streamdiffusion.tools.install-tensorrt
python txt2img/single.py --output output.png --prompt "A cat with a hat"

Thanks. This solves my problem. Works like a charm.