[Issue]: IPEX core dump #3733

RogerWeihrauch · 2025-01-31T10:07:56Z

Issue Description

Hi all. 1st) Thanks for all u'r effort on extending SDNext; it's a overwhelming tool.
So, I am sorry to report the following error which seems to throw a core dump when auto updating on a fresh setup with:
(venv) debian@bigdeb:~/GenArt/SDNext$ ./webui.sh --debug --use-ipex --upgrade ;
so no user-installed extensions yet exist, and no problem when setting SDNext up.
(when not using '--use-ipex' it falls back to CUDA)

' 10:36:44-243007 DEBUG    Extensions all: ['Lora', 'sd-extension-chainner', 'sd-extension-system-info',               
                         'sd-webui-agent-scheduler', 'sdnext-modernui', 'stable-diffusion-webui-rembg']              
10:36:44-762770 DEBUG    Git submodule: extensions-builtin/sd-extension-chainner / main                              
10:36:44-806230 DEBUG    Git submodule: extensions-builtin/sd-extension-system-info / main                           
10:36:44-849414 DEBUG    Git submodule: extensions-builtin/sd-webui-agent-scheduler / main                           
10:36:44-850965 DEBUG    Extension installer:                                                                        
                         /home/debian/GenArt/SDNext/extensions-builtin/sd-webui-agent-scheduler/install.py           
10:36:48-124282 DEBUG    Git detached head detected: folder="extensions-builtin/sdnext-modernui" reattach=main       
10:36:48-125653 DEBUG    Git submodule: extensions-builtin/sdnext-modernui / main                                    
10:36:48-167728 DEBUG    Git submodule: extensions-builtin/stable-diffusion-webui-rembg / master                     
10:36:48-169009 DEBUG    Extension installer:                                                                        
                         /home/debian/GenArt/SDNext/extensions-builtin/stable-diffusion-webui-rembg/install.py       
10:37:23-937165 DEBUG    Extensions all: []                                                                          
10:37:23-938395 INFO     Extensions enabled: ['Lora', 'sd-extension-chainner', 'sd-extension-system-info',           
                         'sd-webui-agent-scheduler', 'sdnext-modernui', 'stable-diffusion-webui-rembg']              
10:37:23-939630 INFO     Install: verifying requirements                                                             
10:37:23-940687 INFO     Updating Wiki                                                                               
10:37:23-959955 DEBUG    Git submodule: /home/debian/GenArt/SDNext/wiki / master                                     
10:37:26-110558 DEBUG    Setup complete without errors: 1738316246                                                   
10:37:26-114937 DEBUG    Extension preload: {'extensions-builtin': 0.0, 'extensions': 0.0}                           
10:37:26-117933 INFO     Command line args: ['--debug', '--use-ipex', '--upgrade'] upgrade=True use_ipex=True        
                         debug=True args=[]                                                                          
10:37:26-120942 DEBUG    Env flags: []                                                                               
10:37:26-122555 DEBUG    Linker flags: preload="libtcmalloc_minimal.so.4" path="/home/debian/GenArt/SDNext/venv/lib/"
10:37:26-124281 DEBUG    Starting module: <module 'webui' from '/home/debian/GenArt/SDNext/webui.py'>                
free(): invalid pointer
Aborted (core dumped) '

HINT: I am running this with integrated VENV activated.
Pls call if more files/extracts may be needed.
Thanks and regards,
Roger

Version Platform Description

(venv) debian@bigdeb:~/GenArt/SDNext$ lsb_release -a
No LSB modules are available.
Distributor ID:	Debian
Description:	Debian GNU/Linux trixie/sid
Release:	n/a
Codename:	trixie
(venv) debian@bigdeb:~/GenArt/SDNext$ 

(venv) debian@bigdeb:~/GenArt/SDNext$ uname -a
Linux bigdeb 6.12.10-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.12.10-1 (2025-01-18) x86_64 GNU/Linux
(venv) debian@bigdeb:~/GenArt/SDNext$ 

(venv) debian@bigdeb:~/GenArt/SDNext$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev             63G     0   63G   0% /dev
tmpfs            13G  3.4M   13G   1% /run
/dev/md1        1.9T   41G  1.9T   3% /
tmpfs            63G     0   63G   0% /dev/shm
efivarfs        128K   43K   81K  35% /sys/firmware/efi/efivars
tmpfs           5.0M   20K  5.0M   1% /run/lock
tmpfs            63G  960K   63G   1% /tmp
tmpfs           1.0M     0  1.0M   0% /run/credentials/systemd-journald.service
/dev/md2         15T   20G   15T   1% /var
/dev/md0        930G  136G  794G  15% /home
/dev/sdc2       1.4G  212M  1.2G  16% /boot
/dev/sdc1        52M  4.4M   47M   9% /boot/efi
tmpfs           1.0M     0  1.0M   0% /run/credentials/getty@tty1.service
tmpfs            13G  296K   13G   1% /run/user/1000
/dev/sdg1        15G  4.3G   11G  29% /media/debian/BACKUP
/dev/sdf2       895G  762G  133G  86% /media/debian/VboxMachine
(venv) debian@bigdeb:~/GenArt/SDNext$ 

(venv) debian@bigdeb:~/GenArt/SDNext$ pip -V
pip 24.0 from /home/debian/GenArt/SDNext/venv/lib/python3.12/site-packages/pip (python 3.12)
(venv) debian@bigdeb:~/GenArt/SDNext$ python -V
Python 3.12.3
(venv) debian@bigdeb:~/GenArt/SDNext$

Relevant log output

2025-01-31 10:36:25,306 | sd | INFO | installer | Verifying submodules
2025-01-31 10:36:35,820 | sd | DEBUG | installer | Git submodule: extensions-builtin/sd-extension-chainner / main
2025-01-31 10:36:38,967 | sd | DEBUG | installer | Git submodule: extensions-builtin/sd-extension-system-info / main
2025-01-31 10:36:39,777 | sd | DEBUG | installer | Git submodule: extensions-builtin/sd-webui-agent-scheduler / main
2025-01-31 10:36:40,304 | sd | DEBUG | installer | Git detached head detected: folder="extensions-builtin/sdnext-modernui" reattach=main
2025-01-31 10:36:40,306 | sd | DEBUG | installer | Git submodule: extensions-builtin/sdnext-modernui / main
2025-01-31 10:36:40,894 | sd | DEBUG | installer | Git submodule: extensions-builtin/stable-diffusion-webui-rembg / master
2025-01-31 10:36:41,395 | sd | DEBUG | installer | Git submodule: modules/k-diffusion / master
2025-01-31 10:36:41,915 | sd | DEBUG | installer | Git detached head detected: folder="wiki" reattach=master
2025-01-31 10:36:41,917 | sd | DEBUG | installer | Git submodule: wiki / master
2025-01-31 10:36:44,176 | sd | DEBUG | paths | Register paths
2025-01-31 10:36:44,241 | sd | DEBUG | installer | Installed packages: 206
2025-01-31 10:36:44,243 | sd | DEBUG | installer | Extensions all: ['Lora', 'sd-extension-chainner', 'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sdnext-modernui', 'stable-diffusion-webui-rembg']
2025-01-31 10:36:44,762 | sd | DEBUG | installer | Git submodule: extensions-builtin/sd-extension-chainner / main
2025-01-31 10:36:44,806 | sd | DEBUG | installer | Git submodule: extensions-builtin/sd-extension-system-info / main
2025-01-31 10:36:44,849 | sd | DEBUG | installer | Git submodule: extensions-builtin/sd-webui-agent-scheduler / main
2025-01-31 10:36:44,850 | sd | DEBUG | installer | Extension installer: /home/debian/GenArt/SDNext/extensions-builtin/sd-webui-agent-scheduler/install.py
2025-01-31 10:36:48,124 | sd | DEBUG | installer | Git detached head detected: folder="extensions-builtin/sdnext-modernui" reattach=main
2025-01-31 10:36:48,125 | sd | DEBUG | installer | Git submodule: extensions-builtin/sdnext-modernui / main
2025-01-31 10:36:48,167 | sd | DEBUG | installer | Git submodule: extensions-builtin/stable-diffusion-webui-rembg / master
2025-01-31 10:36:48,169 | sd | DEBUG | installer | Extension installer: /home/debian/GenArt/SDNext/extensions-builtin/stable-diffusion-webui-rembg/install.py
2025-01-31 10:37:23,937 | sd | DEBUG | installer | Extensions all: []
2025-01-31 10:37:23,938 | sd | INFO | installer | Extensions enabled: ['Lora', 'sd-extension-chainner', 'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sdnext-modernui', 'stable-diffusion-webui-rembg']
2025-01-31 10:37:23,939 | sd | INFO | installer | Install: verifying requirements
2025-01-31 10:37:23,940 | sd | INFO | installer | Updating Wiki
2025-01-31 10:37:23,959 | sd | DEBUG | installer | Git submodule: /home/debian/GenArt/SDNext/wiki / master
2025-01-31 10:37:26,110 | sd | DEBUG | launch | Setup complete without errors: 1738316246
2025-01-31 10:37:26,114 | sd | DEBUG | installer | Extension preload: {'extensions-builtin': 0.0, 'extensions': 0.0}
2025-01-31 10:37:26,117 | sd | INFO | launch | Command line args: ['--debug', '--use-ipex', '--upgrade'] upgrade=True use_ipex=True debug=True args=[]
2025-01-31 10:37:26,120 | sd | DEBUG | launch | Env flags: []
2025-01-31 10:37:26,122 | sd | DEBUG | launch | Linker flags: preload="libtcmalloc_minimal.so.4" path="/home/debian/GenArt/SDNext/venv/lib/"
2025-01-31 10:37:26,124 | sd | DEBUG | launch | Starting module: <module 'webui' from '/home/debian/GenArt/SDNext/webui.py'>
(venv) debian@bigdeb:~/GenArt/SDNext$

Backend

Original

UI

Standard

Branch

Master

Model

StableDiffusion 1.5

Acknowledgements

I have read the above and searched for existing issues
I confirm that this is classified correctly and its not an extension issue

The text was updated successfully, but these errors were encountered:

RogerWeihrauch · 2025-01-31T10:09:18Z

@ALL: I hope I did this correctly. :)
Regards,
Roger

vladmandic · 2025-01-31T14:03:44Z

try to capture the output of dmesg -ku command immediately after crash happens so we can see the kernel log - hopefully it should show where crash happened

Disty0 · 2025-01-31T14:26:28Z

libtcmalloc_minimal.so.4

Don't use TCMalloc, Intel libraries are prone to crashing with it.

JeMalloc can also crash with Intel but less often. You can try JeMalloc instead of TCMalloc.

RogerWeihrauch · 2025-01-31T15:27:47Z

Hi All, thaks for repl's.
@vladmandic: hope this helps:

[16872.184752] show_signal_msg: 37 callbacks suppressed
[16872.184762] python[361705]: segfault at 189 ip 00007f27e82ded98 sp 00007f273e6fbb20 error 4 in libze_intel_gpu.so.1.6.31294[4ded98,7f27e7e00000+6e8000] likely on CPU 12 (core 8, socket 0)
[16872.184794] Code: 31 c0 eb 6a 66 0f 1f 44 00 00 49 3b 70 10 0f 84 c6 00 00 00 48 89 1e 48 83 c6 08 31 c0 49 89 70 08 eb 4b 0f 1f 80 00 00 00 00 <0f> b6 82 89 01 00 00 84 c0 74 92 8b 05 4b 30 74 01 83 f8 ff 0f 84

@Disty0: How to use it (if already installed, automatically):

libjemalloc2/testing,now 5.3.0-2+b2 amd64 [installed,automatic]
  general-purpose scalable concurrent malloc(3) implementation

-> will it be automatuically used if I uninstall tcmalloc?
Regards,
Roger

Disty0 · 2025-01-31T15:30:53Z

export LD_PRELOAD=libjemalloc.so.2

This will also overwrite TCMalloc's activation.

To disable TCMalloc and not use any Malloc, find whatever is setting this env variable and remove it: LD_PRELOAD

RogerWeihrauch · 2025-01-31T15:47:36Z

@Disty0:
Well, t.b.h., since I am a real noob on this, I just searched and installed this malloc kind and it just 'worked' in a kind. ;)
Ok, will try u'r sugg'; thanks alot.
Regards,
Roger

RogerWeihrauch · 2025-01-31T15:54:46Z

@vladmandic: Q:

Could it be that it does not work in a NUMA env?
I was able to kind-of-split my CPU (AMD Threadripper 1900) into an emulated 2-core one with their own memory areas:

(venv) debian@bigdeb:~/GenArt/ComfyUI4ARC$ numactl -H
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 8 9 10 11
node 0 size: 64276 MB
node 0 free: 2549 MB
node 1 cpus: 4 5 6 7 12 13 14 15
node 1 size: 64508 MB
node 1 free: 4746 MB
node distances:
node   0   1 
  0:  10  16 
  1:  16  10 
(venv) debian@bigdeb:~/GenArt/ComfyUI4ARC$

Regards,
Roger

vladmandic · 2025-01-31T16:17:14Z

@RogerWeihrauch btw, pls use triple-backtick ``` (to open and again to close) instead of --- when quoting code/logs

RogerWeihrauch · 2025-01-31T16:18:43Z

@vladmandic 👍

RogerWeihrauch · 2025-01-31T16:21:50Z

@ALL:
Maybe this could be of some help; I am not sure:
(venv) debian@bigdeb:/GenArt/ComfyUI4ARC$ xpu-smi topology -m
GPU 0/0 CPU Affinity
GPU 0/0 S 0-3,8-11
(venv) debian@bigdeb:/GenArt/ComfyUI4ARC$
the other CPU cores belong to my GTX1050Ti GPU.
Regards,
Roger

Disty0 · 2025-01-31T16:24:36Z

Can you try disabling TCMalloc with this command:
This will clear any modification done to the LD_PRELOAD env variable.

unset LD_PRELOAD

free(): invalid pointer
Aborted (core dumped)

This error message is what happens when Intel Compute Runtime itself (libze and libigc) crashes if TCMalloc is enabled.
I don't think it is related to numa as it happens on my R7 5800X3D and R5 1600X (both have only one ccd) as well.

RogerWeihrauch · 2025-02-01T09:38:16Z

Hi All.
Seems I figured out this temporary error:
I had a misconfiguration in my UEFI/BIOS settings on core voltage which seemed to make the CPU Core unstable: similar errors/behaviour I also saw in different other programs. So I tested this more deeply and found the source for this.
BTW: I did not disable tcmalloc/jemalloc, since this works now for me.
So, I am sorry to bother you with an error made on my side.
Pls consider this as closed.
Thanks for your support.
Kind regards,
Roger

vladmandic added the question Further information is requested label Jan 31, 2025

vladmandic changed the title ~~[Issue]: Error on '.. --updater/ --upgrade': 'free(): invalid pointer - Aborted (core dumped)' when doing auto updating SDNext~~ [Issue]: IPEX core dump Jan 31, 2025

vladmandic closed this as completed Feb 1, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Issue]: IPEX core dump #3733

[Issue]: IPEX core dump #3733

RogerWeihrauch commented Jan 31, 2025 •

edited by vladmandic

Loading

RogerWeihrauch commented Jan 31, 2025

vladmandic commented Jan 31, 2025

Disty0 commented Jan 31, 2025 •

edited

Loading

RogerWeihrauch commented Jan 31, 2025 •

edited by vladmandic

Loading

Disty0 commented Jan 31, 2025 •

edited

Loading

RogerWeihrauch commented Jan 31, 2025

RogerWeihrauch commented Jan 31, 2025 •

edited by vladmandic

Loading

vladmandic commented Jan 31, 2025 •

edited

Loading

RogerWeihrauch commented Jan 31, 2025

RogerWeihrauch commented Jan 31, 2025

Disty0 commented Jan 31, 2025 •

edited

Loading

RogerWeihrauch commented Feb 1, 2025

[Issue]: IPEX core dump #3733

[Issue]: IPEX core dump #3733

Comments

RogerWeihrauch commented Jan 31, 2025 • edited by vladmandic Loading

Issue Description

Version Platform Description

Relevant log output

Backend

UI

Branch

Model

Acknowledgements

RogerWeihrauch commented Jan 31, 2025

vladmandic commented Jan 31, 2025

Disty0 commented Jan 31, 2025 • edited Loading

RogerWeihrauch commented Jan 31, 2025 • edited by vladmandic Loading

Disty0 commented Jan 31, 2025 • edited Loading

RogerWeihrauch commented Jan 31, 2025

RogerWeihrauch commented Jan 31, 2025 • edited by vladmandic Loading

vladmandic commented Jan 31, 2025 • edited Loading

RogerWeihrauch commented Jan 31, 2025

RogerWeihrauch commented Jan 31, 2025

Disty0 commented Jan 31, 2025 • edited Loading

RogerWeihrauch commented Feb 1, 2025

RogerWeihrauch commented Jan 31, 2025 •

edited by vladmandic

Loading

Disty0 commented Jan 31, 2025 •

edited

Loading

RogerWeihrauch commented Jan 31, 2025 •

edited by vladmandic

Loading

Disty0 commented Jan 31, 2025 •

edited

Loading

RogerWeihrauch commented Jan 31, 2025 •

edited by vladmandic

Loading

vladmandic commented Jan 31, 2025 •

edited

Loading

Disty0 commented Jan 31, 2025 •

edited

Loading