[1.1.428] IP-Adapter FaceID

[huchenlei]

Update 2024-01-24

SDXL FaceID Plus v2 is added to the models list. You can use it without any code changes.

IP-Adapter FaceID

IP-Adapter FaceID provides a way to extract only face features from an image and apply it to the generated image. Please follow the guide to try this new feature.

Installation of insightface

FaceID requires insightface, you need to install them in your A1111 venv. Check this ComfyUI issue for help.

In summary, building insightface from source requies compiling some C++ code, which in principle should be avoided at all cost in this repository. See #1875. The workaround this PR has is downloading the pre-build wheel for the largest user group (Windows amd64 users). Other users will need build from source by running pip install insightface.

Models

The following table shows the combination of checkpoint and preprocessor to use for each FaceID IPAdapter Model. Any Tensor size mismatch you may get it is likely caused by a wrong combination.

SD v.	IPadapter	Preprocessor	LoRA
v1.5	FaceID	ip-adapter_face_id	FaceID LoRA
v1.5	FaceID Plus	ip-adapter_face_id_plus	FaceID Plus LoRA
v1.5	FaceID Plus v2	ip-adapter_face_id_plus	FaceID Plus v2 LoRA
SDXL	FaceID SDXL	ip-adapter_face_id	FaceID SDXL LoRA
SDXL	FaceID Plus v2 SDXL	ip-adapter_face_id_plus	FaceID Plus v2 SDXL LoRA

It is recommended to load the corresponding LoRA when generating with a weight of 0.5 ~ 0.7.

How to use

Step1: Load LoRA

Step2: Configure ControlNet. Choose preprocessor and model.

Step3: Generate

Face input:

Output:

Next Steps

Please also comment on which of the following items should/should not be implemented. Thanks!

• Automatically apply corresponding LoRA
• Unload insightface model. (Is that necessary? We already run insightface on CPU)
• Documentation on IPAdapter usage
• Automatically pick preprocessor/model when one is selected

Notes

• IP-Adapter FaceID implementation currently only extracts features from the first face detected in the image.

[septerlt]

FaceID Plus v2 not working. The other two adapters - faceid and faceid plus work fine.
If anyone can achieve good results with FaceID Plus v2, please share the workflow

[dan4ik94]

I can confirm, v2 isn't working.

[huchenlei]

Can you share your v2 setup + results?

[xpeng]

confirmed.

portrait of 1girl, <lora:ip-adapter-faceid-plusv2_sd15_lora:0.6>
Negative prompt: easynegative
Steps: 30, Sampler: Euler a, CFG scale: 7, Seed: 3667872229, Size: 768x768, Model hash: ef76aa2332, Model: realisticVisionV51_v51VAE, ControlNet 0: "Module: ip-adapter_face_id_plus, Model: ip-adapter-faceid-plusv2_sd15 [6e14fc1a], Weight: 1, Resize Mode: Crop and Resize, Low Vram: False, Processor Res: 512, Guidance Start: 0, Guidance End: 1, Pixel Perfect: True, Control Mode: Balanced, Hr Option: Both, Save Detected Map: True", Lora hashes: "ip-adapter-faceid-plusv2_sd15_lora: a95a0f4bdcb9", TI hashes: "EasyNegative: 66a7279a88dd", Version: v1.7.0

[huchenlei]

Fixed in #2448. Thanks for reporting the issue!

[Soulreaver90]

Awesome, it is working now.

[Soulreaver90]

Haven't tried v2, but the rest work exceedingly well on initial impressions. I only had a tough time installing insighface (I'm on ubuntu 22.0.4). I had to install python dev and then everything installed correctly. Thanks!

[Soulreaver90]

I've gone back and forth between Plus v2 and regular Plus, and I tend to prefer the original a bit more. Could be my subjects and prompts, but Plus seems more accurate to the model with the trade off being a more "cut and paste" look, whereas Plus v2 seems more dynamic with very similar features, but not exactly as accurate.

[Soulreaver90]

Doesn't seem to work with the regional prompt extension, no face is altered when it's enabled. Bummer.

[huchenlei]

Reference discussion on model comparisons: cubiq/ComfyUI_IPAdapter_plus#195

[BNP1111]

Hi, I have downloaded ip-adapter-faceid-plus_sd15.bin，
which folder should I put it in? thank U

[huchenlei]

Put it under stable-diffusion-webui\models\ControlNet.

[IndigoDosSantos]

Next Steps

Please also comment on which of the following items should/should not be implemented. Thanks!

Automatically apply corresponding LoRA
Documentation on IPAdapter usage
Automatically pick preprocessor/model when one is selected

Especially the documentation on its usage would be cool. And if you find the time, both other points would be nice for more ease of use.

[Mistarille224]

When I use the ip-adapter_face_id_plus preprocessor, an error like this is returned. What should I do?
TypeError: tuple indices must be integers or slices, not str

[johndpope]

seems like a python version error? are you on 3.11?

[huchenlei]

You are trying to use plus preprocessor on a nonplus model. Please pick the correct model/preprocessor pair.

[Mistarille224]

Solved. I forgot to change it. THX.

[Mistarille224]

In fact, these things also work very well on python 3.10.13.

[cxiaowen]

ip-adapter_face_id_plus 这个处理器在哪里可以下载

[suckonw]

[huchenlei]

This is not an error message. The log is expected. You probably just need to wait for it to complete.

[RogerTB22]

O meu em Preprocessor esta aparecendo "ip-adapter_clip_sd15" e nao "ip-adapter_face_id" e nao da certo... pq sera?
Obrigado!

[IndigoDosSantos]

You're most likely not using the latest ControlNet version. Unfortunately couldn't verify, since version number is cut off in your screenshot.

Você provavelmente não está usando a versão mais recente do ControlNet. Infelizmente, não pude verificar, pois o número da versão está cortado na sua captura de tela.

[sd944361]

I downloaded the files for FaceID V2 and XL and placed them in the correct directory. ControlNet has been updated to version 1.431. Why can't I find the corresponding V2 and XL files in the preprocessor?

[sd944361]

Could you please provide a link

[huchenlei]

You just need to scroll up to the top

[sd944361]

Thank you very much! But I downloaded the files for FaceID Plus V2 and FaceID from that page and placed them in the models/ControlNet folder. However, why can't I find these two preprocessors in the preprocessing section on the webpage?

[huchenlei]

You use ip-adapter_face_id_plus. See Preprocessor column.

[sd944361]

Thank you for your patient answers

[treksis]

Always thanks for the work. A request here. h94 just dropped another ip-adapter. It seems better than previous adapter.

https://huggingface.co/h94/IP-Adapter-FaceID/blob/main/faceid_portrait_sd15.jpg
https://huggingface.co/h94/IP-Adapter-FaceID/blob/main/ip-adapter-faceid-portrait_sd15.bin

[IndigoDosSantos]

Do you happen to know how the newly added "ip-adapter-faceid-portrait_sd15" works? According to the IP-Adapter-FaceID model card it neither uses ControlNet, nor LoRA. So, where is it supposed to be used?

[treksis]

I have no idea how controlnet is working. I think we need to add some code below to make it work in auto1111 environment. Honestly I suck at ML, no idea.
tencent-ailab/IP-Adapter@2397f7c

[IndigoDosSantos]

According to @huchenlei it's not functional currently (see #2527). Perhaps it will be released in 1.1.434? ٩( ๑╹ ꇴ╹)۶

Related discussion: #2478 (comment).
Original request: #2245.

User has requested that we provide a way to easily input the whole directory of images into a unit. This PR partially does that by providing the Multi-Inputs tab. Now you can specify a directory with N images. N ControlNet units will be added on generation each unit accepting 1 image from the dir.

The feature can be very useful on IPAdapter units, as we can create "instant LoRA" with multiple input images from a directory. Previously, you would need to enable multiple ControlNet units, and upload images one by one. This also paves the way for implementing IPAdapter portrait: tencent-ailab/IP-Adapter@2397f7c.

Note:
You do not need to increase ControlNet unit count to be greater than the number of images in your directory.
Merging 4 images with IPAdapter (weight=0.2), while only have 3 ControlNet unit in settings.

[HEONDRED]

I have no idea why it's not working...
Even i used a exact model on preprocessor. thx for helping me...

[huchenlei]

You need to install onnxruntime via pip. We already have onnxruntime listed as dependency, but somehow it is not automatically installed for you. You can check the full log message, there might be a line saying that onnxruntime install failed and some preprocessors might not work.

[pixerdream]

Help!Why does the face I change out look nothing like it? Do I need to pay attention to any parameter settings? I have tried to increase the weights of CN and Lora

[Victoryship]

So I guess I haven't installed it properly somewhere. But I can't find any error. :(

[dan4ik94]

@Victoryship
Forgot to add, I'm on huchenlei's branch with optimizations enabled, maybe you can try.
AUTOMATIC1111/stable-diffusion-webui#15821

[Victoryship]

thinks @dan4ik94 I found the problem

I found that the buffalo_l folder in the stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/downloads/insightface/models directory is empty and the buffalo_l.zip file in this directory is only 55MB.

So I re-downloaded this file from https://github.com/deepinsight/insightface/releases/download/v0.7/buffalo_l.zip. And put it in the stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/downloads/insightface/models directory and extract it to the buffalo_l folder in the current directory. like this

and it works

[dan4ik94]

great, as far as I know, it gets downloaded automatically.

[Victoryship]

Yes, it downloaded automatically but some files are missing. Only 55MB.I don't know why

[qbn-smu]

从该页面下载了 FaceID Plus V2 和 FaceID 的模型文件，并将它们放在 models/ControlNet 文件夹中。但是，为什么在web ui 的预处理部分找不到这两个预处理器？

[huchenlei]

Update your ControlNet extension please.

[t59688]

Can multiple images be used as input sources?

[a3nima]

yes, you can use a second controlnet ip adapter for example. I found that weights should be lower in the second one to not get distorted areas in the face. // edit: multi images input then you dont need a second controlnet

[dinosaursarecool]

This is amazing, great job! Wondering if there's any way to speed up the time to hook the controlnet when a new image is used?

I have a distributed application that requires fast inference speeds across multiple servers and I'm looking at optimizing the initial loading time on a new server. When a non-cached image is used, processing and hooking the controlnet takes seconds, while subsequent runs take milliseconds; how can I go about speeding up the first run on a new server when using the API? For instance, is there a way to precompute the image and pass some embedding through the API instead of the image itself? Or perhaps a way to extract the cache for a given image and pass that to another server?

[huchenlei]

There is a WIP PR: #2590. A1111 has some security checks to disallow torch.load unsanitized bytes that I need to find a way to bypass.

[dinosaursarecool]

That's awesome, great to hear

[ryanwitsqo]

I got this error when I try to run IP-adapter on Automatic1111

is there something that i've been missing? I download the model and paste it on the Controlnet folder right away.

[a3nima]

Strange thing I've noticed yesterday: If I set the weight of the controlnet IP Adapter at 1, actually it will be used (or at least the generation info says so) with the value /3 = 0.333.. If I set the weight at 0.6 its 1.9999.. -- what's happening here? (I'm on Forge).

ControlNet 0: "Module: InsightFace+CLIP-H (IPAdapter), Model: ip-adapter-faceid-plusv2_sdxl [187cb962], Weight: 0.3333333333333333, Resize Mode: Crop and Resize, Processor Res: 0.5, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0, Guidance End: 1, Pixel Perfect: True, Control Mode: Balanced, Hr Option: Both", ControlNet 1: "Module: InsightFace+CLIP-H (IPAdapter), Model: ip-adapter-faceid-plusv2_sdxl [187cb962], Weight: 0.3333333333333333, Resize Mode: Crop and Resize, Processor Res: 0.5, Threshold A: 0.5, Threshold B: 0.5, Guidance Start: 0, Guidance End: 1, Pixel Perfect: True, Control Mode: Balanced, Hr Option: Both",

[mehdi7575]

I don't have anything called ip-adapter-face-id-plus in pre-processing and I only have CLIP-ViT-H (IPAdapter), CLIP-ViT-bigG (IPAdapter), InsightFace+CLIP-H (IPAdapter). Let me say that I am using web ui forge. Please guide me how can I add ip-adapter-face-id-plus to preprocessing?

[huchenlei]

The ip-adapter-face-id-plus preprocessor is called InsightFace+CLIP-H (IPAdapter) in forge. BTW, we also have ip-adapter-auto now to automatically select the correct preprocessor based on model name here. Forge impl probably need to wait until lvmin comes back. #2723

[INTstinkt]

I think the new update broke FaceID it doesn't matter which picture i use in Controlnet i always get the same error message:
2024-04-01 00:04:12,215 - ControlNet - INFO - ip-adapter-auto => ip-adapter_face_id_plus
2024-04-01 00:04:12,215 - ControlNet - INFO - unit_separate = False, style_align = False
2024-04-01 00:04:12,510 - ControlNet - INFO - Loading model: ip-adapter-faceid-plusv2_sdxl [187cb962]
2024-04-01 00:04:12,774 - ControlNet - INFO - Loaded state_dict from [C:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\models\ip-adapter-faceid-plusv2_sdxl.bin]
2024-04-01 00:04:15,128 - ControlNet - INFO - ControlNet model ip-adapter-faceid-plusv2_sdxl 187cb962 loaded.
2024-04-01 00:04:15,129 - ControlNet - INFO - using mask
2024-04-01 00:04:15,133 - ControlNet - INFO - Using preprocessor: ip-adapter_face_id_plus
2024-04-01 00:04:15,133 - ControlNet - INFO - preprocessor resolution = 512
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: C:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads\insightface\models\buffalo_l\1k3d68.onnx landmark_3d_68 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: C:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads\insightface\models\buffalo_l\2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: C:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads\insightface\models\buffalo_l\det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: C:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads\insightface\models\buffalo_l\genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: C:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads\insightface\models\buffalo_l\w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5
set det-size: (640, 640)
*** Error running process: C:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py
Traceback (most recent call last):
File "C:\AI\stable-diffusion-webui\modules\scripts.py", line 784, in process
script.process(p, script_args)
File "C:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1245, in process
self.controlnet_hack(p)
File "C:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1230, in controlnet_hack
self.controlnet_main_entry(p)
File "C:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 995, in controlnet_main_entry
controls, hr_controls = list(zip([preprocess_input_image(img) for img in optional_tqdm(input_images)]))
File "C:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 995, in
controls, hr_controls = list(zip(*[preprocess_input_image(img) for img in optional_tqdm(input_images)]))
File "C:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 952, in preprocess_input_image
detected_map, is_image = self.preprocessor[unit.module](
File "C:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\utils.py", line 81, in decorated_func
return cached_func(*args, **kwargs)
File "C:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\utils.py", line 65, in cached_func
return func(*args, **kwargs)
File "C:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\global_state.py", line 37, in unified_preprocessor
return preprocessor_modules[preprocessor_name](*args, **kwargs)
File "C:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\processor.py", line 848, in face_id_plus
face_embed, _ = g_insight_face_model.run_model(img)
File "C:\AI\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\processor.py", line 771, in run_model
raise Exception("Insightface: No face found in image.")
Exception: Insightface: No face found in image.

I used the same pictures just yesterday to do some testing also tried some new pictures but it doesnt work at all anymore

[huchenlei]

Can you file an issue including the input image you used? Thanks!

[INTstinkt]

Yes sir, i did as you told me!

[huchenlei]

Fixed. Thanks for spotting the issue!

[TigerHH6866]

v2 is not good for skin color in img2img

img for input

img for faceid input

using plus
1 girl,upper body,lora:ip-adapter-faceid-plusv2_sd15_lora:0.4
Negative prompt: (worst quality:2),(low quality:2),(normal quality:2),lowres,watermark,tail,fuzzy,blurry, watermark, badhandv4 UnrealisticDream
Steps: 30, Sampler: DPM++ 2M Karras, CFG scale: 7, Seed: 3150228071, Face restoration: CodeFormer, Size: 680x1024, Model hash: ef76aa2332, Model: 摄影-realsticVisionV51_v51VAE, Denoising strength: 0.5, Clip skip: 2, ControlNet 0: "Module: ip-adapter_face_id_plus, Model: ip-adapter-faceid-plusv2_sd15 [6e14fc1a], Weight: 1, Resize Mode: Crop and Resize, Low Vram: False, Processor Res: 512, Guidance Start: 0, Guidance End: 1, Pixel Perfect: True, Control Mode: Balanced, Hr Option: Both, Save Detected Map: True", Lora hashes: "ip-adapter-faceid-plusv2_sd15_lora: a95a0f4bdcb9", TI hashes: "badhandv4: 5e40d722fc3d, UnrealisticDream: a77451e7ea07", Version: v1.8.0

is doing good

usnig plusv2

1 girl,upper body,lora:ip-adapter-faceid-plusv2_sd15_lora:0.5
Negative prompt: (worst quality:2),(low quality:2),(normal quality:2),lowres,watermark,tail,fuzzy,blurry, watermark, badhandv4 UnrealisticDream
Steps: 20, Sampler: DPM++ 2M Karras, CFG scale: 7, Seed: 524885479, Face restoration: CodeFormer, Size: 680x1024, Model hash: ef76aa2332, Model: 摄影-realsticVisionV51_v51VAE, Denoising strength: 0.5, Clip skip: 2, ControlNet 0: "Module: ip-adapter_face_id_plus, Model: ip-adapter-faceid-plusv2_sd15 [6e14fc1a], Weight: 1, Resize Mode: Crop and Resize, Low Vram: False, Processor Res: 512, Guidance Start: 0, Guidance End: 1, Pixel Perfect: True, Control Mode: Balanced, Hr Option: Both, Save Detected Map: True", Lora hashes: "ip-adapter-faceid-plusv2_sd15_lora: a95a0f4bdcb9", TI hashes: "badhandv4: 5e40d722fc3d, UnrealisticDream: a77451e7ea07", Version: v1.8.0

skin color NG

BTW plusv2 t2i is ok

[Polukeev]

You are doing something wrong.

1. No need to use any neg or pos prompts if you are doing img2img. Keep lora in positive and nothing else.
2. You don't have to use any face restoration features, it is better to turn it off and forget about it.
3. Try to use DPM++ SDE 30-50 steps with 0.5-0.7 denoising strength.
4. Crop the face that you are using this way, but not smaller than 512x512
5. Try to use another checkpoint
   

[Polukeev]

After fresh reinstall of a1111 sd-web-ui 1.9.0 i have encountered few errors in log which i didn't have before

Does anyone know how to fix that? Before i didn't have anything like this, i still can make images, but i really want to figure out.
Happens only while using face-id models.
Additional system information:

UPD: it seems like cuda 12.4 isn't supported by onnxruntime-gpu by any versions yet. Please let me know if i am right or no about it.

UPD2: i have found the solution for all cuda 12.x users

1. open your venv "call activate.bat" in the "\stable-diffusion-webui\venv\Scripts" directory
2. remove all installed ORT "pip uninstall onnxruntime-gpu onnxruntime"
3. download specific ORT for cuda 12.x "pip install onnxruntime-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/"

This will solve any issues. Thanks.

[playcule]

Please help

webui version 1.9.4 / extensions are all updated

Model loaded in 14.8s (load weights from disk: 1.4s, create model: 0.8s, apply weights to model: 11.8s, move model to device: 0.2s, load textual inversion embeddings: 0.1s, calculate empty prompt: 0.2s).
2024-06-20 15:28:40,924 - ControlNet - INFO - Preview Resolution = 512
2024-06-20 15:29:37,146 - ControlNet - INFO - unit_separate = False, style_align = False
2024-06-20 15:29:37,349 - ControlNet - INFO - Loading model: ip-adapter-faceid-plusv2_sdxl (1) [187cb962]
2024-06-20 15:29:37,857 - ControlNet - INFO - Loaded state_dict from [E:\sd\webui\models\ControlNet\ip-adapter-faceid-plusv2_sdxl (1).pth]
2024-06-20 15:29:41,921 - ControlNet - INFO - ControlNet model ip-adapter-faceid-plusv2_sdxl (1) 187cb962 loaded.
2024-06-20 15:29:41,956 - ControlNet - INFO - Using preprocessor: ip-adapter_clip_sdxl_plus_vith
2024-06-20 15:29:41,957 - ControlNet - INFO - preprocessor resolution = 512
*** Error running process: E:\sd\webui\extensions\sd-webui-controlnet\scripts\controlnet.py
Traceback (most recent call last):
File "E:\sd\webui\modules\scripts.py", line 832, in process
script.process(p, *script_args)
File "E:\sd\webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1222, in process
self.controlnet_hack(p)
File "E:\sd\webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1207, in controlnet_hack
self.controlnet_main_entry(p)
File "E:\sd\webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 1162, in controlnet_main_entry
param.control_model.hook(
File "E:\sd\webui\venv\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "E:\sd\webui\extensions\sd-webui-controlnet\scripts\ipadapter\plugable_ipadapter.py", line 148, in hook
*[self.ipadapter.get_image_emb(o) for o in preprocessor_outputs]
File "E:\sd\webui\extensions\sd-webui-controlnet\scripts\ipadapter\plugable_ipadapter.py", line 148, in
*[self.ipadapter.get_image_emb(o) for o in preprocessor_outputs]
File "E:\sd\webui\extensions\sd-webui-controlnet\scripts\ipadapter\ipadapter_model.py", line 340, in get_image_emb
preprocessor_output.face_embed,
AttributeError: 'dict' object has no attribute 'face_embed'

I'm not sure why this error came out and every versions of faceID is not working.

[wackyli]

My controlnet version: 1.1.449
version: [v1.9.4] •  python: 3.10.6  •  torch: 2.1.2+cu121  •  xformers: N/A  •  gradio: 3.41.2

In the img2img , the 'ip-adapter-faceid-plusv2_sd15 [6e14fc1a]' not workin well.
Please refer to the following log information. It is very strange that the ip-adapter-faceid-plusv2_sd15 basically cannot call my GPU performance, and can only occasionally appear one or two times that it can be called. The parameter images used by both are consistent.

2024-06-25 10:44:24,286 - ControlNet - INFO - Loading model from cache: control_v11p_sd15_openpose [cab727d4]
2024-06-25 10:44:24,294 - ControlNet - INFO - Using preprocessor: dw_openpose_full
2024-06-25 10:44:24,295 - ControlNet - INFO - preprocessor resolution = 512
2024-06-25 10:44:24,901 - ControlNet - INFO - Loading model: ip-adapter-faceid_sd15 [0a1757e9]
2024-06-25 10:44:24,943 - ControlNet - INFO - Loaded state_dict from [D:\Codes\python\stable-diffusion-webui\extensions\sd-webui-controlnet\models\ip-adapter-faceid_sd15.bin]
2024-06-25 10:44:25,207 - ControlNet - INFO - ControlNet model ip-adapter-faceid_sd15 0a1757e9 loaded.
2024-06-25 10:44:25,210 - ControlNet - INFO - Using preprocessor: ip-adapter-auto
2024-06-25 10:44:25,210 - ControlNet - INFO - preprocessor resolution = 736
2024-06-25 10:44:25,210 - ControlNet - INFO - ip-adapter-auto => ip-adapter_face_id
2024-06-25 10:44:26,624 - ControlNet - INFO - ControlNet Hooked - Time = 2.3557376861572266
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 16/16 [00:06<00:00, 2.58it/s]
Total progress: 100%|███████████████████████████████████████████████████████████████████████████████████████████| 16/16 [00:06<00:00, 2.54it/s]
2024-06-25 10:45:09,837 - ControlNet - INFO - unit_separate = False, style_align = False████████████████████████| 16/16 [00:06<00:00, 2.92it/s]
2024-06-25 10:45:10,111 - ControlNet - INFO - Loading model: control_v11p_sd15_openpose [cab727d4]
2024-06-25 10:45:18,734 - ControlNet - INFO - Loaded state_dict from [D:\Codes\python\stable-diffusion-webui\extensions\sd-webui-controlnet\models\control_v11p_sd15_openpose.pth]
2024-06-25 10:45:18,734 - ControlNet - INFO - controlnet_default_config
2024-06-25 10:45:29,508 - ControlNet - INFO - ControlNet model control_v11p_sd15_openpose cab727d4 loaded.
2024-06-25 10:45:29,513 - ControlNet - INFO - Using preprocessor: dw_openpose_full
2024-06-25 10:45:29,513 - ControlNet - INFO - preprocessor resolution = 512
2024-06-25 10:45:29,694 - ControlNet - INFO - Loading model: ip-adapter-faceid-plusv2_sd15 [6e14fc1a]
2024-06-25 10:45:29,744 - ControlNet - INFO - Loaded state_dict from [D:\Codes\python\stable-diffusion-webui\extensions\sd-webui-controlnet\models\ip-adapter-faceid-plusv2_sd15.bin]
2024-06-25 10:45:30,188 - ControlNet - INFO - ControlNet model ip-adapter-faceid-plusv2_sd15 6e14fc1a loaded.
2024-06-25 10:45:30,190 - ControlNet - INFO - Using preprocessor: ip-adapter-auto
2024-06-25 10:45:30,190 - ControlNet - INFO - preprocessor resolution = 736
2024-06-25 10:45:30,191 - ControlNet - INFO - ip-adapter-auto => ip-adapter_face_id_plus
2024-06-25 10:45:55,137 - ControlNet - INFO - ControlNet Hooked - Time = 45.304054260253906
38%|████████████████████████████████████████▌ | 6/16 [01:39<02:45, 16.59s/it]
Total progress: 38%|██████████████████████████████████▌ | 6/16 [01:22<02:32, 15.24s/it]

[donleo78]

Hello.
I was trying faceid, but although the resulting images are similar, they are different enough to think that it is not the same person. I tried all the configurations and loras, but still, sometimes the resemblance is pretty close, but most of the time it isn't. I tried several models, both SD 1.5 and SDXL. I use a1111. Any suggestions? By the way, I'm not an expert in python and all this. Thank you.

[dan4ik94]

it depends on many things but you won't get 100% resemblance. You could combine multiple face swap techniques to get better results but it won't be the exact same face.

[X-rapido]

I have successfully installed insightface and onnxruntime, and the installation is smooth without any exceptions.
Why does this pop up when running the preprocessor?
Manual installation of insightface is still prompted each time it starts. My environment is macos m3, this issue has been bothering me for a few days, also tried to reinstall sdwebui, still can not be resolved.

[maojiuju]

InsightFace+CLIP-H(IPAdapter) 这个预处理器有知道在哪里下载的嘛，可以给个链接嘛

[dan4ik94]

https://huggingface.co/h94/IP-Adapter/tree/main/sdxl_models
https://huggingface.co/h94/IP-Adapter-FaceID/tree/main

[a3nima]

There has been a release of a new ip adapter face id model: "Kolors-IP-Adapter-FaceID-Plus". Can this already be used with controlnet or will it be supported soon ? when I've tried to load the model I got the error it's not being recognized.

[dan4ik94]

when kolors will get more community love I guess.

[a3nima]

Is there a way to get ip adapter face id v2 to work with pony-models? is anything from controlnet is working well with it (or is it only ip adapter, that doesnt)?

[Soulreaver90]

It works but terribly. Reactor is good for face swapping, but I haven't had good outputs using controlnet with Pony. I still go back to SD1.5 for that unfortunately. Plus alot of ram usage, so there is that.

[a3nima]

I had tried it and it did not really work, the faces became anime like instead of the reference images..
SD1.5 for what? sdxl face id plus v2 works pretty well :)

[Soulreaver90]

sdxl works, but the ram requirements tank my machine so I can't really use it lol. I can maybe get a few 1024x done before it says I go OOM. I can still generate incredibly realistic outputs with upscaling so its fine. But to the topic at hand, faceid + pony = blah results lol

[dan4ik94]

sdxl works, but the ram requirements tank my machine so I can't really use it lol. I can maybe get a few 1024x done before it says I go OOM. I can still generate incredibly realistic outputs with upscaling so its fine. But to the topic at hand, faceid + pony = blah results lol

what are your specs and what webui you use?

[rossfoxyai]

Is there plans for a Flux version of IPAdapter Faceid?
If you need help or support in any way? Let me know

[IndigoDosSantos]

You can find the authors of IP-Adapter-FaceID here. It's probably best to ask there right away.

[rossfoxyai]

Thanks bud. I will do