[New Feature] Guidelines for using inpaint in A1111

[lllyasviel]

Now the ControlNet Inpaint can directly use the A1111 inpaint path to support perfect seamless inpaint experience. It supports arbitary base model without merging and works perfectly with LoRAs and every other addons.

This inpaint implementation is technically correct and will NOT make unwanted modifications to unmasked areas.

To use it, update your ControlNet to latest version, restart completely including your terminal, and go to A1111's img2img inpaint, open ControlNet, set preprocessor as "inpaint_global_harmonious" and use model "control_v11p_sd15_inpaint", enable it.

You do not need to add image to ControlNet.

This is my setting

Then you can just go to A1111's inpaint and inpaint images as normal. Your images will be improved by ControlNet automatically.

We support both inpaint "Whole picture" and "Only masked".

For example this is my setting

This is A1111 inpainting with seed 12345 (whole picture) with ControlNet turned OFF:

This is A1111 inpainting with seed 12345 (whole picture) with ControlNet turned ON:

[lllyasviel]

ControlNet OFF:

ControlNet ON:

[ghpkishore]

@lllyasviel any ideas on how to translate this inpainting to diffusers library. MultiControlnet with inpainting in diffusers doesn't exist as of now. Given that you have been able to implement it in A1111 extension, any suggestions or leads on how to do it for diffusers would proves really helpful.

[Rkkss]

can ip2p model work with inpaint model somehow?

[duffercn]

I am trying to use controlnet to inpaint a product photo, by which I mean I am inpainting a background for a product.
I tried this preprocessor/model pair but doesn't seems working.
Will the model be the right one to fulfill my needs? Thank you.

[PaperOrb]

Is there a specific reason you use euler A in the examples, or is that just a coincidence?

edit: it looks like this isn't present in the readme. And another question: can you add a bit more detail about how this differs from regular inpainting models without going into too much technical detail, or maybe explain the technical stuff and then add a short dumbed down answer at the end?

[ride5k]

i'm curious about the difference between inpaint_only and inpaint_global_harmonious

[psykokwak-com]

Hello all,
Do you plan to offer the inpaint model for SDXL ?
Thanks.

[arafatx]

Can I use other inpainting model like Dreamshaper for this extension. I'm new to this

[ywaby]

any workflow for comfyui?

[maguro97]

In order to avoid unwanted modifications to unmasked areas, which loss do you use?