How to retrieve information from 'CROP_DATA'?

[TheMoye]

Hello everyone,

In the Workflow I'm working on, I would like to manipulate the 'CROP_DATA' output from an 'Image Crop Face' node. More specifically, I would like to multiply them by two, but I can't find any node that allows me to do that.

Through 'Image Crop Face', I detect a character's head in an image generated by an SDXL checkpoint. The image of the head outputted by the 'Image Crop Face' node is approximately 400 x 400 pixels. I convert this image into latent and use it to generate a new head using an SD 1.5 checkpoint and a compatible LoRA.
I noticed that my graphics card supports generating a head by doubling the resolution of the latent using the 'HighRes-Fix Script' node, and the details were even better. 🤪
I would like to reintegrate this head at 800 x 800 pixels onto my base image (which also has its resolution doubled via another technique) by modifying the 'CROP_DATA' to account for the resolution doubling.

Is there an integrated way to do this?

I see in the file 'WAS_Node_Suite.py' (starting from line 3055) that the 'Image Crop Face' node returns the following:

RETURN_TYPES = ("IMAGE", "CROP_DATA")

# Return face image and coordinates
return (pil2tensor(face_img.convert('RGB')), (original_size, (left, top, right, bottom)))

I imagine it's possible for me to create a Custom Node that allows me to retrieve the size and coordinates?

Thank you for your insights! 👍

[WASasquatch]

Yeah it's just outputting a tuple with original size, and coordinates of the crop. So something like

((400, 400), (left, top, right, bottom)) would be the output, where coords are just pixel values.

You can manipulate this data how you want. Though, it may be better to just upscale your image to desired scale, then detect and crop face, diffuse, and paste back, avoiding the need to do all that, as the reason we use crop data is to paste back any sized result at correct size and location.

[TheMoye]

Hello WASasquatch and thank you for this response,

Though, it may be better to just upscale your image to desired scale, then detect and crop face, diffuse, and paste back, avoiding the need to do all that, as the reason we use crop data is to paste back any sized result at correct size and location.

I've already tried resizing the image before performing the face replacement, but I'm facing two issues:
_ With a larger image, I encounter a lot of false positives; certain details in the image are detected as faces.
_ The head generated by the checkpoint based on SD 1.5 is of significantly lower quality when it comes from a higher resolution latent. I imagine this stems from the fact that models based on SD 1.5 are trained on 512 x 512 images.

I attempted another solution: searching for the head from the image at the desired resolution, resizing the found head to provide a latent of a resolution usable by SD 1.5, and then upscaling (by the same reduction factor) using the 'HighRes-Fix Script' node.
The result is better, but still below the initial head generation quality.

Therefore, I'm left with no choice but to delve into Python 😱🥳

[TheMoye]

Here I am again!

So, I've created two nodes:

MoyeCropDataExpose, which exposes the components width, height, left, top, right, and bottom of 'CROP_DATA'.
MoyeCropDataRecompose, which conversely returns an element of type 'CROP_DATA' from the inputs width, height, left, top, right, and bottom.

I wanted to test my nodes by inserting them between the 'CROP_DATA' output of 'Image Crop Face' and the 'crop_data' input of 'Image Paste Face', but it raises an error in the console during the queue execution :

File "C:\ArtificialIntelligence\ComfUI\custom_nodes\was-node-suite-comfyui\WAS_Node_Suite.py", line 3264, in paste_image
crop_size, (top, left, right, bottom) = crop_data
TypeError: cannot unpack non-iterable int object

I checked by displaying in the console what I receive in my MoyeCropDataExpose node: ((396, 396), (1015, 432, 1411, 828)), and what I return in my MoyeCropDataRecompose node: ((396, 396), (1015, 432, 1411, 828)).

It seems exactly the same, right? What mistake could I have made?

The code for my nodes:

class MoyeCropDataExpose:
    
    
    @classmethod
    def INPUT_TYPES(s):
        return {"required":
                    {"CROP_DATA": ("CROP_DATA",),
                    }}
    
    RETURN_TYPES = ("INT", "INT", "INT", "INT", "INT", "INT",)
    RETURN_NAMES = ("width", "height", "left", "top", "right", "bottom",)
    
    CATEGORY = "WAS Suite/Utilities"
    
    FUNCTION = "expose_cropdata"
    
    def expose_cropdata(self, CROP_DATA):
        print(f"""Je reçois crop_data:
                {CROP_DATA}
            """)
        # Extraire les valeurs du tuple dans des variables distinctes
        original_size_tuple, crop_values = CROP_DATA
        
        # Extraire les valeurs de original_size_tuple dans des variables distinctes
        width, height = original_size_tuple
        
        # Extraire les valeurs de crop_values dans des variables distinctes
        left, top, right, bottom = crop_values
        
        return(width, height, left, top, right, bottom)


class MoyeCropDataRecompose:

    @classmethod
    def INPUT_TYPES(s):
        return {"required":
                    {"width": ("INT", {"default": 0}),
                    "height": ("INT", {"default": 0}),
                    "left": ("INT", {"default": 0}),
                    "top": ("INT", {"default": 0}),
                    "right": ("INT", {"default": 0}),
                    "bottom": ("INT", {"default": 0}),
                    }}
    
    RETURN_TYPES = ("CROP_DATA",)
    
    CATEGORY = "WAS Suite/Utilities"
    
    FUNCTION = "recompose_cropdata"
    
    def recompose_cropdata(self, width, height, left, top, right, bottom):
        print(f"""Les données reçu:
                width: {width}
                height: {height}
                left: {left}
                top: {top}
                right: {right}
                bottom: {bottom}
            """)
        crop_data = (width, height),(left, top, right, bottom)
        print(f"""Je retourne:
                {crop_data}
            """)
        return(crop_data)

Bonus question: Is it possible to display content in a node outside of an input or output? For example, displaying the width, height, left, top, right, and bottom information in the node during execution.

[WASasquatch]

Put your output inside another tupple, as ComfyUI also requires tuple output. Additionally coordinate are their own tuple.

return (((width, height), (left, top, right, bottom)),)

As for face detection, I would suggest using CLIPSeg and detect "face", then use the Mask Crop Region on the CLIPSeg mask to get coordinates of the face for Image Crop Location

[WASasquatch]

Here is a workflow which contains a face fix flow I use:

SDXL_Refiner_FaceFix_Workflkow.zip

[TheMoye]

Put your output inside another tupple, as ComfyUI also requires tuple output

Perfect, the double tuple works! 👍

As for face detection, I would suggest using CLIPSeg and detect "face", then use the Mask Crop Region on the CLIPSeg mask to get coordinates of the face for Image Crop Location

I've already tried using CLIPSeg (specifically following this workflow: https://www.reddit.com/r/StableDiffusion/comments/17cpa3w/i_noticed_some_coherent_expression_workflows_got/ ) but I didn't get the expected result:

• I'm aiming to replace the entire head, not just the face. Changing the term "face" to "head," for example, didn't make any difference. Perhaps by increasing the selection radius...
• The head I want to replace wears a crown, which is also present in the selection returned by CLIPSeg. Maybe it's possible to subtract this crown using a second CLIPSeg (I noticed that the term "crown" was detected correctly), but I haven't delved deeper into this (I assume there's a node for subtracting masks?)

Thank you for this Workflow; I'll take a look at it! 👀