Frequency Separation model

[MagnificentPenguin]

Hello,

I'd like to propose the creation of a new ControlNet module aimed at preserving the low and medium frequencies of an image. This can be achieved through the implementation of either a bilateral filter or a median filter. In theory, these filters should enable the generation of images that closely resemble the original in terms of shapes, low-medium features, contours, and color. Could be perfect for upscaling, enhancing details and more.

The current widely adopted method for preserving these frequencies involves the use of the Tile/Blur module. However, I've found it to be imperfect due to the nature of the filters it employs. To illustrate, I've included a comparison between the source image, the blur_gaussian preprocessor, the tile_resample preprocessor, the proposed median filter, and the proposed bilateral filter.

Method	Description (to my experience…)
blur_gaussian	Blurs the image, resulting in the loss of sharp edges and important medium frequency details. Additionally, it tends to produce haloing artifacts in the generated images.
tile_resample	Resamples the image by downscaling, but fails to preserve source contours and medium frequencies effectively. The generated images often do not match the source image perfectly.
tile_colorfix	The workings of this preprocessor are unclear to me due to the absence of visible results, but it also struggles to preserve medium frequencies, often leading to blocky color artifacts in the generated images.
median filter	Eliminates high frequencies and fine details while preserving low-medium features. Can be enhanced by employing two passes of the filter for better low-frequency results.
bilateral filter	Similar to the median filter, removes high frequencies and fine details while retaining low-medium features, but with a much better accuracy. Dual passes can also be applied for improved low-frequency results.

I lack the knowledge and computing power to implement these models myself. I hope that someone within this community might have the time and resources to undertake this. It's worth noting that the proposed solution doesn't involve any new algorithms; rather, it relies on classic and simple filters, which could potentially make the implementation easier.

Thank you for your time, and I appreciate the continuous efforts in enhancing ControlNet!