Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What the "Controlnet tile" model really does and How was it trained? #129

Open
moonryul opened this issue Oct 29, 2023 · 2 comments
Open

Comments

@moonryul
Copy link

moonryul commented Oct 29, 2023

Hi,

I read that "controlnet tile" was trained by "image tiling" or on "tiled image".
It suggests that the input condition to this model is a tiled image; But....
When I read gradio_tile.py, the "control" input is just a upsampled/resized image obtained from an input image.
It has NOTHING to do with image tiling.

I know this model is used in conjunction with "Ultimate SD supscale" script in Automatic1111. But
when I looked at the script, it splits the input image to tiles and invokes the controlNet for each tile, and combines the resulting
tiles. So, tiling has to do with the script and nothing to do with the "controlenet-tile", it seems.

So, I wonder:

  1. Q1. In what sense, is it a "controlnet-tile" model?
  2. Q2. What kind of dataset is used to train this tile model?
    In the case of "inpaint" model, the dataset consists of pairs of (image, [mask, masked_image]).
    In the case of "upscaling" model, the dataset are pairs of (image, low_resolution version).

But in the case of "controlnet-tile" model, the dataset is not clear.

Sincerely
Moon

@geroldmeisinger
Copy link

duplicate of #125 which was 5 lines down

@moonryul
Copy link
Author

duplicate of #125 which was 5 lines down

Thanks for the answer. Do you mean the answers to my questions
can be found in #125?

But I could not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants