Improve "Align Image to Reference"-node further

[pifroggi]

Hey! When creating the alignment node, I noticed the alignment was not always 100% accurate, but sometimes shifted up to 1 pixel off to the right. I'm not sure what exactly determines the amount of shift, but it seems to have something to do with the image brightness. This seems to be consistent, so a model trained on such pairs learns this and should still work, but will in the end do a similar shift.

Because of this problem, to compensate, the node currently has a manual shift of 1 pixel in the opposite direction. This makes it more correct in many cases, but not always and this could be different with different datasets.

This new approach completely removes this shift by instead:

1. computing flow from Image to Reference as flow1
2. computing flow from Reference to itself as flow2
3. subtracting flow2 from flow1 to compensate for any inherent shift by the rife model
4. finally warping Image with the new flow

Unfortunately this will double processing time, but will get a much more accurate and consistent alignment. If multiple iterations are set, flow2 is reused, so the extra processing time overall gets smaller percentage wise.

I was thinking about adding a checkbox for that called "Improve Alignment" for example, but I'm not sure if it makes sense. It is not very clear what is happening and how it is different from the "Precision" dropdown. Maybe it should just always be active, since rife is pretty fast. What do you think and would you like this to be added?

[RunDevelopment]

Sorry for the delay @pifroggi!

I was thinking about adding a checkbox for that called "Improve Alignment" for example, but I'm not sure if it makes sense. It is not very clear what is happening and how it is different from the "Precision" dropdown. Maybe it should just always be active, since rife is pretty fast. What do you think and would you like this to be added?

I think such an option would be good. As for the UX of this option: I would probably make this a dropdown, because this switches between 2 methods of fixing minor misalignment. So maybe the dropdown could be called "Misalignment Fix" and have the option "???" and "Flow". I'm not sure what to call the old way of doing it, so I used a placeholder.

What do you think?

[pifroggi]

No worries, we could maybe call the two options "Fast" and "Precise" to better show when to use which.

In further testing with this it also showed improvements in general, which makes aligning easier, more precise and artifact free. Because of that I'm no longer sure if it would be good to add an additional option for that and instead was thinking about letting the user control the speed just trough the precision dropdown and iteration count. This would avoid having too many combinations of options, which makes it harder to find the right one. I also couldn't find a good reason to turn the new method off yet. Here are a few reasons why I think so:

1. One does not need to go as high with the precision to get good results now. For example 200% with 4 iterations is similar to 400% with 1 iteration, but still faster and needs a quarter of the VRAM.
2. Previously it was hard to fix large misalignments because one would need to increase the iterations, or chain multiple nodes with increasing precision, but this created artifacts that were doubled up with each iteration, creating large errors fast. Now one can go over 10 without issues and I was able to correct 300 pixels of misalignment.
3. I think when choosing higher precisions, one expects the alignment to get better. With 400% and 800% approaching subpixel level, I think it feels weird for it to be technically aligned to such a degree, but overall still shifted a little to the right, or if wrongly compensated, a little to the left.

Do you think that makes sense?

[pifroggi]

Hey! Sorry for the message, I just wanted to check what you think would be the best way to proceed with this.
I've also tried to fix the ruff errors, but let me know if this needs further changes.

[Splendide-Imaginarius]

Here are a few reasons why I think so:

@pifroggi Your arguments sound good to me. Also, given that RIFE is much faster than most of the models that chaiNNer works with (e.g. ESRGAN), and even much faster than OpenCV's homography alignment in a lot of cases, I don't see any reason to clutter the UX with an option that will probably induce users to think "let's choose the faster option" when in reality they are unlikely to notice the speed difference and picking the "faster" option will just waste their cognitive time.