Scale Invariance
Harris corners are not invariant to scale!
How to fix this?
The corresponding sizes in the same images, look the same.
So you have to pick the circle where they look the same. And we have to do this independently for each image!
We need a function on a region which is scale invariant. That is, if everything is doubled precisely, then the function value will be the same. e.g. average intensity would satisfy this, though it won't work for feature point detection.
Given the function, we can compute the scale-invariant function over regions of different sizes. Pick the region size which provides the same value:
These cover the same region of the image, between two pictures - matching scales.
You want one stable sharp peak. That is, it has to have peaks (not a plateau), and it has to have only one peak.
You want one which responds sharply to contrast (sharp local intensity change).
At the right scale, you want that sharp contrast. If too close, no contrast, if too far, not enough contrast.
So finding points of high contrast (i.e. edges, corners).
The function is often just a kernel - see earlier lecture.
The Mexican Hat Operator - the Laplacian of Gaussian. Review the edge detection lectures.
Instead of the Laplacian of Gaussian, which is annoying, just do the Laplacian of Gaussians. You get a curve which looks really similar to the Laplacian of Gaussian.
Find the local extremum (either max or min) in scale as well as space. Same as with corner detection, it has neighbors in x and y, a keypoint/interest point has a neighbor in one scale up and one scale down:
Find the local extremum, then eliminate the edges. (OpenCV SIFT's edgeThreshold parameter is to cut off keypoints below a certain threshold if they're on an edge.)
Consider image at the first octave (first scale):
Blurrier and blurrier is the gaussian being applied more and more. Then you get your difference of gaussians:
- Take each pixel, compare to neighbors in space and neighbors in scale. You get the extrema.
- Then, you ensure a certain contrast > a certain amount (
contrastThresholdparameter in OpenCV SIFT) - Then you remove edges.
Results:
These are robust with respect to scale.
Use Harris corner to get the corners, do it to all scales. Then you get the Laplacians in the scale direction. You find points that are the maximum in space re: Harris, and are maximum in scale re: the Laplacian direction. This is called Harris-Laplacian. In their paper, it does way better than SIFT.