docs/keypoints_detector_and_descriptor.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d5661be5-4677-4f48-b6ad-617de2ab40b7",
   "metadata": {},
   "source": [
    "## 1. Image Feature Detectors and Descriptor \n",
    "\n",
    "list of notable `cv::Feature2D` implementations in OpenCV with a brief description of each:\n",
    "\n",
    "1. **SIFT (cv::SIFT)**  \n",
    "   - Detects scale- and rotation-invariant keypoints.  \n",
    "   - Uses 128-dimensional descriptors for robust matching.  \n",
    "\n",
    "2. **SURF (cv::SURF)**  \n",
    "   - A faster alternative to SIFT with similar invariance.  \n",
    "   - Employs 64- or 128-dimensional descriptors.  \n",
    "\n",
    "3. **ORB (cv::ORB)**  \n",
    "   - Fast and lightweight; uses binary descriptors.  \n",
    "   - Open-source, scale- and rotation-invariant.  \n",
    "\n",
    "4. **BRISK (cv::BRISK)**  \n",
    "   - Detects and describes keypoints with binary descriptors.  \n",
    "   - Designed for high-speed performance.  \n",
    "\n",
    "5. **AKAZE (cv::AKAZE)**  \n",
    "   - Efficient for scale-invariant feature detection.  \n",
    "   - Uses nonlinear scale spaces and binary descriptors.  \n",
    "\n",
    "6. **KAZE (cv::KAZE)**  \n",
    "   - Similar to AKAZE but with floating-point descriptors.  \n",
    "   - More precise but computationally expensive.  \n",
    "\n",
    "7. **FAST (cv::FastFeatureDetector)**  \n",
    "   - Extremely fast keypoint detector.  \n",
    "   - Not scale- or rotation-invariant.  \n",
    "\n",
    "8. **GFTT (cv::GFTTDetector)**  \n",
    "   - \"Good Features to Track\" detector based on corner detection.  \n",
    "   - Often used in optical flow and tracking.  \n",
    "\n",
    "9. **MSER (cv::MSERDetector)**  \n",
    "   - Detects stable regions in images (e.g., blobs).  \n",
    "   - Commonly used for text detection.  \n",
    "\n",
    "10. **HarrisLaplace (cv::HarrisLaplaceFeatureDetector)**  \n",
    "    - A combination of Harris corner detection and Laplacian for scale-invariance.  \n",
    "    - Detects keypoints for multiscale analysis.  \n",
    "\n",
    "11. **SimpleBlobDetector (cv::SimpleBlobDetector)**  \n",
    "    - Detects circular, blob-like regions.  \n",
    "    - Ideal for detecting uniform regions such as coins or bubbles.  \n",
    "[List of all available 2D image feature detectors and descriptor](https://docs.opencv.org/3.4/d0/d13/classcv_1_1Feature2D.html#a40182e88bf6aa2c74311c9927ba056dc)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2056e3ab-b8c5-43eb-8f45-5defacbbbb23",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## 2. cv::KeyPoint\n",
    "The **cv::KeyPoint** contains information about the detected keypoints.\n",
    "- `pt`: Coordinates of the keypoint in the image (`(x, y)`).\n",
    "- `size`: Diameter of the neighborhood considered around the keypoint.\n",
    "- `angle`: Orientation of the keypoint (useful for rotation-invariant features). If not computed, it will be `-1`.\n",
    "- `response`: Strength of the detected keypoint (a measure of how strong the keypoint feature is).\n",
    "- `octave`: Image pyramid level where the keypoint was detected.\n",
    "- `class_id`: A user-defined ID (optional).\n",
    "\n",
    "convert **vector of keypoints** to **vector of points** meaning Array of (x,y) coordinates of each keypoint\n",
    "\n",
    "```python\n",
    "img_pts = detector.detect(img, None)\n",
    "\n",
    "point = img_pts[0]\n",
    "\n",
    "x = point.pt[0]\n",
    "y = point.pt[1]\n",
    "(x, y) = point.pt\n",
    "```\n",
    "\n",
    "convert vector of keypoints to vector of points -> Array of (x,y) coordinates of each keypoint\n",
    "\n",
    "python\n",
    "\n",
    "```python\n",
    "pts = cv2.KeyPoint_convert(img_pts)\n",
    "print(pts)\n",
    "pts = np.float64([key_point.pt for key_point in img_pts]).reshape(-1, 1, 2)\n",
    "```\n",
    "\n",
    "\n",
    "c++: \n",
    "```cpp\n",
    "std::vector<cv::Point2f> points;\n",
    "cv::KeyPoint::convert(k_pts, points);\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7dcb1bf-10ab-4565-b481-20d72e15a5a2",
   "metadata": {},
   "source": [
    "#### Example of Feature Detectors and Detecting keypoints \n",
    "\n",
    "C++\n",
    "\n",
    "```cpp\n",
    "cv::Ptr<cv::SIFT> siftr = cv::SIFT::create();\n",
    "std::vector<cv::KeyPoint> k_pts;\n",
    "sift->detect(img, k_pts);\n",
    "```\n",
    "\n",
    "Python\n",
    "\n",
    "```python\n",
    "sift = cv2.SIFT_create()\n",
    "img_pts, img_descriptor = sift.compute(img, img_pts)\n",
    "\n",
    "orb = cv2.ORB_create()\n",
    "img_pts, img_descriptor = orb.compute(img, img_pts)\n",
    "```\n",
    "\n",
    "\n",
    "## 3. Type of and name of detector:\n",
    "\n",
    "sift is `CV_32F` and orb is `CV_8U`\n",
    "\n",
    "```cpp\n",
    "// Check descriptor type\n",
    "int descType = sift->descriptorType(); // Returns CV_32F, <=>5 \n",
    "std::cout << \"SIFT Descriptor Type: \" << descType << \" (CV_32F = \" << CV_32F << \")\" << std::endl;\n",
    "\n",
    "\n",
    "// Check descriptor type\n",
    "descType = orb->descriptorType(); // Returns CV_8U\n",
    "std::cout << \"ORB Descriptor Type: \" << descType << \" (CV_8U = \" << CV_8U << \")\" << std::endl;\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d4dd073-2016-4333-800c-2d5b8161136a",
   "metadata": {},
   "source": [
    "The `descriptorType()` method in any `cv::Feature2D` implementations, returns the data type of the descriptors generated by the `compute()` method. This helps you understand the format of the `descriptors` matrix produced.\n",
    "\n",
    "\n",
    "It returns the data type of the descriptors that `sift->compute()` will produce. \n",
    "\n",
    "### Descriptors in SIFT\n",
    "For **SIFT**, the descriptors are:\n",
    "- **Type**: `CV_32F` (32-bit floating-point values).  \n",
    "- **Shape**: A matrix where:\n",
    "  - **Rows**: Correspond to the number of keypoints detected.\n",
    "  - **Columns**: Correspond to the dimensionality of the SIFT descriptor, typically 128.\n",
    "\n",
    "\n",
    "### Descriptors in ORB\n",
    "The ORB algorithm produces **binary descriptors**, meaning that each descriptor is a compact, binary vector. \n",
    "\n",
    "- **Type**: `CV_8U` (8-bit unsigned integers).  \n",
    "- **Format**: Each descriptor is represented as a binary vector stored in bytes (each bit encodes feature information).\n",
    "- **Shape**: A matrix where:\n",
    "  - **Rows**: Correspond to the number of keypoints.\n",
    "  - **Columns**: Correspond to the length of the binary descriptor in bits divided by 8 (e.g., 32 bytes for a 256-bit descriptor).\n",
    "\n",
    "---\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85e792d9-d1b5-4af4-af1d-6cb0cd8fdfbe",
   "metadata": {},
   "source": [
    "## 3. keypoints descriptors\n",
    "\n",
    "In the function `compute (InputArray image, std::vector< KeyPoint > &keypoints, OutputArray descriptors)`\n",
    "\n",
    "**descriptors**: An output array to store the computed feature descriptors.\n",
    "Each row corresponds to a keypoint, and the row contains the descriptor values for that keypoint.\n",
    "\n",
    "The format depends on the algorithm:\n",
    "\n",
    "- SIFT: 128 columns (each keypoint has a 128-dimensional float descriptor).\n",
    "- ORB: 32 columns (each keypoint has a 256-bit binary descriptor).\n",
    "- SURF: Typically 64 or 128 columns (float descriptors).\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8ba7721-32ad-4396-a380-07d1bad8a6fe",
   "metadata": {},
   "source": [
    "#### Draw detected keypoints:\n",
    "\n",
    "c++\n",
    "\n",
    "```cpp\n",
    "cv::drawKeypoints(img, k_pts, imgWithKeypoints, cv::Scalar::all(-1), cv::DrawMatchesFlags::DRAW_RICH_KEYPOINTS);\n",
    "```\n",
    "\n",
    "\n",
    "Here are the available flags you can use in `cv::drawKeypoints()`:\n",
    "\n",
    "1. **`cv::DrawMatchesFlags::DEFAULT`**: This is the default flag that draws the keypoints with circles around them.\n",
    "2. **`cv::DrawMatchesFlags::DRAW_OVER_OUTIMG`**: This flag draws the keypoints on top of the input image (output image will be modified).\n",
    "3. **`cv::DrawMatchesFlags::DRAW_KEYPOINTS`**: Draws keypoints as simple circles (default behavior).\n",
    "4. **`cv::DrawMatchesFlags::DRAW_RICH_KEYPOINTS`**: Draws keypoints with size, orientation, and more detailed visualizations.\n",
    "\n",
    "\n",
    "python\n",
    "\n",
    "```python\n",
    "prevImg_marked = cv.drawKeypoints(\n",
    "    img, img_pts, None, color=(0, 255, 0), flags=0)\n",
    "\n",
    "plt.imshow(prevImg_marked)\n",
    "plt.show()\n",
    "```\n",
    "\n",
    "1. `cv.DRAW_MATCHES_FLAGS_DEFAULT`\n",
    "2. `cv.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS`\n",
    "3. `cv.DRAW_MATCHES_FLAGS_DRAW_OVER_OUTIMG`\n",
    "4. `cv.DRAW_MATCHES_FLAGS_NOT_DRAW_SINGLE_POINTS`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0dd34e1c-07c8-4f33-baf6-cbfe6d2822ea",
   "metadata": {},
   "source": [
    "#### SIFT keypoints\n",
    "\n",
    "<img src=\"images/SIFTKeypoints.png\" />\n",
    "\n",
    "\n",
    "#### ORB keypoints\n",
    "\n",
    "<img src=\"images/ORBKeypoints.png\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3e63583-09bd-432f-aadf-375ffd24db6c",
   "metadata": {},
   "source": [
    "## goodFeaturesToTrack\n",
    "\n",
    "The goodFeaturesToTrack function in OpenCV is a method to detect the most prominent corners or features in an image. This function is based on the Shi-Tomasi corner detection method, which is a modification of the Harris corner detection. While Harris scores corners based on a combination of the eigenvalues of the corner's covariance matrix, the Shi-Tomasi method simply considers the minimum of these eigenvalues.\n",
    "\n",
    "\n",
    "params for corner detection: \n",
    "If `useHarrisDetector` set to True, Harris corner detection is used instead of `Shi-Tomasi`.\n",
    "```python\n",
    "feature_params = dict(maxCorners=100,\n",
    "                      qualityLevel=0.3,\n",
    "                      minDistance=7,\n",
    "                      blockSize=7,useHarrisDetector=False)\n",
    "\n",
    "keypoints = cv.goodFeaturesToTrack(img_gray, mask=None,**feature_params)\n",
    "\n",
    "```"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "437b668d-fae6-4fc0-ab05-3ab08a674d36",
   "metadata": {},
   "source": [
    "<img src=\"images/goodFeaturesToTrack.png\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1600b6e-0b29-4336-971b-577064b7bc8e",
   "metadata": {},
   "source": [
    "[c++ code](../src/feature_detection_description.cpp)\n",
    "\n",
    "\n",
    "[python code](../scripts/keypoint_feature_detection_description.py)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}