Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Port to C++] Blob Detection #85

Merged
merged 14 commits into from
Mar 28, 2023
Empty file.
Empty file.
7 changes: 7 additions & 0 deletions 4_cv_basics/7_blob_detection/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
blob_detection: main.cpp
@echo "Building.."
@g++ main.cpp src/blob_detection.cpp -o blob -lopencv_core -lopencv_imgcodecs -lopencv_highgui -lopencv_imgproc -lopencv_videoio -lsqlite3 -lgdal


clean:
rm blob_detection
152 changes: 152 additions & 0 deletions 4_cv_basics/7_blob_detection/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
# Blob Detection

BLOB stands for Binary Large Object

Informally a blob is a region of an image in which some properties like intensity or color are approximately constant.

## Table of Contents

- [Understanding the code](#understanding-the-code)
- [Detecting the blob](#detecting-the-blob)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be more informative if the contents in the Readme can be divided into the different topic sections similar to this


## Understanding the code

---

* The code below captures a video using the webcam and stores the video by the name 'frame'

```cpp
int main()
{
VideoCapture video(0);
int h = 0, s = 0, v = 0;
while (video.isOpened())
{
Mat frame;
video.read(frame);
imshow("Image", frame);

```

---

* Pressing 'q' would select the frame at that instance.
* `selectROI()` allows you to select that part of the captured frame in which the blob exists (and has to be detected later).
* Next, convert the frame to hsv.
* obj_img is hsv image of the size of the selected ROI.
<br>Why converting to HSV?<br>
Since we are using the web cam the intensity and illumination of consecutive frame does not remain same.
Hence to find the color in range instead of particular color. HSV format is useful as H value denotes specific color and S, V can be used for illumination and intensity.

* Next, we want the median values of each of hue, saturation and value, of the <b>selected ROI</b>.

```cpp
while (1)
{
if (waitKey(10) == 'q')
{
Rect bbox = selectROI("Image", frame, false, false);
Mat hsv;
cvtColor(frame, hsv, COLOR_BGR2HSV);
Mat obj_img = hsv(Rect(bbox.x, bbox.y, bbox.width, bbox.height));
tuple<double, double, double> medians = getMedianPixelValues(obj_img);
h = get<0>(medians);
s = get<1>(medians);
v = get<2>(medians);
break;
}
}
```

---

* We keep capturing the video via webcam until the program is running.
* The captured video is coverted to hsv
* Using the median values of h,s,v obtained in the previous section, we calculate the upper and lower bounds of each (for masking done later).

```cpp
while (video.isOpened())
{
Mat frame;
video.read(frame);

// Converting to hsv
Mat hsv;
cvtColor(frame, hsv, COLOR_BGR2HSV);

// Getting lower and upper bounds for h,s,v values
Scalar lower(h - 5, max(0, s - 50), max(0, v - 50));
Scalar upper(h + 5, min(s + 50, 255), min(v + 50, 255));

```
* We make a mask using `inRange()` function with the lower and upper scalars obatined.
<br>
What is a mask?<br>
A mask is a binary image consisting of zero and non-zero values. If a mask is applied to another image of the same size, all pixels which are zero in the mask are set to zero in the output image. All others remain unchanged.
* We blur the mask to remove noises.
* Placing blurred mask over frame to find `bitwise_and()`.

```cpp
// Masking
Mat masked;
inRange(hsv, lower, upper, masked);

// Median Blur
Mat blur;
medianBlur(masked, blur, 5);

// Performing bitwise_and using the 'blur' mask
Mat blob_mask;
bitwise_and(frame, frame, blob_mask, blur);

imshow("blob_mask", blob_mask);
```

---

## Detecting the blob
* Find the contour from the generated mask using `findContours()`

- What is a contour?<br>
Contours can be explained simply as a curve joining all the continuous points (along the boundary), having same color or intensity. The contours are a useful tool for shape analysis and object detection and recognition

* Find the contour having the maximum area using `contourArea()`
* Draw the largest contour on the frame using `drawContours()`

```cpp
// Find contours
vector<vector<Point>> contours;
vector<Vec4i> hierarchy;
findContours(blur, contours, hierarchy, RETR_TREE, CHAIN_APPROX_SIMPLE);

// Find largest contour
int idx = 0;
double current_max = 0;
int counter = 0;

for (const auto &n : contours)
{
double area = contourArea(n);
if (area > current_max)
{
current_max = area;
idx = counter;
}
counter++;
}

// Draw the largest contour detected
drawContours(frame, contours, idx, Scalar(0, 255, 255), 2);
imshow("Output", frame);

if (waitKey(10) == 'x')
{
destroyAllWindows();
video.release();
break;
}
}

return 0;
}
```
13 changes: 13 additions & 0 deletions 4_cv_basics/7_blob_detection/include/blob_detection.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#ifndef BLOB_DETECTION_HPP
#define BLOB_DETECTION_HPP

#include <opencv2/opencv.hpp>

using namespace std;
using namespace cv;

double median(vector<double> vec);

tuple<double, double, double> getMedianPixelValues(Mat img);

#endif
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a new line

94 changes: 94 additions & 0 deletions 4_cv_basics/7_blob_detection/main.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
#include <bits/stdc++.h>
#include <opencv2/opencv.hpp>
#include "include/blob_detection.hpp"
using namespace std;
using namespace cv;

int main()
{
VideoCapture video(0); // Read input video
int h = 0, s = 0, v = 0;
while (video.isOpened())
{
Mat frame;
video.read(frame); // Read one frame from the video
imshow("Image", frame);

if (waitKey(10) == 'q') //Wait for "q" key to be pressed
{
Rect bbox = selectROI("Image", frame, false, false); // Select a region of interest on the image
Mat hsv;
cvtColor(frame, hsv, COLOR_BGR2HSV); // Convert BGR color model to HSV
Mat obj_img = hsv(Rect(bbox.x, bbox.y, bbox.width, bbox.height));
tuple<double, double, double> medians = getMedianPixelValues(obj_img); // Find the median of HSV in the selected region.
h = get<0>(medians);
s = get<1>(medians);
v = get<2>(medians);
break;
}
}

while (video.isOpened())
{
Mat frame;
video.read(frame);

// Converting to hsv
Mat hsv;
cvtColor(frame, hsv, COLOR_BGR2HSV);


// Getting lower and upper bounds for h,s,v values
Scalar lower(h - 5, max(0, s - 50), max(0, v - 50));
Scalar upper(h + 5, min(s + 50, 255), min(v + 50, 255));

// Masking
Mat masked;
inRange(hsv, lower, upper, masked);

// Median Blur
Mat blur;
medianBlur(masked, blur, 5);

// Performing bitwise_and using the 'blur' mask
Mat blob_mask;
bitwise_and(frame, frame, blob_mask, blur);

imshow("blob_mask", blob_mask);

// Find contours
vector<vector<Point>> contours;
vector<Vec4i> hierarchy;
findContours(blur, contours, hierarchy, RETR_TREE, CHAIN_APPROX_SIMPLE);


// Find largest contour
int idx = 0;
double current_max = 0;
int counter = 0;

for (const auto &n : contours)
{
double area = contourArea(n);
if (area > current_max)
{
current_max = area;
idx = counter;
}
counter++;
}

// Draw the largest contour detected
drawContours(frame, contours, idx, Scalar(0, 255, 255), 2);
imshow("Output", frame);

if (waitKey(10) == 'x')
{
destroyAllWindows();
video.release();
break;
}
}

return 0;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a newline

Empty file.
93 changes: 93 additions & 0 deletions 4_cv_basics/7_blob_detection/src/blob_detection.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
#include <opencv2/opencv.hpp>
#include <iostream>

/*
Specifying Namespaces
Namespace is a declarative region that provides a scope to the identifiers (the names of types, functions, variables, etc) inside it.
*/

using namespace std;
using namespace cv;

/*
Function to find median of a set of pixels

Purpose:
-----
Takes a vector of pixels of one channel and outputs the median of the values.

Input Args:
-----
vector<double> vec: A dynamic array with data type double.

Returns:
-----
Median value
*/

double median(vector<double> vec)
{
int size = vec.size();
if (size == 0)
{
return 0; // Handle empty vector case
}
sort(vec.begin(), vec.end());
if (size % 2 == 0)
{
// Even number of elements, take average of middle 2
return (vec[size / 2 - 1] + vec[size / 2]) / 2.0;
}
else
{
// Odd number of elements, take middle element
return vec[size / 2];
}
}

/*
Function to find median of each channel

Purpose:
-----
Finds the median value of each channel in a given image

Input Args:
-----
Mat img: An n-dimensional numerical array that represents an image.

Returns:
-----
A tuple containing median values for each channel.
*/

tuple<double, double, double> getMedianPixelValues(Mat img)
{
// Initialize vectors to store the pixel values of each color channel
vector<double> blue_pixels;
vector<double> green_pixels;
vector<double> red_pixels;

// Iterate over the rows and columns of the image
for (int i = 0; i < img.rows; i++)
{
for (int j = 0; j < img.cols; j++)
{
// Access the pixel value at (i, j)
Vec3b pixel = img.at<Vec3b>(i, j);

// Add the pixel values to the corresponding vectors
blue_pixels.push_back(pixel[0]);
green_pixels.push_back(pixel[1]);
red_pixels.push_back(pixel[2]);
}
}

// Calculate the median of each color channel
double blue_median = median(blue_pixels);
double green_median = median(green_pixels);
double red_median = median(red_pixels);

// Return the median values as a tuple
return make_tuple(blue_median, green_median, red_median);
}