As a newcomer to image processing, I have attempted to achieve the tracking of coloured objects in some sample video footage. In my case, my little one’s blue gloves moving in a snow-covered landscape (a bitterly cold Musselburgh allotments, December 2010).
Pre-requisites
To get this to build and work properly, please ensure the following pre-requisites have been installed to your satisfaction:
Set up OpenCV and cvBlobsLib in Visual Studio.
Install and set up ffmpeg.
In the example given here we essentially do the following four things:
1. Obtain footage from the camera or video file.
CvCapture* capture = cvCaptureFromFile( "MOV.MPG" );
2. Convert video formats
Convert frame from its default BGR (blue, green, red) format into HSV (Hue, Saturation, Value) format and extract the binary (black and white) image from it:
cvCvtColor( img, imgHSV, CV_BGR2HSV );
It is much easier to detect coloured areas using the HSV (hue, saturation, value) format rather than the RGB (red, green, blue) format. HSV has the advantage of only having to use a single number to detect the colour (“hue”), in spite of there being several shades of that colour, ranging from relatively from light shades to darker shades. The amount of colour and the brightness of the colour are handled by the “saturation” and “value” parameters respectively.
3. Do the thresholding
The original image is kept intact for later use, and we then threshold the HSV reference image (imgHSV) to obtain our black and white image (imgThresh):
cvInRangeS( imgHSV,
cvScalar( 104, 178, 70 ),
cvScalar( 130, 240, 124 ),
imgThresh );
The two cvScalar parameters represent the lower and upper bound of HSV (in that order) values that are blueish in colour. I got some suitable max/min values by grabbing some screenshots of the coloured objects I was interested in tracking and observing the kinds of hue/saturation/lum values that occur.
By selecting the region of interest (eg the blue gloves), and copying and pasting into another MS Paint document and expanding the area of interest, you can get a reasonable overview of the pixel colours that you are interested in tracking:
Dab on selected pixels and use the colour picker tool to display the hue/saturation/luminosity values mentioned earlier:
After writing down a few of these results I was able to get estimates of where these upper and lower values occur in the coloured regions of interest. This is far quicker than using trial and error.
It should be noted in MS Paint, the hue values range from 0-239, while OpenCV’s hue values range from 0-179. So any hue value you take from MS Paint for use in your code will need to be scaled using the factor 180 / 240. A MS Paint Hue value of 147 for example becomes 147 * (180/240) = 110 in OpenCV.
In MS Paint the Saturation and Luminosity are also in the range 0-239 while the OpenCV equivalents are in the range 0..255 (8 bit encoding). A MS Paint saturation of 210, for example, becomes (210/240) * 255 = 223 in OpenCV.
4. Do the blob detection
Use the cvBlobsLib APIs to detect the white blobs from the black background, excluding those of an inappropriate size.
blobs = CBlobResult( imgThresh, NULL, 0 );
blobs.Filter( blobs,
B_EXCLUDE,
CBlobGetArea(),
B_LESS,
10 );
5. Insert bounding rectangles
Attach bounding rectangles to the detected coloured areas in the input video:
cvRectangle( frame,
pt1,
pt2,
cvScalar(0, 0, 0, 0),
1,
8,
0 );
So that the bounding rectangles covering the regions of interest are visible in the orginal input video:
Full Code Listing
// ObjectTracking.cpp : Define the entry point for console app. #include "cv.h" #include "highgui.h" #include "BlobResult.h"
// Get thresholded image in HSV format
IplImage* GetThresholdedImageHSV( IplImage* img )
{
// Create an HSV format image from image passed
IplImage* imgHSV = cvCreateImage( cvGetSize( img ),
8,
3 );
cvCvtColor( img, imgHSV, CV_BGR2HSV );
// Create binary thresholded image acc. to max/min HSV ranges
// For detecting blue gloves in "MOV.MPG - HSV mode
IplImage* imgThresh = cvCreateImage( cvGetSize( img ),
8,
1 );
cvInRangeS( imgHSV,
cvScalar( 104, 178, 70 ),
cvScalar( 130, 240, 124 ),
imgThresh );
// Tidy up and return thresholded image
cvReleaseImage( &imgHSV );
return imgThresh;
}
int main()
{
CBlobResult blobs;
CBlob *currentBlob;
CvPoint pt1, pt2;
CvRect cvRect;
int key = 0;
IplImage* frame = 0;
// Initialize capturing live feed from video file or camera
CvCapture* capture = cvCaptureFromFile( "MOV.MPG" );
// Get the frames per second
int fps = ( int )cvGetCaptureProperty( capture,
CV_CAP_PROP_FPS );
// Can't get device? Complain and quit
if( !capture )
{
printf( "Could not initialize capturing...\n" );
return -1;
}
// Windows used to display input video with bounding rectangles
// and the thresholded video
cvNamedWindow( "video" );
cvNamedWindow( "thresh" );
// An infinite loop
while( key != 'x' )
{
// If we couldn't grab a frame... quit
if( !( frame = cvQueryFrame( capture ) ) )
break;
// Get object's thresholded image (blue = white, rest = black)
IplImage* imgThresh = GetThresholdedImageHSV( frame );
// Detect the white blobs from the black background
blobs = CBlobResult( imgThresh, NULL, 0 );
// Exclude white blobs smaller than the given value (10)
// The bigger the last parameter, the bigger the blobs need
// to be for inclusion
blobs.Filter( blobs,
B_EXCLUDE,
CBlobGetArea(),
B_LESS,
10 );
// Attach a bounding rectangle for each blob discovered
int num_blobs = blobs.GetNumBlobs();
for ( int i = 0; i < num_blobs; i++ )
{
currentBlob = blobs.GetBlob( i );
cvRect = currentBlob->GetBoundingBox();
pt1.x = cvRect.x;
pt1.y = cvRect.y;
pt2.x = cvRect.x + cvRect.width;
pt2.y = cvRect.y + cvRect.height;
// Attach bounding rect to blob in orginal video input
cvRectangle( frame,
pt1,
pt2,
cvScalar(0, 0, 0, 0),
1,
8,
0 );
}
// Add the black and white and original images
cvShowImage( "thresh", imgThresh );
cvShowImage( "video", frame );
// Optional - used to slow up the display of frames
key = cvWaitKey( 2000 / fps );
// Prevent memory leaks by releasing thresholded image
cvReleaseImage( &imgThresh );
}
// We're through with using camera.
cvReleaseCapture( &capture );
return 0;
}
Sample (compressed) MPG file “MOV.MPG” downloadable from here.
See this similar post for tracking colored objects using OpenCV calls only, no cvBlobsLib:
https://www.technical-recipes.com/2015/using-opencv-to-find-and-draw-contours-in-video/
Other posts related to image detection
Displaying AVI Video using OpenCV
Analyzing FlyCapture2 Images obtained from Flea2 Cameras
Integrating the FlyCapture SDK for use with OpenCV
OpenCV Detection of Dark Objects Against Light Backgrounds
Getting Started with OpenCV in Visual Studio
Object Detection Using the OpenCV / cvBlobsLib Libraries
