A short example of how to utilize various open source library functions that can be used to identify and analyse strongly connected components for a given input image.
In the example I have given here, the image represents microarray sample spots printed to a slide using a Xaar inket printer. Using our robotic equipment, a camera is mounted to the printhead, so that images are taken of the spots, as they are being printed on-the-fly, usually in linear groups of 12 or 32 at a time:
As part of an investigation into how we may improve our quality control (QC) processes, one task (of many) will be to analyse such an input image, checking the spot images for things like misalignment in the x,y axes, spot shape (circularity), missing spots or tiny spots (satellites). It is anticipated that this would greatly speed up QC, which at present rely on manual validation.
In reality, the initial spot images will be subject to a degree of perspective distortion, given that the camera is mounted at an angle of approximately 30 degrees to the perpendicular of the slides. Some corrective matrix transformations would be needed to make the image rectangular, as opposed to quadrilateral. Another time.
For blob extraction, I have used the CvBlobsLib, a library to perform connected component labelling on binary images, available at the OpenCVblobslib github page. It is reasonably simple to use. I use the OpenCV libraries to open a colour image, convert it to a grayscale and then thresholding to convert it to a black and white (binary) image:
I then use the filter method in CBlobResult to extract all blobs in the image that conform to a certain size, count the number of ‘proper’ blobs discovered and display them in red:
C++ example code is here:
#include "stdafx.h"
#include "BlobResult.h"
#include <cv.h>
#include <cxcore.h>
#include <highgui.h>
const std::string filepath = "spots.bmp";
int _tmain(int argc, _TCHAR* argv[])
{
CBlobResult blobs;
CBlob *currentBlob;
// Load grayscale version of coloured input image
IplImage* original = cvLoadImage( filepath.c_str(),
CV_LOAD_IMAGE_GRAYSCALE );
// Make sure image file is available
assert ( original );
// Obtain binary (black and white) version of input image
IplImage* img_bw = cvCreateImage( cvGetSize( original ),
IPL_DEPTH_8U,
1 );
// Threshold to convert image into binary (B&W)
cvThreshold( original, // source image
img_bw, // destination image
100, // threhold val.
255, // max. val
CV_THRESH_BINARY ); // binary type );
// Find the white blobs in the B&W image
blobs = CBlobResult( img_bw, NULL, 0 );
// Exclude all white blobs smaller than the given value (80)
// The bigger the last parameter, the bigger the blobs need
// to be for inclusion
blobs.Filter( blobs,
B_EXCLUDE,
CBlobGetArea(),
B_LESS,
80 );
// Get the number of blobs discovered
int num_blobs = blobs.GetNumBlobs();
// Display the filtered blobs
IplImage* filtered = cvCreateImage( cvGetSize( img_bw ),
IPL_DEPTH_8U,
3 );
cvMerge( img_bw, img_bw, img_bw, NULL, filtered );
for ( int i = 0; i < num_blobs; i++ )
{
currentBlob = blobs.GetBlob( i );
currentBlob->FillBlob( filtered, CV_RGB(255,0,0));
}
// Display the input / output windows and images
cvNamedWindow( "input" );
cvNamedWindow( "output" );
cvShowImage("input", img_bw );
cvShowImage("output", filtered);
// Wait for user key press and then tidy up
cvWaitKey(0);
cvReleaseImage( &original );
cvReleaseImage( &filtered );
cvDestroyWindow( "input" );
cvDestroyWindow( "output" );
return 0;
}
Sample image “spots.jpg” available here.
To make this actually build and work you first need to ensure the following things are in place:
Install OpenCV
Make sure the OpenCV libraries have been properly installed in your Visual Studio environment. Make sure this works all right before proceeding further.
Download and build the cvBlobsLib library
Download and extract the cvBlobsLib library placing the extracted folder in a suitable location.
After downloading and extracting this cvBlobsLib Visual Studio project, you then build it, so that the necessary cvblobslib.lib file gets created, either within a Debug or Release folder. Any new project you are working on that uses the cvBlobsLib library will need the cvblobslib.lib file in order to correctly work.
On building this for the first time, you will probably encounter compiler errors like these:
c:\dump\cvblobslib_opencv_v8_3\blobcontour.h(6): fatal error C1083: Cannot open include file: 'cv.h': No such file or directory
1> BlobResult.cpp
1>c:\dump\cvblobslib_opencv_v8_3\blobresult.h(24): fatal error C1083: Cannot open include file: 'cxcore.h': No such file or directory
1> BlobOperators.cpp
Notice that this project comes with the original project settings which would need to be changed:
As with any other project that uses OpenCV, the cvblobslib VC++ project will also need to be set up so that OpenCV is correctly installed, and it knows where to find the library files, additional includes etc.
See the same OpenCV posting for details, which includes a section for Visual Studio 2010 considerations.
If you still have trouble building and getting the necessary cvblobslib.lib file, here’s one I made earlier.
Set the Visual Studio Project Properties
1. If you’re not using a Windows console application as in my example but are using an empty project instead, then omit the “#include stdafx.h” bit. “stdafx” is created with either a new Win32 Console Project or a Console Application (.NET) application. (Thanks hiperchelo)
2. Make sure the cvblobslib.lib file you created earlier is copied into the project folder of the application you are working on.
3. In Project -> Properties -> C/C++ -> Additional Include Directories, add the location of the folder where the cvBlobsLib library was installed. If this isn’t done, the compiler will complain it can’t find the “BlobResult.h” header file.
4. In Project -> Properties -> Linker -> Input, add the cvblobslib.lib entry, in addition to any existing OpenCV library files.
If you’re using OpenCV2.1 for example, you will also need to include the {cv210, cvaux210, highgui210, cxcore210, etc}.lib files. For OpenCV1.x versions, these will be {cv, cvaux, highgui, cxcore, etc}.lib files.
5. In Project -> Properties -> C/C++ -> Pre-Compiled Headers, select Not use precompiled headers.
6. In Project -> Properties -> C/C++ -> Code generation -> Run-time library, select Debug Multithreaded DLL (for debug version) or Multithreaded DLL (for release version).
7. In Project -> Properties -> General -> Use of MFC, select Use MFC in a shared DLL.
Build the Project
You may get a linker error during building your project that uses cvBlobsLib, similar to this garbage:
error LNK2019: unresolved external symbol "public: virtual __thiscall CBlobResult::~CBlobResult(void)" (??1CBlobResult@@UAE@XZ) referenced in function "public: struct _IplImage * __thiscall OpenCV_Handler::FindConnectedComponents(struct _IplImage *,class ConnComponentSet *,int const &,int const &)" (?FindConnectedComponents@OpenCV_Handler@@QAEPAU_IplImage@@PAU2@PAVConnComponentSet@@ABH2@Z)
If this is the case, then Visual Studio does not yet know about the cvblobslib.lib library file, against which it should link. In Project Properties -> Linker -> Input -> Additional Dependencies check that this file has been included:
During compilation you might get the following error message:
fatal error C1083: Cannot open include file: 'BlobResult.h': No such file or directory
If this is the case then make sure you have specified the necessary include file for using the cvBlobsLib libraries. In the Project Properties -> C/C++ -> General -> Additional Include Directories, ensure that the path to the includes has been added:
Other Issues: access violation errors when running under Release Mode
I noticed that using CBlobResult objects in Visual Studio’s Release Mode can cause access violation errors similar to the one shown:
This can be corrected by making sure that the proper version of the cvblobslib.lib file is being used – it needs to be the one built under Release mode, not Debug Mode:
Open the CvBlobsLib Visual Studio project and do a clean and rebuild under Release Mode. Grab hold of the newly built cvblobslib.lib file contained in the Release folder and copy it into your project that is using the cvBlobsLib library. In other words, replace the old (probably Debug Mode) cvblobslib.lib file you have been using with the new Release Mode one.
Update: 22 July 2011
The library does the job for real-world instances too. See this posting for tips on how to integrate the OpenCV/cvBlobsLib with the FlyCapture camera, by Point Gray Research. The input image used was a sample subset of microarray spots printed using a Xaar inkjet printer approximately 150 microns in diameter, printed to a 75.0 x 25.0 mm glass slide, with black background, camera approximately 10 degrees to the perpendicular:
Which is then converted into a grayscale image…
And converted into a black and white binary image using the thresholding…
And finally filtered, discarding blobs that are less than a given area, ignore the single-pixel blobs in white, this seems to be a display issue. The important thing is that the correct number and location of acceptable blobs were identified:
See here for related posts.
If you are having problems compiling the original cvBlobsLib project to obtain the necessary .lib file, a copy is available here.
Download example Visual Studio 2010 Projects – 32/64 bit
If you are experiencing difficulty getting this up and running (I know what a pain this can be to set up), then some example Visual Studio 2010 projects are available for download that contain all the necessary configuration settings, lib/dll files etc. Projects are available for both x86 (32-bit) and x64 (64-bit) processors:
Everything is self contained within the download, using relative path settings so that you don’t have to worry about getting everything set up correctly according to your machine – just extract the 7zip file to a location of your choice, open the Visual Studio 2010 solution as per your processor type (x86 or x64) and start experimenting with OpenCV / cvBlobsLib libraries straight away.
Because of the size of the complete OpenCV library, only the lib/dlls necessary for running these examples are included in the download, but feel free to expand on these.
As always, comments and suggestions are always welcome. Feel free to contact me if you have any questions.
[easyfileshop]
Other posts related to image detection
Tracking Coloured Objects in Video using OpenCV
Displaying AVI Video using OpenCV
Analyzing FlyCapture2 Images obtained from Flea2 Cameras
Integrating the FlyCapture SDK for use with OpenCV
OpenCV Detection of Dark Objects Against Light Backgrounds
Getting Started with OpenCV in Visual Studio
