Face Detection using OpenCV

Note: This tutorial uses the OpenCV 1 interface and (as far as I can tell) is not compatible with the version of `haarcascade_frontalface_alt.xml` included in the OpenCV 2 code source. See http://doc.opencv.org/doc/tutorials/objdetect/cascade_classifier/cascade_classifier.html for the OpenCV 2 version of the tutorial which is compatible with the current XML files.

How to compile and run the facedetect.c is one of the frequently asked question in the OpenCV Yahoo! Groups. I'll try to explain it to my best. Feel free to edit it if you have some more details..

Haar Like Features: What is that?

A recognition process can be much more efficient if it is based on the detection of features that encode some information about the class to be detected. This is the case of Haar-like features that encode the existence of oriented contrasts between regions in the image. A set of these features can be used to encode the contrasts exhibited by a human face and their spacial relationships. Haar-like features are so called because they are computed similar to the coefficients in Haar wavelet transforms.

The object detector of OpenCV has been initially proposed by Paul Viola and improved by Rainer Lienhart. First, a classifier (namely a cascade of boosted classifiers working with haar-like features) is trained with a few hundreds of sample views of a particular object (i.e., a face or a car), called positive examples, that are scaled to the same size (say, 20x20), and negative examples - arbitrary images of the same size.

After a classifier is trained, it can be applied to a region of interest (of the same size as used during the training) in an input image. The classifier outputs a "1" if the region is likely to show the object (i.e., face/car), and "0" otherwise. To search for the object in the whole image one can move the search window across the image and check every location using the classifier. The classifier is designed so that it can be easily "resized" in order to be able to find the objects of interest at different sizes, which is more efficient than resizing the image itself. So, to find an object of an unknown size in the image the scan procedure should be done several times at different scales.

The word "cascade" in the classifier name means that the resultant classifier consists of several simpler classifiers (stages) that are applied subsequently to a region of interest until at some stage the candidate is rejected or all the stages are passed. The word "boosted" means that the classifiers at every stage of the cascade are complex themselves and they are built out of basic classifiers using one of four different boosting techniques (weighted voting). Currently Discrete Adaboost, Real Adaboost, Gentle Adaboost and Logitboost are supported. The basic classifiers are decision-tree classifiers with at least 2 leaves. Haar-like features are the input to the basic classifers.The feature used in a particular classifier is specified by its shape , position within the region of interest and the scale (this scale is not the same as the scale used at the detection stage, though these two scales are multiplied).

Ok. Enough of the theory part. Now to the coding part: (This code might have a memory leak. Scroll down to the bottom of the page for information on how to fix it.)

Here is my commented facedetect.c file:

  1// OpenCV Sample Application: facedetect.c
  2
  3// Include header files
  4#include "cv.h" 
  5#include "highgui.h" 
  6
  7#include <stdio.h>
  8#include <stdlib.h>
  9#include <string.h>
 10#include <assert.h>
 11#include <math.h>
 12#include <float.h>
 13#include <limits.h>
 14#include <time.h>
 15#include <ctype.h>
 16
 17// Create memory for calculations
 18static CvMemStorage* storage = 0;
 19
 20// Create a new Haar classifier
 21static CvHaarClassifierCascade* cascade = 0;
 22
 23// Function prototype for detecting and drawing an object from an image
 24void detect_and_draw( IplImage* image );
 25
 26// Create a string that contains the cascade name
 27const char* cascade_name =
 28    "haarcascade_frontalface_alt.xml";
 29/*    "haarcascade_profileface.xml";*/
 30
 31// Main function, defines the entry point for the program.
 32int main( int argc, char** argv )
 33{
 34
 35    // Structure for getting video from camera or avi
 36    CvCapture* capture = 0;
 37
 38    // Images to capture the frame from video or camera or from file
 39    [[IplImage]] *frame, *frame_copy = 0;
 40
 41    // Used for calculations
 42    int optlen = strlen("--cascade=");
 43
 44    // Input file name for avi or image file.
 45    const char* input_name;
 46
 47    // Check for the correct usage of the command line
 48    if( argc > 1 && strncmp( argv[1], "--cascade=", optlen ) == 0 )
 49    {
 50        cascade_name = argv[1] + optlen;
 51        input_name = argc > 2 ? argv[2] : 0;
 52    }
 53    else
 54    {
 55        fprintf( stderr,
 56        "Usage: facedetect --cascade=\"<cascade_path>\" [filename|camera_index]\n" );
 57        return -1;
 58        /*input_name = argc > 1 ? argv[1] : 0;*/
 59    }
 60
 61    // Load the [[HaarClassifierCascade]]
 62    cascade = (CvHaarClassifierCascade*)cvLoad( cascade_name, 0, 0, 0 );
 63
 64    // Check whether the cascade has loaded successfully. Else report and error and quit
 65    if( !cascade )
 66    {
 67        fprintf( stderr, "ERROR: Could not load classifier cascade\n" );
 68        return -1;
 69    }
 70
 71    // Allocate the memory storage
 72    storage = cvCreateMemStorage(0);
 73
 74    // Find whether to detect the object from file or from camera.
 75    if( !input_name || (isdigit(input_name[0]) && input_name[1] == '\0') )
 76        capture = cvCaptureFromCAM( !input_name ? 0 : input_name[0] - '0' );
 77    else
 78        capture = cvCaptureFromAVI( input_name ); 
 79
 80    // Create a new named window with title: result
 81    cvNamedWindow( "result", 1 );
 82
 83    // Find if the capture is loaded successfully or not.
 84
 85    // If loaded succesfully, then:
 86    if( capture )
 87    {
 88        // Capture from the camera.
 89        for(;;)
 90        {
 91            // Capture the frame and load it in [[IplImage]]
 92            if( !cvGrabFrame( capture ))
 93                break;
 94            frame = cvRetrieveFrame( capture );
 95
 96            // If the frame does not exist, quit the loop
 97            if( !frame )
 98                break;
 99
100            // Allocate framecopy as the same size of the frame
101            if( !frame_copy )
102                frame_copy = cvCreateImage( cvSize(frame->width,frame->height),
103                                            IPL_DEPTH_8U, frame->nChannels );
104
105            // Check the origin of image. If top left, copy the image frame to frame_copy. 
106            if( frame->origin == IPL_ORIGIN_TL )
107                cvCopy( frame, frame_copy, 0 );
108            // Else flip and copy the image
109            else
110                cvFlip( frame, frame_copy, 0 );
111
112            // Call the function to detect and draw the face
113            detect_and_draw( frame_copy );
114
115            // Wait for a while before proceeding to the next frame
116            if( cvWaitKey( 10 ) >= 0 )
117                break;
118        }
119
120        // Release the images, and capture memory
121        cvReleaseImage( &frame_copy );
122        cvReleaseCapture( &capture );
123    }
124
125    // If the capture is not loaded succesfully, then:
126    else
127    {
128        // Assume the image to be lena.jpg, or the input_name specified
129        const char* filename = input_name ? input_name : (char*)"lena.jpg";
130
131        // Load the image from that filename
132        IplImage* image = cvLoadImage( filename, 1 );
133
134        // If Image is loaded succesfully, then:
135        if( image )
136        {
137            // Detect and draw the face
138            detect_and_draw( image );
139
140            // Wait for user input
141            cvWaitKey(0);
142
143            // Release the image memory
144            cvReleaseImage( &image );
145        }
146        else
147        {
148            /* assume it is a text file containing the
149               list of the image filenames to be processed - one per line */
150            FILE* f = fopen( filename, "rt" );
151            if( f )
152            {
153                char buf[1000+1];
154
155                // Get the line from the file
156                while( fgets( buf, 1000, f ) )
157                {
158
159                    // Remove the spaces if any, and clean up the name
160                    int len = (int)strlen(buf);
161                    while( len > 0 && isspace(buf[len-1]) )
162                        len--;
163                    buf[len] = '\0';
164
165                    // Load the image from the filename present in the buffer
166                    image = cvLoadImage( buf, 1 );
167
168                    // If the image was loaded succesfully, then:
169                    if( image )
170                    {
171                        // Detect and draw the face from the image
172                        detect_and_draw( image );
173
174                        // Wait for the user input, and release the memory
175                        cvWaitKey(0);
176                        cvReleaseImage( &image );
177                    }
178                }
179                // Close the file
180                fclose(f);
181            }
182        }
183
184    }
185
186    // Destroy the window previously created with filename: "result" 
187    cvDestroyWindow("result");
188
189    // return 0 to indicate successfull execution of the program
190    return 0;
191}
192
193// Function to detect and draw any faces that is present in an image
194void detect_and_draw( IplImage* img )
195{
196    int scale = 1;
197
198    // Create a new image based on the input image
199    IplImage* temp = cvCreateImage( cvSize(img->width/scale,img->height/scale), 8, 3 );
200
201    // Create two points to represent the face locations
202    [[CvPoint]] pt1, pt2;
203    int i;
204
205    // Clear the memory storage which was used before
206    cvClearMemStorage( storage );
207
208    // Find whether the cascade is loaded, to find the faces. If yes, then:
209    if( cascade )
210    {
211
212        // There can be more than one face in an image. So create a growable sequence of faces.
213        // Detect the objects and store them in the sequence
214        CvSeq* faces = cvHaarDetectObjects( img, cascade, storage,
215                                            1.1, 2, CV_HAAR_DO_CANNY_PRUNING,
216                                            cvSize(40, 40) );
217
218        // Loop the number of faces found.
219        for( i = 0; i < (faces ? faces->total : 0); i++ )
220        {
221           // Create a new rectangle for drawing the face
222            CvRect* r = (CvRect*)cvGetSeqElem( faces, i );
223
224            // Find the dimensions of the face,and scale it if necessary
225            pt1.x = r->x*scale;
226            pt2.x = (r->x+r->width)*scale;
227            pt1.y = r->y*scale;
228            pt2.y = (r->y+r->height)*scale;
229
230            // Draw the rectangle in the input image
231            cvRectangle( img, pt1, pt2, CV_RGB(255,0,0), 3, 8, 0 );
232        }
233    }
234
235    // Show the image in the window named "result" 
236    cvShowImage( "result", img );
237
238    // Release the temp image created.
239    cvReleaseImage( &temp );
240}
241

Hope you understood what the code meant.

How to run this code? As said in the code, you have to specify the exact location of the haar classifier xml file. It is normally present in the /OpenCV/data/haarcascades location. You should verify that the file haarcascade_frontalface_alt.xml is present inside the folder or not.

Suppose that you have installed OpenCV in its default location: C:/Program Files. Then, the exact location of the classifier is: "C:/Program Files/OpenCV/data/haarcascades/haarcascade_frontalface_alt.xml" Now, you can run the facedetect sample with this command in the prompt: facedetect --cascade="C:/Program Files/OpenCV/data/haarcascades/haarcascade_frontalface_alt.xml"

If you are using an IDE, and want to play with the source code, you'll need to add the classifier location to the additional command line arguments. In Visual C++, you can go to Project->Debugging->Command Arguments In Additional Options tab, enter this: --cascade="C:/Program Files/OpenCV/data/haarcascades/haarcascade_frontalface_alt.xml" Now, the source code will compile and run fine.

You can optionally specify the avi file or image file to the command prompt, after specifying the classifier. For example, If you have your own image: face.jpg, then you can add that to the command line like this:

--cascade="C:/Program Files/OpenCV/data/haarcascades/haarcascade_frontalface_alt.xml face.jpg

On Linux

  1. Go to the directory: opencv-0.9.7/samples/c/
  1. Change the permissions of build_all.sh file: chmod 777 build_all.sh
  1. Run the build_all.sh file: ./build_all.sh
  1. Run facedetect: ./facedetect --cascade="/home/bob/opencv-0.9.7/data/haarcascades/haarcascade_frontalface_alt.xml"
    1. (make sure you specify the exact path of an xml file!!!)
  1. To run with a parameter: ./facedetect --cascade="/home/bob/opencv-0.9.7/data/haarcascades/haarcascade_frontalface_alt.xml" baboon.jpg

What if you want to add this as a part of your source code?

Pass your image to the detect_and_draw function . Thats it.... Here is a sample code:

  1// Sample Application to demonstrate how Face detection can be done as a part of your source code.
  2
  3// Include header files
  4#include "cv.h" 
  5#include "highgui.h" 
  6
  7#include <stdio.h>
  8#include <stdlib.h>
  9#include <string.h>
 10#include <assert.h>
 11#include <math.h>
 12#include <float.h>
 13#include <limits.h>
 14#include <time.h>
 15#include <ctype.h>
 16
 17// Create a string that contains the exact cascade name
 18const char* cascade_name =
 19    "C:/Program Files/OpenCV/data/haarcascades/haarcascade_frontalface_alt.xml";
 20/*    "haarcascade_profileface.xml";*/
 21
 22// Function prototype for detecting and drawing an object from an image
 23void detect_and_draw( IplImage* image );
 24
 25// Main function, defines the entry point for the program.
 26int main( int argc, char** argv )
 27{
 28
 29    // Create a sample image
 30    [[IplImage]] *img = cvLoadImage("face.jpg");
 31
 32    // Call the function to detect and draw the face positions
 33    detect_and_draw(img);
 34
 35    // Wait for user input before quitting the program
 36    cvWaitKey();
 37
 38    // Release the image
 39    cvReleaseImage(&img);
 40
 41    // Destroy the window previously created with filename: "result" 
 42    cvDestroyWindow("result");
 43
 44    // return 0 to indicate successfull execution of the program
 45    return 0;
 46}
 47
 48// Function to detect and draw any faces that is present in an image
 49void detect_and_draw( IplImage* img )
 50{
 51
 52    // Create memory for calculations
 53    static CvMemStorage* storage = 0;
 54
 55    // Create a new Haar classifier
 56    static CvHaarClassifierCascade* cascade = 0;
 57
 58    int scale = 1;
 59
 60    // Create a new image based on the input image
 61    IplImage* temp = cvCreateImage( cvSize(img->width/scale,img->height/scale), 8, 3 );
 62
 63    // Create two points to represent the face locations
 64    [[CvPoint]] pt1, pt2;
 65    int i;
 66
 67    // Load the [[HaarClassifierCascade]]
 68    cascade = (CvHaarClassifierCascade*)cvLoad( cascade_name, 0, 0, 0 );
 69
 70    // Check whether the cascade has loaded successfully. Else report and error and quit
 71    if( !cascade )
 72    {
 73        fprintf( stderr, "ERROR: Could not load classifier cascade\n" );
 74        return;
 75    }
 76
 77    // Allocate the memory storage
 78    storage = cvCreateMemStorage(0);
 79
 80    // Create a new named window with title: result
 81    cvNamedWindow( "result", 1 );
 82
 83    // Clear the memory storage which was used before
 84    cvClearMemStorage( storage );
 85
 86    // Find whether the cascade is loaded, to find the faces. If yes, then:
 87    if( cascade )
 88    {
 89
 90        // There can be more than one face in an image. So create a growable sequence of faces.
 91        // Detect the objects and store them in the sequence
 92        CvSeq* faces = cvHaarDetectObjects( img, cascade, storage,
 93                                            1.1, 2, CV_HAAR_DO_CANNY_PRUNING,
 94                                            cvSize(40, 40) );
 95
 96        // Loop the number of faces found.
 97        for( i = 0; i < (faces ? faces->total : 0); i++ )
 98        {
 99           // Create a new rectangle for drawing the face
100            CvRect* r = (CvRect*)cvGetSeqElem( faces, i );
101
102            // Find the dimensions of the face,and scale it if necessary
103            pt1.x = r->x*scale;
104            pt2.x = (r->x+r->width)*scale;
105            pt1.y = r->y*scale;
106            pt2.y = (r->y+r->height)*scale;
107
108            // Draw the rectangle in the input image
109            cvRectangle( img, pt1, pt2, CV_RGB(255,0,0), 3, 8, 0 );
110        }
111    }
112
113    // Show the image in the window named "result" 
114    cvShowImage( "result", img );
115
116    // Release the temp image created.
117    cvReleaseImage( &temp );
118}

How to make the Face Detector work in MFC Application:

Contributed by*Salman Aslam*

If you're working with an MFC application, follow these steps to get your Haar Facial Detector working:

Step 1 Install Intel OpenCV. I will assume it's installed in the default directory : c:\program files\opencv

Step 2 Create a standard MFC application using the AppWizard.

Step 3 In Project Settings (Alt+F7), go to the C/C++ tab, then select Category Preprocessor, and for Additional Include directories, add the following: C:\Program Files\opencv\cv\include,C:\Program Files\opencv\otherlibs\highgui,C:\Program Files\opencv\cxcore\include,C:\Program Files\opencv\cvaux\include

Step 4 In Project Settings (Alt+F7), go to the Link tab, then select Category "Input", and for "Object/library modules:", add the following: cxcored.lib cvd.lib highguid.lib (see that there are no commas like in Step 3 above)

Step 5 While staying in the Link tab, and Category "Input", for the "Additional Library path:", add the following: C:\Program Files\opencv\lib

Step 6 In your main application file, eg Simple.cpp, include the following files: #include "cv.h" #include "highgui.h"

Step 7 In the same file, go to the InitInstance() function. Right after you see the lines:

1m_pMainWnd->ShowWindow(SW_SHOW);
2m_pMainWnd->UpdateWindow();

and before the line

1 return TRUE;

add the following lines of code

 1 //declarations
 2  CvHaarClassifierCascade* cascade;
 3  CvMemStorage* storage;
 4  [[IplImage]] *image;
 5  CvSeq* faces;
 6  const char* file_name;
 7  int i;
 8
 9  //initializations
10  storage=cvCreateMemStorage(0);
11  cvFirstType();
12  file_name="haarcascade_frontalface_alt.xml";
13  cascade = (CvHaarClassifierCascade*)cvLoad(file_name,NULL, NULL, NULL);
14  image = cvLoadImage("lena.jpg", 1 );
15  faces = cvHaarDetectObjects( image, cascade, storage, 1.2, 2,
16                CV_HAAR_DO_CANNY_PRUNING,cvSize(0, 0));
17
18  //draw rectangles
19  for(i=0;i<(faces ? faces->total:0); i++ )
20  {
21   CvRect* r = (CvRect*)cvGetSeqElem( faces, i );
22   [[CvPoint]] pt1 = { r->x, r->y };
23   [[CvPoint]] pt2 = { r->x + r->width, r->y + r->height };
24   cvRectangle( image, pt1, pt2, CV_RGB(255,0,0), 3, 8, 0 );
25  }
26
27     // create window and show the image with outlined faces
28     cvNamedWindow( "faces", 1 );
29     cvShowImage( "faces", image );
30     cvSaveImage("Result.jpg", image);
31
32     cvWaitKey();
33     cvReleaseImage( &image );      // after a key pressed, release data
34     cvReleaseHaarClassifierCascade( &cascade );
35     cvReleaseMemStorage( &storage );

Step 8 Now go to c:\program files\opencv\bin, and copy the following files to the folder where your MFC files reside:
cv097d.dll cxcore097d.dll highgui097d.dll

Step 9 Now go to c:\program files\opencv\samples\c and copy lena.jpg to the folder where your MFC files reside.

Step 10 Go to c:\program files\opencv\data\haarcascades, and copy the following file to the folder where your MFC files reside: haarcascade_frontalface_alt.xml

Step 11 Run your application.

A lot of people have been receiving an error while executing the following line:

1cascade = (CvHaarClassifierCascade*)cvLoad(file_name,NULL, NULL, NULL);

The error looks like this: Unspecified error (The node does not represent a user object (unknown type?)) in function cvRead, C:\Program Files\OpenCV\cxcore\src\cxpersistence.cpp (5040)

The solution is to use cxcored.lib, cvd.lib and highguid.lib instead of cxcored_i7.lib, cv.lib and highgui.lib.
You can use highgui.lib, but you get an error after the face detected image has been displayed and you're unloading it.

Another simple workaround is to call any function from cv.lib before the call to cvLoad. For example: create a dummy empty image, apply cvErode to it and release the image.

The above code (function detect and draw) has a serious memory leak when run in an infinite for loop for real time face detection. Please add "cvReleaseMemStorage(&storage);" after you release the temp image in the detect and draw function. The cvClearMemStorage only clears the previous set of values but does not free memory. I did not modify the initial code as it is simply not mine. I have also cleared "faces" using cvClearSeq in my process of trying to find the memory leak as the program would quickly crash for 640x480 res 30fps video. I am very thankful to the original programmer for the code. Please let me know if this helps.
AbhinayE

Tested on:

  • OpenCV Beta 5
  • Microsoft Visual C++
  • Windows XP