CSEP576 Project 4: Face Detection & Recognition

Author: Ansuman Kar

 

Overview:

The aim of the project was to create a face recognition system using Principal Component Analysis (PCA) of face images. This project was lots of work but the experiments using the developed system were lots of fun.

 

Implementing all parts of the face recognition methods was comparatively straight forward. I also implemented the proposed speedup early and that helped reduce the experiment time by an order of magnitude. With the speedup, the release executable is able to calculate 100x100 eigenfaces within a few seconds.

 

I also implemented the verifyFace and morphFaces methods and spent part of the weekend morphing various pairs of faces & laughing at the intermediate faces generated (sort of like the Conan O’Brien show).

 

Face detection was a much harder problem. From early experiments I found that my code was not very good at making the detection and the process was also quite slow. Using the hints from last year’s class I implemented a skin detection mechanism to limit the face search to start around skin pixels and ignore regions of non-skin pixels. I have used the approach described in Jones & Rehg paper available at:  (http://www.cc.gatech.edu/~rehg/Papers/SkinDetect-IJCV-lowres.pdf). This greatly reduced the run time and also improved the results.

 

However a second problem I encountered is that the executable would find the same faces several times even though I was comparing the face positions. I realized that an exact match was too hard so I added a ‘soft’ matching logic – face positions that are distant only by x*eigen_face_size are now assumed to be the same face, where x can be specified by the user. This nudges the algorithm to find multiple faces.

 

I have also used the hints of multiplying the face MSE by distance from mean face and dividing it by the face variance and that helps improve the results.

 

My final tests on large images run within a few minutes but the results are not perfect. I can think of two possible causes – the training set only consists of non-smiling faces and when the utility encounters smiling faces with teeth showing, it does do as good a job. Secondly, I added a scaling factor to the face variance to correct for the fact that images in larger scale sizes have higher variance and therefore lower MSE. However the utility now favors faces in smaller scale so I fear I may have overcorrected.

 

 

Extra Credit implementations

Extra credit items implemented are listed below:

1)      Speedup for eigenface calculation using the ATA approach (in Faces::eigenFaces)

 

2)      Implemented morphing (in EigFaces::morphFaces)

 

3)      Implemented verifyface (in EigFaces::verifyFace) – Note that I have tested that it works but due to lack of time could not perform the proposed experiment.

 

4)      Implemented a skin detection utility and used it for improved face detection. This was also a second speedup for face detection and greatly reduces the run time.

 

 

Experiments and explanations

All script files and results are also available under the experiments folder.

Experiment 1: Average face and 10 eigenfaces

Sizes 25x25

 

               

 

Sizes 100x100

               

 

     

 

 

Experiment 2: Computer userbase and recognize smiling faces

The detailed results are available in the experiments folder under exp2. The accuracy graph is as below:

 

With as few as 8 eigen faces, the utility is achieving 62.5% accuracy. Surprisingly, adding more eigen faces does not improve accuracy. However I observed that visually, adding more faces upto 32 does have improvements, especially in the morphing cases.

 

Some Faces that the utility does not recognize correctly:

Running the utility with 10 eigenfaces of 25x25 pixels and listing the best 5 matches for each smiling face, I observer that for some faces it does not list the actual person even in the best 5. Here they are (for each pair: on left is the smiling face, on the right the best match):

 

                        

 

                            

 

                             

 

 

The only noticeable pattern I observe is that the head tilt seems to contribute, and also that because the utility is trained only on non-smiling faces it does not do a very good job of accurately matching smiling ones.

 

Note that increasing eigen faces to 32 only reduces mismatches by 1 (the 3rd pair).

 

Experiment 4: Face Detection on President Bush’s image

I have used skin detection on the original image to cue the face detection searches. Here are the results:

 

                                      

Original Image  Skin pixel mask            2 best faces found         crop result

 

Experiment 5: Face Detection on Self Portrait

I have used skin detection on the original image to cue the face detection searches. Here are the results:

 

                 

Original Image  Skin pixel mask            2 best faces found         crop result

 

It seems to do very well on my image, I think because my face was in the training set.

 

Experiment 6, 7: Face Detection on Large Images

The utility ran quickly but did not have as good results on large images.

 

Skin pixel mask generated

 

Seven best faces found

 

 

Here’s another set:

 

Skin pixel mask generated

 

Seven best faces found

 

 

And here’s a third set on the Bush family photograph:

(It seems to find faces on the floor artifacts due to their colors)

 

Skin pixel mask generated

 

 

Best faces found

 

Morphing Experiments:

Finally for fun, I did some face morphing using the utility. I had the videos on Movie maker but for some reason I cannot embed them properly in this page. So here are the static images instead (not as much fun but still OK).

 

The first one is a morph from my face to Marcelo’s face. This was done using 32 eigen faces for best visual clarity.

    

 

    

 

 

The second one is a morph from President Bush to a monkey (or is it really a morph…) Note that this was done only using 2 eigen faces and it looks surprisingly good.