Author: Ansuman Kar
The aim of the project was to create a face recognition system using Principal Component Analysis (PCA) of face images. This project was lots of work but the experiments using the developed system were lots of fun.
Implementing all parts of the face recognition methods was comparatively straight forward. I also implemented the proposed speedup early and that helped reduce the experiment time by an order of magnitude. With the speedup, the release executable is able to calculate 100x100 eigenfaces within a few seconds.
I also implemented the verifyFace and morphFaces methods and spent part of the weekend morphing various pairs of faces & laughing at the intermediate faces generated (sort of like the Conan O’Brien show).
Face detection was a much harder problem. From early experiments I found that my code was not very good at making the detection and the process was also quite slow. Using the hints from last year’s class I implemented a skin detection mechanism to limit the face search to start around skin pixels and ignore regions of non-skin pixels. I have used the approach described in Jones & Rehg paper available at: (http://www.cc.gatech.edu/~rehg/Papers/SkinDetect-IJCV-lowres.pdf). This greatly reduced the run time and also improved the results.
However a second problem I encountered is that the executable would find the same faces several times even though I was comparing the face positions. I realized that an exact match was too hard so I added a ‘soft’ matching logic – face positions that are distant only by x*eigen_face_size are now assumed to be the same face, where x can be specified by the user. This nudges the algorithm to find multiple faces.
I have also used the hints of multiplying the face MSE by distance from mean face and dividing it by the face variance and that helps improve the results.
My final tests on large images run within a few minutes but the results are not perfect. I can think of two possible causes – the training set only consists of non-smiling faces and when the utility encounters smiling faces with teeth showing, it does do as good a job. Secondly, I added a scaling factor to the face variance to correct for the fact that images in larger scale sizes have higher variance and therefore lower MSE. However the utility now favors faces in smaller scale so I fear I may have overcorrected.
Extra credit items implemented are listed below:
1) Speedup for eigenface calculation using the ATA approach (in Faces::eigenFaces)
2) Implemented morphing (in EigFaces::morphFaces)
3) Implemented verifyface (in EigFaces::verifyFace) – Note that I have tested that it works but due to lack of time could not perform the proposed experiment.
4) Implemented a skin detection utility and used it for improved face detection. This was also a second speedup for face detection and greatly reduces the run time.
All script files and results are also available under the experiments folder.
Sizes 25x25
Sizes 100x100
The detailed results are available in the experiments folder under exp2. The accuracy graph is as below:
With as few as 8 eigen faces, the utility is achieving 62.5% accuracy. Surprisingly, adding more eigen faces does not improve accuracy. However I observed that visually, adding more faces upto 32 does have improvements, especially in the morphing cases.
Running the utility with 10 eigenfaces of 25x25 pixels and listing the best 5 matches for each smiling face, I observer that for some faces it does not list the actual person even in the best 5. Here they are (for each pair: on left is the smiling face, on the right the best match):
The only noticeable pattern I observe is that the head tilt seems to contribute, and also that because the utility is trained only on non-smiling faces it does not do a very good job of accurately matching smiling ones.
Note that increasing eigen faces to 32 only reduces mismatches by 1 (the 3rd pair).
I have used skin detection on the original image to cue the face detection searches. Here are the results:
Original Image Skin pixel mask 2 best faces found crop result
I have used skin detection on the original image to cue the face detection searches. Here are the results:
Original Image Skin pixel mask 2 best faces found crop result
It seems to do very well on my image, I think because my face was in the training set.
The utility ran quickly but did not have as good results on large images.
Skin pixel mask generated
Seven best faces found
Here’s another set:
Skin pixel mask generated
Seven best faces found
And here’s a third
set on the Bush family photograph:
(It seems to find faces on the floor artifacts due to their colors)
Skin pixel mask generated
Best faces found
Finally for fun, I did some face morphing using the utility. I had the videos on Movie maker but for some reason I cannot embed them properly in this page. So here are the static images instead (not as much fun but still OK).
The first one is a morph from my face to Marcelo’s face. This was done using 32 eigen faces for best visual clarity.
The second one is a morph from President Bush to a monkey (or is it really a morph…) Note that this was done only using 2 eigen faces and it looks surprisingly good.