The goal of this project was to use EigenFaces to create a program that can automatically detect and recognize faces. This document describes two tests that were done on my implementation of the facial recognition algorithm.
For this test, I first used the nonsmiling_cropped images to generate various numbers of eigenfaces. The eigenfaces for the 10 eigenface case are shown in the table below.
|
|
|
|
|
|
|
|
|
|
|
Using these 10 eigenfaces, the program was able to correctly recognize 9/21 faces. With more eigenfaces, however, the score improved. The chart below shows the performance with respect to the number of eigenfaces used.
The chart shows a general upward trend, which implies that more eigenfaces are clearly better. However, even with 21 eigenfaces the software can only correctly recognize two out of three faces. I suspect that using the same number of eigenfaces as there are images in the training set means the program is basically overfitting the data. I wouldn't be surprised to see this not perform well as more test images are added.
The table below shows the recognition errors for the 21 eigenface case.
| Input Image | Program's choice | Correct result's rank |
 |
 |
3 |
 |
 |
4 |
 |
 |
4 |
 |
|
4 |
 |
 |
4 |
 |
|
12 |
 |
|
9 |
The mistakes seem somewhat hit or miss. In general, the correct result was in the top five. At a quick glance, in a lot of cases it seems reasonable that the computer would mistake the faces as being similar.
The first row, for example, has a lot of similarities. Both students wear glasses, and their heads are tilted in about the same amount. Furthermore, this is especially difficult because the expression on the student's face causes him to look very different than he did in his nonsmiling image.
In a lot of the other cases, there are arguably similar features between the input image and the result from the computer. If you just looked at the nose, for example, the matches tend to look more reasonable.
This section shows how well the program is able to automatically find faces in a larger image. My program did reasonable well on the elf image, but it had trouble on other images because it was not able to correctly handle low-texture areas.
In all cases, I set the minimum scale to 0.45, the maximum scale to 0.55, and the step size to 0.01.
This was one of the more successful images I looked at. Below is the face it cropped, and also the result of running the program in mark mode. In both cases it successfully drew a box around or cropped each face. It did not get the scale quite as tight as it should have, however, and there is a lot of non-face material in the cropped image and bounding boxes.
This clearly didn't work well. All of the "faces" were actually extremely low texture areas. I tried a different method for dealing with this issue than those suggested on the project web page. After looking at a few images showing the MSE for a face starting at each pixel, I noticed a couple of trends. The good face scores tended to follow edges around clothing and things like that. For places that were actually a face, there would be a small patch of extremely good scores surrounding by relatively bad scores. The score image looked a lot like I had run the original image through an edge detector.
Base on this, I decided to use a ratio test to score images. For each point in the image, I would also find the best score out of eight points on an 8x8 square centered at the given pixel. I used to ratio of the pixel's score to the best of its neighbors' scores to rank whether the point was a face. The idea was that the true faces would be a strong local minimum, while the high scores along edges would not score as well, because the neighbors would also have low scores.
The results show that this is not actually a good way of finding the best true faces in an image. Using color cues or weighting faces by their distance from the facial mean would have been a better choice.
This pulled a face out of my jacket. It seems that in many ways a face detector works kind of like an edge or corner detector. Thus, it put the upper left corner of a "face" where there is a strong corner in the image.
This was another group image form my own collection with four faces. As in the first group image, my program seems to have found anything but faces. In this case, some of the supposed faces are even in areas of fairly high texture. This gives further evidence that a ratio test as I have implemented is a very bad way to score potential faces.
I implemented the speedup algorithm suggested, which allowed much quicker experimentation, even with large eigenface sizes.
I also came up with my own way of scoring potential faces, although it does not appear to have been very effective.