These images have been produced at 50 x 50 just to make them easier to see
| Average face | |
| Eigenfaces |
|
The graph plots the number of faces recognized against the number of eigenfaces used. This was done with 25 x 25 pixel eigenfaces.
Once we get to 8 eigenfaces we reach a plateau - there is one peak and a dip but the overall trend is steady. The suggested value of 10 eigenfaces seems reasonable - increasing the number of eigenfaces does not improve the accuracy. Whilst a good result is reached at 8 eigenfaces it is probably not a good idea to be working at the lower limit of the high accuracy area.
There isn't a clear answer to the question. Using too few eigenfaces reduces the accuracy, using too many eigenfaces may lead to overfitting.
In addition to finding the best match I also implemented a "confidence" calculation as follows:
confidence = ( second ranked MSE - top ranked MSE ) / second ranked MSE
The closer the confidence value is to 1 the more the top ranked MSE differs from the second ranked MSE. I've divided the results into pass and fail categories, then ranked the results within each category by the confidence.
| Smiling image | Actually matched | Should have matched | Confidence | Rank of correct result in answers | |
|---|---|---|---|---|---|
 |  | | |||
 |  | | |||
 |  | | |||
 |  | | |||
 |  | | |||
 |  | | |||
 |  | | |||
 |  | | |||
 |  | | |||
 |  | | |||
 |  | | |||
 |  | | |||
 |  | | |||
 |  | | |||
 |  | | |||
 |  | | |||
 |  | | |||
 |  | | |||
 |  | | |||
 |  | | |||
 | |  | |||
 |  |  | |||
 | |  | |||
 | |  | |||
 | |  | |||
 | |  | |||
 | |  | |||
 | |  | |||
 | |  | |||
 | |  | |||
 | |  | |||
 | | | |||
Small changes - a slight smile or sticking the tongue out a little way do not confuse the algorithm. There are a couple of good examples here:
| Smiling image | Actually matched | Should have matched |
|---|---|---|
| | |
| | |
The more extreme the "smiling" the more likely the algorithm would match the face incorrectly - in particular opening the mouth wide or sticking the tongue out a long way leads to a big change in colour across a large part of the image as seen here:
| Smiling image | Actually matched | Should have matched |
|---|---|---|
| | |
| | |
| | |
It looks as though the tongue, whether sticking out or visible through the open mouth, is often mistaken for a light bottom lip.
Tilting of the head also confuses the algorithm - perhaps one useful preprocessing step would be to try and get all faces upright, or maybe the training data should include more faces at an angle.
| Smiling image | Actually matched | Should have matched |
|---|---|---|
| | |
| | |
The confidence rating is interesting, although not conclusive. None of the incorrectly identified images have a confidence greater than 0.58, however the correctly identified images go down to a confidence of 0.04. Perhaps a more sophisticated measure would make this more useful - sometimes it's good to know when you're likely to be correct and when you're likely to be incorrect.
Many of the failures don't even have the correct image in the top three - when this algorithm fails it really fails.
I implemented findFace as described with the extra tweaks of multiplying the MSE by the distance of the face from the mean and dividing by the variance of the face. These tweaks helped a little, but it was still prone to getting fooled by areas of low variance.
George Bush's face was found with the given parameters:
 |
 |
 |
min_scale = 0.25 max_scale = 0.55 step = 0.01 |
Since my face was a lot bigger I used a much smaller scale to find it:
 |
 |
 |
min_scale = 0.15 max_scale = 0.30 step = 0.01 |
Using 25 x 25 pixel eigenfaces meant I needed to make sure that the scales were set so that at one of the scales, the faces in the image would be around 25 x 25 pixels. I also wanted to make sure that there was some latitude either side - I made the program print out the scale of the faces it found and used that to tweak the parameters for subsequent runs. I used a large step size and wide bounds to get a rough idea of the scales involved, then used a smaller step size and narrower bounds to fine tune it.
The algorithm did well on all of the images from the class:
min_scale = 0.55
max_scale = 0.65
step = 0.01
min_scale = 0.50
max_scale = 0.70
step = 0.01
However it did a horrible job on a different group image:
min_scale = 0.20
max_scale = 0.30
step = 0.04
The different lighting gave the algorithm problems, and despite dividing by the variance it still has a tendency to be fooled by low texture areas. Some colour matching would also be useful in eliminating many of the errors.
I implemented the suggested speedup. Use --eigenfaces_fast to run the faster version of the algorithm.
Having implemented the suggested speedup I tested the recognition rate for a variety of eigenface sizes:
There's no real advantage in using eigenfaces larger than 15 x 15 pixels, ther graphs are all pretty similar, and 15 x 15 actually does slightly better with lots of eigenfaces. The larger the eigenfaces the more likely we are to overfit to our training set, sticking with smaller eigenfaces may well be better.
I produced 6 eigenfaces and a userbase from the non-smiling images then verified every student against every smiling image. This gives 1024 results, in particular it gives an MSE value for every smiling / non-smiling pair. With a perfect algorithm there would be a single MSE value which would divide these results into "match and "non-match" groups.
To find the best MSE threshold to use I took every MSE value generated by the 1024 tests and calculated what the rate of false positives and false negatives would be using that threshold. This gave me the probability of getting a false positive and the probability of getting a false negative, I squared the two probabilities then added them. The squared error (which we'd like to minimize), was plotted on a graph:
The best MSE was 123265.07 which gave 5 false negatives and 191 false positives.
As described above I implemented a confidence value for the face recognition task. This was of limited success, the correlation isn't really good enough to be useful.
The algorithm allows us to find faces in images, an obvious next step is to allow faces to cut and pasted on top of each other. I implemented a pyramid blending technique for this. The results are interesting, and a little disturbing:
The blending can be accessed by using the --blend option and specifiying the number of levels in the pyramid, and two sub areas of images. Each sub area is given by the x, y coordinates of its upper left hand corner and its size (the areas must be square, hence only one size is required). The given area from image 2 will be blended on top of the given area from image 1 and the resulting image written to the output file.
--blend <nLevels> <targa filename1>
<x1> <y1> <size1>
<targa filename2>
<x2> <y2> <size2>
<outfile name>
The image above was produced with the following set of commands:
eigenfaces --blend 4 group\img_8271.tga 127 196 50 group\img_8271.tga 425 198 48 blendtest1.tga
eigenfaces --blend 4 blendtest1.tga 270 196 49 group\img_8271.tga 425 198 48 blendtest2.tga
eigenfaces --blend 4 blendtest2.tga 574 215 49 group\img_8271.tga 425 198 48 blendtest3.tga
The code is in the files blendimages.cpp and blendimages.h.