Note that these images are 25x25 pixels, but are displayed as 100x100 in the browser for easy viewing.
Average Face:
Eigenfaces:
Recognition performance
The plot of recognition accuracy vs number of eigenfaces used clearly indicates two things: first, performance increases sharply until around 7 eigenfaces, and second that increasing the eigenface dimension beyond 7 has a debatable performance increase. The optimal number of eigenfaces is unclear since the performance goes up and down slightly and does reach a maximum at the far end (33 eigenfaces), but around 6 eigenfaces seems to be a good compromise between complexity and performance.
One failure was smiling-12 being recognized as neutral-20 with 9 eigenfaces:
The correct answer (neutral-12) appeared in spot 3. Although it's hard to judge whether this mistake is reasonable, the faces do have some similarities (especially considering that color is ignored).
Image of me. I searched from scale=0.9 to scale=1.2 in increments of 0.05.
Group Picture
Big group picture looking for 20 faces
Big group picture looking for 40 faces
MSE histogram for comparing all smiling images to their neutral pair
MSE histogram comparing all smiling images to neutral-32
True posives and false positives for image verification as the image/not-image threshold is varied.
I first computed the MSE values between each smiling image to its neutral pair as well as to a randomly chosen image (neutral-32 in this case) and output them to a file. I plotted the above histograms and noticed that a threshold of around 100000 was a good separator between correct and incorrect matches. I verified this by plotting the true positive rate and false positive rate and confirmed that 100000 was roughly the optimal threshold for minimizing false positives.
With threshold=100000, the false negative rate was 12% and the false positive rate was 3%.
I implemented the speedup to the eigenfaces computation. This made an enormous difference in performance and the only tricky part was making my own matrix multiplication functions, and making sure that my eigenfaces were normalized properly. This, along with a script to repeatedly run my program, made the first experiment go very quickly.
I tested different scoring functions for determining if an image is a face, and found that scaling the MSE by var(F-A)^2 / var(F) (where F is the potential face and A is the average eigenface) worked better than the proposed scaling. It penalized potential faces with low variance, and strongly penalized images which have much variance after being subtracted from the average image.