Rodrick Megraw

CSEP 576

Winter 2005

Project 4 - Eigenfaces

Extra Credit

I implemented the following extra credit items: 

1. aTa speedup - please see the speedup code in Faces::eigenFaces
2. verifyFace - please see code in EigFaces::verifyFace and experiment results below

Experiments - below are the results of the experiments

Testing recognition with cropped class images

Below is the average face and 10x25x25 eigenfaces.

Average Face

Eigenfaces

Face Recognition Accuracy - I plotted the percentage of correctly recognized smiling faces for 1-32 eigenfaces

Questions

1. Describe the trends you see in your plots. Discuss the tradeoffs; how many eigenfaces should one use? Is there a clear answer? 

The accuracy generally improved as the number of eigenfaces increased.At 8 eigenfaces accuracy began to plateau at 20 of 32 correct. Accuracy peaked at 21 of 32 correct with 12 eigenfaces. Around 5-10 eigenfaces seems to give the best accuracy for the total computational expense. More than 10 adds no significant additional accuracy and takes longer to compute. Of course, this is only true in this one experiment. For other groups of faces, there may be other thresholds that make sense. What is clear is that there is a tradeoff between number of eigenfaces and accuracy, and that there is diminishing returns to more eigenfaces. In general one should use enough eigenfaces to just reach the beginning of the accuracy plateau.

2. You likely saw some recognition errors in step 3; show images of a couple. How reasonable were the mistakes? Did the correct answer at least appear highly in the sorted results?

Here are examples of some mistakes 

Smiling Face	Correct Non-smiling Face	Mismatched Non-smiling Face	Rank of Correct Non-smiling Face
			3
			2
			2
			12
			16

The mistakes appear very reasonable. In most cases the MSE's are very close and the correct face is in the top 5 in the ranking. In cases where the correct face is further down in the ranking the person's face is often in a very different position than their smiling face. The background also appears to be a factor in the mismatches. 


Cropping and finding faces

I performed these experiments using 10x25x25 eigenfaces.

Cropping Bush - This worked great. I settled on the suggested step parameters of .25, .55, .01. .1, .5, .1 worked equally well.

Class group (4/4)

7-person Group (4/7)

1 person (1/1)

Questions

1. What min_scale, max_scale, and scale step did you use for each image? 

For the 8270 image I has the best success with .55 .55 .1, but .55 .7 .1 also worked well and demonstrated the the ability to find faces of different scales.

For the 7-person group image I used .8 .95 .01. This image required multiple scales to find the most faces. Note the wood flooring and light carpet gave false positives.

For the single person crop I used .11 .11 .1. It was very difficult to find a 1 person shot that worked well. Any photo with outdoor lighting or woodgrain in the image gave false positives. This image or a former co-worker works well because of the dark background.

2. Did your attempt to find faces result in any false positives and/or false negatives? Discuss each mistake, and why you think they might have occurred.

I encountered many false positives and false negatives. In many cases the found face was close to the persons shoulder or a piece of dark clothing. In outdoor pictures it was difficult to get good results. I  am guessing this is due to the brighter lighting and more complex background.

Verify Face (Extra Credit)

Procedure

For this experiment I used the cropped, non-smiling students to compute 6 eigenfaces. I then generated a userbase with the same non-smiling faces. I then wrote a script to verify each student against his or her smiling face, as well as each student against a smiling face which is of someone else. I repeated the experiment for MSE thresholds between 0 and 10000 in steps of 50. Below is a plot of the results.

We can see that at the MSE threshold increases, the false negatives decreases and the false positives increases. A threshold of about 2750 gives the fewest total incorrect identifications. However the optimal MSE threshold would depend on the use of the system. In cases where it is ok to have some more false positives to decresse false negatives, we may wish to choose a higher threshold. Above 6000 there are no false negatives but there are many false positives. A low threshold might be used if false positives are to be avoided.

1. What MSE thresholds did you try? Which one worked best? What search method did you use to find it?

I tried 0-10000 in steps of 50. A threshold of about 2750 minimizes total incorrect identifications. I used a script to brute force search and test the thresholds. I then made a plot to identify the point of fewest total incorrect identifications.

2. Using the best MSE threshold, what was the false negative rate? What was the false positive rate?

At an MSE threshold of 2750 the false negative rate was 15.6% and the false positive rate was 31.2%.