Questions:

#0: Why is it best to use as few eigenfaces as possible while still getting good results?

As you can vaguely see from the results in the table above, the number of faces correctly recognized starts to grow as the number of eigenfaces increases, but after a certain amount of increase the number of faces correctly recognized starts to decrease. Thus it appears that the number of faces that can be recognized correctly forms a curve with an optimal position or peak which corresponds to the point at which the number of eigenfaces recognizes the most number of faces correctly. However, my results shown here are not enough for me to conclude that this hypothesis of mine is always true.

In addition, using too many eigenfaces simply wastes space and computation time, both of which could be large for databases with thousands of users in the user set.

#1: How many students did it recognize correctly?

From the table above, we see that when using 10 eigenfaces created from the undergraduate "cropped" images, the algorithm recognized 4 out of the 17 students. Using 12 or 15 eigenfaces, 3 out of the 17 stuends were recognized correctly. Lastly, using 5 or 20 eigenfaces, only 2 out of the 17 students were recognized correctly.

#2: Are the incorrect identifications reasonable?

The results are somewhat, what should I say, "fishy" for this part. I think that the changes in the brightness of the images makes a difference in what faces it can properly represent with eigenfaces. Thus, some faces are returned as the nearest result for many of our in class images, although they might not represent the person in that image, they are the most similar kind of picture to our in class picture.

Actually, 15 out of the 17 people were recognized as girls. I think for some reason the girls faces, such as "tamoore", "esp", "melissa", "margaux", and "mhl" were recognized a lot.

For example, "tamoore" was an undergraduate face that mapped well to eigen space and returned #1 for a lot of my queries.

Searching for: 'mhl'   Recognized as: 'tamoore'

Notice how the faces are both strait on, non-smiling, with dark eyes and distinct eyebrows.

Searching for: 'tshail'   Recognized as: 'tamoore'

Notice that these pictures are girls and how closely the eyebrows and eyes match tamoore's picture as well as mhl's picture.

Searching for: 'tamoore'   Recognized as: 'tamoore'

Notice how this is indeed tamoore and she has been recognized correctly.

Last, but not least, the results of the search for my very own face.

Searching for: 'paullarp'   Recognized as: 'melissa'   Followed by: 'esp'

I don't know how it figured out that I wear glasses... but I do. ...um, no comment.

Just for a sanity check, I searched for what "jwkim" would look like if she had a undergrad picture.

Searching for: 'jwkim'   Recognized as: 'esp'

This is what I was expecting, the glasses were recognized correctly. Horray! My program is actually working correctly after all. Thus in conclusion I have shown that even though the results of this recognition process are mostly incorrect, they are still rational most of the time.

#3: Why is the program worse at recognizing people now?

The programs decrease in accuracy can be explained by the following:

• The userbase created is now much smaller than the user set that was used to create the eigenfaces. Thus, the eigenfaces are not tailored as much towards the 17 people in our class but are rather more genereal and unspecific. This will decrease how close any of our 17 faces can get to eigen space, and when faces in the userbase are far from eigenspace, it is hard to accurately recognize an individuals face.
• The brightness of the images used to create the eigenfaces has greatly changed from the brightness of the images that we are trying to recognize. Skin tone and different features are different under different lighting. The change in brightness also makes most of our MSE's seem very large and the images should be corrected for brightness before being inputted into this algorithm.
• The emotional expressions of the faces are different in most of the eigenfaces when compared to our expressions in the "non"-smiling in class pictures. This might decrease the accuracy of some of our results.

#4: How did changing the number of eigenfaces affect your results?

Please read my comment above for question "#0" from this section. As you can see, my best results were found using 10 eigenfaces in this experiment.

Experiment  #3: Cropping the undergraduate images.

My Cropping Results:

Original Picture	Cropped Picture	Your Optimal Cropped Result (if in our class)

#1. What scales did you use?

I ended up using a min_scale of .15 and a max_scale of .20 with a step size of .01. These parameters seemed to work for the face sizes found in most of the images, while taking about 10 seconds to compute. The scale step of .01 seemed to work out pretty good too for getting the correct part of the face that we want.

#2: How many of your crop results look correct? How many look incorrect?

6 out of the 10 images came out perfectly. 2 of the 10 images came out good enough but included a little bit too much of the neck. And lastly 2 of the images were slightly off because the face was either too small or too big. To correct this, these images would have to have the min/max scale changed a little bit to work properly for them.

However, 6 out of the 10 were perfect and thus the face recognition seems to be working pretty good when there is only one face to recognize.

#3. What's the problem with using a min_scale that is too small?

Sometimes when I make the min_scale too small, the algorithm closest matches a far away picture of the persons face, which is not exactly what we want.

If I decrease the min_scale so small that it is smaller than an eigenface, I will simply be wasting time because I am only searching for eigenfaces that can actually fit into the image. Also, the image may become so small and distorted that it no longer contains any distinct facial features. It also takes a long time for my program to process images so I try to set the scale as close to correct as possible.

Experiment  #4: Finding faces in group photos.

Picture #1

This is one of my graduation pictures with my friends, at the Tacoma dome.

This is the result of the face finding operation.

The parameters used for this image were: min_scale = .42; max_scale = .54; step = .02;

As you can see, it nearly got two faces correct! But this is acceptable because the orientations of some of the peoples faces are slightly tilted and my face has weird lighting conditions. My two African American friends were also not recognized, which might be due to the lack of training information similar to their faces in the user set that I created the eigenfaces from.

Picture #2

This is a picture of my favorite J-POP singing group, SPEED. Heck'of a lot nicer picture than the one above! ;-)

Here's the results of my face search.

It recognized, from left to right, Eriko, Takako, and Hitoe, but it sorta missed Hiro's face on the far right.

The parameters used for this image were: min_scale = .26; max_scale = .42; step = .02;

There was one error in this image, so I tried dividing the MSE by the variance, but this caused some more errors and selected regions with strong edges in them, such as where the left side of the left-most girl's hair meets the background. To improve the algorithm, I think that some extra normalization steps could be taken on the input image to ensure that regions that are totally dark and could not possibly be a face are made even darker, while the colors that usually represent faces are preserved.

Conclusion

This project has also proved to be a very fun one, although challenging at first and slightly discouraging until you get it to start recognizing faces CORRECTLY! But once I had that working, the project was fun, however I read there there is supposed to be an N squared algorithm for solving the eigenvector equation instead of an N cubed equation. I think the math is much harder though, but that might be an interesting thing to look into someday. I really like how well the program is at cropping a single persons face when you give the program of just one person.

Thanks

Thanks to David Dewey for the time he spent on the skeleton source code, and also to Jiwon Kim for all the help that she has given us in creating and grading the interesting projects in this class. Also thanks to our Professor Steve Seitz for making the class material very understandable, even though much of what we learned and discussed is not yet even in a textbook, but rather more on the research frontier. And as I'm thanking everybody, I thank my classmates for making some outstanding projects, especially some of the panoramas and those project 3 artifacts with all of the stairs! Oh my goodness, that must have taken a lot of patience!

Thanks for the great quarter,

-Paul Larpenteur

Visit Paul's website at: http://students.washington.edu/paullarp/