In this project, I used the concept treating images as vectors in a high-dimensional vector space and the use of eigen vectors to project those images onto a hyperplane to build a system for finding and identifying faces.

The find-faces was especially difficult because there are many areas of the image that are a good match for a face from the eigenvectors but are not a face. Trying to fix this required a lot of tinkering with the weight of the difference between a window on the image and the average as well as the varience of the window towards the final score.

Based on the find faces feature, I learned that the more information you use in the program about whether some image is a face or not, the better the results will be. For example, including the difference of the average face and the image in the final score gives better results than just using the mean squared error alone. I would imagine that giving wieght to skin color and contrast as well would give even better results.

This program treats images as vectors in a many dimensional vector space based on the values of the pixel. It uses the sample faces to define a hyperplane in this vector space that is described by eigenvectors found from the sample faces. It determines whether an image is a face or not by looking at how far it is from the hyperplane, and can recognize a face by projecting an unknown face onto the hyperplane and seeing which known face on the hyperplane it is closest to.

The project is structured as follows:

• Faces::eigenFaces - Defines the hyperplane by computing the eigenvectors from the sample faces
• EigFaces::projectFace - Projects a face onto the hyperplane and returns the eigenvector coefficients for that face that defines where the face lies on the hyperplane.
• EigFaces::constructFace - Re-creates a face from eigenvector coefficients.
• EigFaces::isFace - Determines whether or not a given image is a face.
• EigFaces::verifyFace - Verifies whether a given face matches a given known face.
• EigFaces::recognizeFace - Takes an unknown face and sorts the known faces in order of likelyhood that each face matches the unknown face.
• EigFaces::findFace - Searches for faces within an image

I created 10 eigen-faces from the sample faces and attempted recognition on the 33 smiling faces.

I also attempted recogintion on the 33 faces using many different numbers of eigenfaces.

Question 1

Increasing the number of eigen faces improved the results only until I had 5 eigen faces. After that, using more eigen faces barely improved the recognition capability. Since using the smallest number of eigen faces possible is the most efficient, it would probably be best to use only 5 or 7.

Question 2

There were 10 errors, for a total of 22/33 (67%) correct. Most of the errors were fairly resonable and the correct answer appeared high in the list of candidates. An example is how this face was matched to this face . Both faces have glasses and are at about the same angle. A little less understandable was that this face was matched to this face .

Recognition tended to match faces that had similar attributes such as angle, hair length, whether or not glasses were being worn, etc. It makes sense that it would make these mistakes since the faces are similar and it is doing exactly what we expect it to do. This also explains why adding more eigen faces did not improve the results, since one smiling face is more similar to a different neutral face.

The number of faces guessed correctly as a function of the number of eigen faces used.

the average of the neutral faces

the 10 eigen faces

I experimented with different scales until I got satisfactory results.

Question 1

See images below

Question 2

The only image that had false negatives or positives was the large class image. Here many faces remain unmarked and many marks do not correspond to a face. I think this is partially due to to the limits of using eigenvectors, which is just an approximation for a face and does not truley capture the various elements of a face. Another reason is that I had to weigh the results by the variance to avoid featureless areas. However, this gives a greater score for places that aren't a face but have high variance. Another issue is that many of the boxes overlap. I tried to implement a box-collision detection feature to avoid this, but I was never able to get it working for every case.

main --findface elf.tga eigenface .45 .55 .01 crop 1 elf-cropped.tga

The cropped elf picture

main --findface self.tga .2 .3 .01 crop 1 self-cropped.tga

The cropped picture of myself

main --findface group.tga .3 .4 .01 mark 3 group-marked.tga

The marked faces in the group image

main --findface oldclass.tga .5 .7 .02 mark 32 class-marked.tga

The marked faces in a different group image

I ran verify face on each of the smiling images, first against the corresponding non-smiling image for the true positives and then against the next numbered non-smiling image for the true negatives.

Question 1

I first chose 500 out of the blue (I had no idea what range of MSE would result, so this is obviously much too low). I then took the MSEs as reported by the program output and entered them into excel. A made a graph of the MSEs of the true negatives and the true positives, then estimated a value that seemed like it would divide the two the best. I then checked this numer against the reported MSEs to see that it would in fact produce the best results. I chose 100000 as my best value. I reran the tests and got the results.

Question 2

I had 4 false negatives and 5 false positives

A graph of the MSEs for the true positives and the true negatives. I chose the best mse by just picking a value that seemed to best divide them (100000).

Neither bells nor whistles were implemented