In this project, we treated faces as vectors and did a PCA to make eigenfaces, which we then used for face matching, reconstruction, and location.

The biggest challenge in this project was getting face detection to work, which is still only kind of does. The face matching and reconstruction were quite easy, but face finding... man.

It might be a wise idea to go to office hours when stuck. I started the assignment early, put over twenty hours into it, and still didn't finish.

The project is structured as follows:

• Faces::eigenFaces - This method loads in a set of faces, then does a PCA on them to find the components of face space. Once we have these, we can approximate any face by adding multiples of these components to the average face.
• EigFaces::projectFace - Given a face, we can project it onto face space to get an approximation that we are then able to construct using only the mean face and the eigenfaces.
• EigFaces::constructFace - projectFace returns a list of coefficients. constructFace takes those and multiplies them by the eigenface, then adds them to the mean face to reconstruct the face projection.
• EigFaces::isFace - This method projects a face onto the face space, then finds how different the projection is from the original. Closer? Probably a face (or a wall, or a crotch). Farther? It's probably something like a puppy (or, given my performance below, a face -_-).
• EigFaces::verifyFace - Given a face, a username, and a database of faces and users, this method determines if the face belongs to that user. It projects both faces onto the face space (well, that part is already done. We're actually given two sets of coefficients), then finds the difference between the two sets of coefficients. The larger the difference, the less likely it is that they're the same person.
• EigFaces::recognizeFace - This method takes a face and a database, and searches through the entire database to find which user's face is closest to the one given.
• EigFaces::findFace - This method (supposedly) finds faces. It resamples at every given scale, then makes a face out of every possible area, and keeps the top few that are likeliest to be faces. Despite every other method working, and making sure to multiply weights by the distance to the average face and dividing by variance, this method still barely works.

It seems that more eigenfaces tends to correspond with more matches, at a rate that seems logarithmic. Going from 1 to 3 was an incredible jump, but the rate slows down incredibly quickly.

Here are some failed matches at ten eigenfaces:

and

and

They seem reasonably similar to each other, so I think that this isn't really that bad of a matching failure.

For the picture of myself, I used a min of 0.1, a max of 0.3, and a step of 0.01. For the group picture that was provided, I used a min of 0.8, a max of 1.2, and a step of 0.01. For the group picture I was in, I used a min of 0.2, a max of 0.5, and a step of 0.01.

I got LOTS of both false positives and false negatives. The program kind of found my face, but not really that well. With the photo of my frieds and I, it found no faces. It did manage to find a face in the group shot, but it also found a guy's crotch... I've checked, and my program tends to really like areas that either have a high variance (because I divide by variance, as is suggested in the assignment) or areas that tend to just be a texture that can kind of look like a face if you squint just right, giving it a close MSE to the mean face. I've done projections, and some actual faces are definitely further than some blankish areas are.

...oops. My algorithm seems to really prefer blank areas, despite multiplying by the distance to the mean.

...better. Seems to also really like complex areas, perhaps due to dividing by variance?

It found a face! And a crotch!

Sigh. Looks like it found everything but a face.

I tried every MSE between 2500 and 500,000, with a step of 2500. As for which worked best, that depends on your definition of best. The true positives overtake the false negatives at an MSE of 45,000, while the false positives overtake the true negatives at about 32,000. The attached graph shows it more clearly.

Once again, the question of "best MSE" is an unanswerable one until we know exactly what we're looking for. But the attached graph does have a wide selection of values at different MSEs.