CSE 576 Project 3: Eigenfaces

I experimented with two settings: eigenface size 15x15 and size 25x25. The graph on the right shows the relationship between the number of eigenfaces and its performance. In general, the accuracy increases as the number of eigenfaces increases. In 15x15 case, there is a tendancy that if the number of eigenfaces is above a certain amount, the accuracy does not improve much (nine eigenfaces are enough). In 25x25 case, it is hard to pick the optimal number of eigenfaces, because the accuracy improves even there are more than fifteen eigenfaces. The image below shows the average face and ten eigenfaces (size 25x25).

The following are sample matching results with ten eigenfaces (size 50x50). For each sample, the original image (left), and its reconstructed image (right) are shown. In the second example, MSEs are high in general. The original face is pretty close to the average face, which makes candidates not distinguishable to the other faces. The first example has pretty similar reconstruction from the original, and its matching MSE is considerably low from the MSE of its second matching.

% main --eigenfaces 10 50 50 nonsmiling_cropped/list.txt 50.face;
% main --constructuserbase 50.face nonsmiling_cropped/list.txt 50.user;
% main --recognizeface smiling_cropped/05.tga 50.user 50.face 5;
   Face 'smiling_cropped/05.tga' recognized as being closest too:
   0: nonsmiling_cropped/04; MSE: 140356
   1: nonsmiling_cropped/16; MSE: 203532
   2: nonsmiling_cropped/19; MSE: 214515
   3: nonsmiling_cropped/18; MSE: 261169
   4: nonsmiling_cropped/05; MSE: 278644
% main --projectface smiling_cropped/05.tga 50.face;

% main --recognizeface smiling_cropped/16.tga 50.user 50.face 5;
   Face 'smiling_cropped/06.tga' recognized as being closest too:
   0: nonsmiling_cropped/16; MSE: 22251.2
   1: nonsmiling_cropped/19; MSE: 58286.5
   2: nonsmiling_cropped/04; MSE: 314666
   3: nonsmiling_cropped/01; MSE: 338866
   4: nonsmiling_cropped/18; MSE: 396204
% main --projectface smiling_cropped/16.tga 50.face;

To enhence the performance, I used a color cue to find a proper area. The graph on the right shows red, green, and blue value of the mean of each twenty one sample faces. We can see 1) red value is much greater than green and blue values, and 2) the variance of each color is not huge. Therefore, I added a few rules in my face detection code. If the mean color of cropped image has 1) red < 1.2 x green, 2) red < 1.2 x blue, 3) green < blue, and 4) the mse of the color and the mean color from sample faces are greater than 100, then we reject the image. Actually it improves the performance by far especially when there are green or blue backgrounds. Also, I adapted the technique multiplying MSE by the distance from the image from the face mean and then dividing by the variance of the image.

There are four experiments in total. "elf.tga", "pope.tga", "group1.tga" and "group2.tga". We made a perfect result for "pope.tga" and "group1.tga", while it's not showing a good performance for the others.

Since the sample photos were taken in a certain environment, the program can only detect faces in such a condition. All the samples are taken at the same environment as of "group1.tga" - light is coming from above, creating a deep, dark shadow around people's eyes. Thus, most of the eigenfaces have significantly different tone around their eyes compared to other area. In such sense, "pope.tga" is a great example. Since his eye area is dark (even though the lights is coming in front of his face), it matches to our eigenfaces well (MSE 492.728). Meanwhile, "elf.tga" and "group2.tga" are in a different environment: the light is coming from in front of the faces. Especially, the baby doesn't have any shadow around eyes.

With a lot of textures, non-face image can be recognized as a face easily. Figure 5 shows the original image and reconstructed image for "elf.tga" example. The complicated texture of a round picture gives a similar tone as that of the reconstructed image.

% main --eigenfaces 6 25 25 nonsmiling_cropped/list.txt verif.face;
% main --findface elf.tga 25.face 0.45 0.55 0.01 mark 4 elf_res.tga;
% main --findface pope.tga 25.face 0.30 0.40 0.01 mark 1 pope_res.tga;
% main --findface group1.tga 25.face 0.45 0.55 0.01 mark 3 group1_res.tga;
% main --findface group2.tga 25.face 0.70 0.80 0.01 mark 4 group2_res.tga;

I added "--testverification" option on the main program. It takes userbase filename and eigenface filename as inputs, and calculates all MSE for each user-face match. Then we calculate false positive rate and false negative rate with threshold from 0 to 500,000 at a granularity of 1,000. I searched thresholds greedly. Since I do not call "--verifyface" methods for each threshold, the whole experiments takes less than ten seconds. In my point of view, I think threshold 121000 is good enough: it gives 0.2 of false positive rate and 0.190476 of false negative rate. The result is shown on the right graph.

% main --eigenfaces 6 25 25 nonsmiling_cropped/list.txt verif.face;
% main --constructuserbase verif.face nonsmiling_cropped/list.txt verif.user;
% main --testverification verif.user verif.face;

% main --eigenfaces 10 50 50 nonsmiling_cropped/list.txt 50.face;
% main --morphfaces nonsmiling_cropped/01.tga nonsmiling_cropped/05.tga 50.face 0.0 res_00.out;
% main --morphfaces nonsmiling_cropped/01.tga nonsmiling_cropped/05.tga 50.face 0.1 res_01.out;
% main --morphfaces nonsmiling_cropped/01.tga nonsmiling_cropped/05.tga 50.face 0.2 res_02.out;
...

Calculate eigenvalues() and eigenvectors () of . From the second equation, is eigenvector of . Since the rank of is same as the rank of , it returns a same result. However, is the square of the number of total image pixels, while is the square of the number of samples images, which saves much more time.

• Computer vision project 3: eigenfaces, URL http://www.cs.washington.edu/education/courses/576/08sp/projects/project3/project3.html. (URL accessed on May 10, 2008).
• D. Pissarenko. Eigenface-based facial recognition. URL http://dapissarenko.com/resources/2002_12_01_eigenfaces.pdf. (URL accessed on May 10, 2008).
• M. Turk and A. Pentland. Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1), 1991a. URL http://www.cs.ucsb.edu/~mturk/Papers/jcn.pdf. (URL accessed on May 10, 2008).
• M. A. Turk and A. P. Pentland. Face recognition using eigenfaces. In Proc. of Computer Vision and Pattern Recognition, pages 586591. IEEE, June 1991b. URL http://www.cs.wisc.edu/~dyer/cs540/handouts/mturk-CVPR91.pdf. (URL accessed on May 10, 2008).