Project 3 Report: Eigenfaces

CSE576, Spring 2008

Xiaoyu Chen

May 12, 2008

In this project, I developed a system to recognize faces of a person, and to find and crop faces from photos.

Recognize cropped, smiling faces

Methods

Each face can be defined as a vector of pixels. Principal Component Analysis (PCA) can be used to find the face space, which is spanned by only a few vectors (eigenfaces). A face can then be represented by a vector of coefficients weighting the eigenfaces.

The vector of coefficients of a user’s face can be computed by projecting the face onto the face space. We can then store such vector of coefficients into a user base to represent the face of each user.

Given a query face Q, we first compute the vector of coefficients for it. We then compute the MSE between its coefficients and the coefficients of each user’s face in the user base. The face in the user based of the least MSE will be the best match, and the user of that face will be recognized as the user of face Q.

Results

Firstly, I computed ten 25x25 eigenfaces from the cropped, nonsmiling faces. The average face and the ten eigenfaces are shown below.

average eigen_0 eigen_1 eigen_2 eigen_3 eigen_4 eigen_5 eigen_6 eigen_7 eigen_8 eigen_9

Secondly, I contructed a user base for the cropped, nonsmiling faces based on the ten eigenfaces. Finally, using the ten eigenfaces and the user base, I applied the program to recognize the cropped, smiling faces of the class. The recognition accuracy is 43% (9/12). The correctly recognized smiling faces include 03, 04, 06, 10, 12, 16, 19, 20, and 21.

To evaluate the effect of the number of used eigenfaces in recognizing faces, I varied the number of eigenfaces from 1 to 21 with a step size of 2, and repeated the above process. The followed plot shows the number of correctly recognized faces v.s. the number of eigenfaces used.

Discussion

The above plot shows: the more the used eigenfaces, the more the correctly recognized faces. When the number of eigenfaces increase from 1 to 7, the number of correctly recognized faces increases 80% (4/5). It then remains the same number (9) until the number of used eigenfaces arrives 15. After that, it increases 44% (4/9) while the number of eigenfaces increases up to 21. However, the increased number of eigenfaces will lead to the increased computational cost. Therefore, there is a trade off between recognition accuracy and computational cost (i.e. the number of eigenfaces). When computational cost is a concern, we may choose 7 eigenfaces to minimize the computational cost and achieve a reasonable recognition accuracy. When the computational cost is not a concern, we may choose 21 eigengaces to maximize the recognition accuracy.

There are some recognition errors. For example, when using 10 eigenfaces, smiling face 02 02_smiling was recognized as face 06 06_nonsmiling. These two faces are not very similar to each other by eyes, except for the orientation. However, the correct answer ranks #2 in the sorted results. Another example is that smiling face 02 07_smiling was recognized as face 04 04_nonsmiling (when using 10 eigenfaces). These two faces are actually very similar to each other. The correct answer ranks #6 in the sorted results.

Crop and find faces

Methods

Given an image, the system searches the image and return the best n faces (n is specified by users). To determine whether an area is a face, the area is first projected onto the face space. The projection and the original area are then used to compute MSE. If the MSE is lower than the given threshold, then the area will be detected as a face. To make the face detection scale invariant, the program actually searches various scales specified by the input parameters.

To keep the algorithm from reporting low-texture areas or areas close to face space but far from the average face, the computed MSE of an area is multiplied by the distance of the area from the average face and then divided by the standard deviation of the area. This basically penalizes areas far from the average face or having low variance (i.e. low-texture).

According to the input parameters, the program either crops the best face or draws green lines in the original image to highlight the detected best n faces.

Results

First, I used the program to crop the elf.tga image. The parameters used were: min_scale=0.45, max_scale=0.55, and scale_step=0.01.

elf

The cropped image is elf_cropped

Second, I used the program to crop my image xyc.tga. The parameters used were: min_scale=0.45, max_scale=0.55, and scale_step=0.01.

xyc

The cropped image is xyc_cropped

Next, I used the program to find faces in the group1.tga image. The parameters used were: min_scale=0.45, max_scale=0.55, and scale_step=0.01. Green boxes were drawn to highlight the found faces.

group1_mark.

At last, I used the program to find faces in a group image myGroup.tga downloaded from the Internet. The parameters used were: min_scale=0.23, max_scale=0.30, and scale_step=0.01. Green boxes were drawn to highlight the found faces.

myGroup_mark.

Discussion

There are both false positives and false negatives in face detection. 1) In my observations, background with a certain variance can sometimes fool the algorithm. For example, the right-most face detected in the last group photo contains half background and half face. 2) Although I modified MSE (as described above) to avoid detecting low-texture areas, the program sometimes reported areas on clothes as faces. In such cases, color cues may be helpful. 3) The orientation of a face is also important. Because the eigenfaces were learned from faces facing straight forward, it is reasonable that the program may fail to detect faces with some orientation.

Verify faces (extra credit)

Methods

Given a face, a user, and a vector of coefficients computed from another face of the user, the program determines whether the face is the user’s. We first compute the coefficients of the given face by projecting it onto the face space. Then, we compute the MSE between the two vectors of coefficients. If the MSE is lower than the given threshold, the program will report that the given face is the user’s.

Results

Firstly, I computed six 25x25 eigenfaces from the cropped, nonsmiling faces. Secondly, I contructed a user base for the cropped, nonsmiling faces based on the six eigenfaces. Finally, using the six eigenfaces and the user base, I applied the program to verify each user against his smiling face, as well as each user against a smiling face of his neighbor (e.g. 01 being 02’s neighbor, 02 being 03’s neighbor, etc.).

The MSE thresholds I tried include 1e+5, 1.3e+5, 1.5e+5, 1.8e+5, and 2e+5. Among them, 1.5e+5 worked best. Using 1.5e+5, the false negative rate is 9.5%(2/21), and the false positive rate is 23.8%(5/21). I basically used a strategy of binary search, trying to minimize both the false positive rate and the false negative rate.