Amir Akbarzadeh - Project 4, CSE 455 Winter 2003

Some results

The eigenfaces created from class images and the undergraduate images along with the average faces for these sets.

Notice that since I did the 2 bell credit (computing eigenfaces faster) I was enabled to compute very large eigenfaces very quickly.

Average face and eigenfaces created from class_nonsmiling_cropped (notice that I have removed the normalizing (after grayscale conversion) of the loaded images, since increasing the contrast for every image doesn't seem like a good idea, specially when we want to see if things are faces or not, normalizing will make the variance of e.g. "all shirt" areas larger which we don't want specially in findface, I changed image.subimage as well) anyways this makes my eigenfaces look a little bit different:

Average face and eigenfaces created from ugrads_cropped:

Some examples of image recognition attempts that failed (if there were any) and some that succeeded.

Using 12 150 by 150 eigenfaces built from class_nonsmiling_cropped paullarp was mistaken to be alissah:

      smiling paullarp            proj. smil. paullarp        proj. nonsmil. alissah         nonsmiling alissah         nonsmiling paullarp          proj. nonsmil. paullarp

Looking at the projected images of smiling paullarp and nonsmiling alissah they sure look a like, however projected nonsmiling paullarp doesn't look to different either, which explains why paullarp was the second best match (notice that above, in question/answer part, when smaller eigenfaces where used paullarp was the 3:rd best match of himself)

Your two images showing the results of face detection

Friends, min_scale = 0.55, max_scale = 0.6, step = 0.05, Famous, min_scale = 0.8, max_scale = 0.91, step = 0.05

        

Some results of successful and unsuccessful automatic image cropping attempts

        

Questions and Answers:

Testing recognition with cropped class images

Procedure

  1. Use the cropped, non-smiling students (in class_nonsmiling_cropped) to compute 7 eigenfaces.
  2. Use the same set of images to compute a userbase.
  3. Have the program recognize the cropped, smiling student images (in class_smiling_cropped) in the smiling userbase.

Questions

  1. How many faces did the program recognize correctly? Incorrectly?
  2. For instances where the program was wrong, what was the average position of the correct answer in the list of closest matches?
  3. If you recognize a female face, will the second, third and fourth matches usually be female? How about for male faces? Give supporting data.
  4. In a real-life scenario with thousands of users would you use the entire user set to compute the face space? Why or why not?
  5. Why might it be better to use a face set independent of the user set to compute the eigenfaces?

Answers

  1. 17 correct, 5 incorrect
  2. 2-3
  3. yes, females usually follow by females, equiv. for males, but not always:

    4.            Face 'class_smiling_cropped\rhennig.tga' recognized as being closest too:
               0: class_nonsmiling_cropped\mbixby; MSE: 142640
               1: class_nonsmiling_cropped\rhennig; MSE: 146351

               Face 'class_smiling_cropped\mhl.tga' recognized as being closest too:
               0: class_nonsmiling_cropped\melissa; MSE: 342233
               1: class_nonsmiling_cropped\mhl; MSE: 348104

               however we also have:

               Face 'class_smiling_cropped\paullarp.tga' recognized as being closest too:
               0: class_nonsmiling_cropped\alissah; MSE: 106214
               1: class_nonsmiling_cropped\ddewey; MSE: 167364
               2: class_nonsmiling_cropped\paullarp; MSE: 237571

  4. What would be good to do is that take users that are different, e.g. have different skin color, gender etc. In this way the spanned face space won't be biased towards a certain group of people
  5. If the eigenfaces are computed from the user set then it is a greater chance that when reconstructing faces that are not in the user set the face will end up looking like a user and being recognized as someone in the user set, this is bad if for example a company uses face recognition for security reasons, letting people in to the building etc.

Recognizing the undergraduate faces

Procedure

  1. Use the cropped undergraduate students (in ugrads_cropped) to compute 12 eigenfaces.
  2. Use the small set of class undergraduate images (in class_ugrads_cropped) to compute a userbase.
  3. Have the program recognize the cropped, non-smiling student images in this userbase.
  4. Repeat your experimentation with 5, 10, 15, and 20 eigenfaces. Remember to regenerate the userbase each time.
  5. Why is it best to use as few eigenfaces as possible while still getting good results?

Questions

  1. Of the students in the class set who are also in the class undergraduate set, how many did the program recognize correctly? Incorrectly?
  2. Are the incorrect identifications reasonable? Do they look similar to the actual person? Give some example images.
  3. Why does the program perform more poorly in this recognition task than the previous one? Give at least three reasons.
  4. How did changing the number of eigenfaces used change your results? What number worked best?

Answers

  1. 15 incorrect, 2 correct
  2. No they are not reasonable, using 12 100 by 100 pixels eigenfaces, I got most of the people being recognized as mbixby (and 3 other persons), this is a bit weird, looking at mbixby's picture the only thing special for him is that his face is very light.
  3. The faces used to construct the eigenfaces and the userbase where photographed under very different lighting conditions compared to the class_nonsmiling pictures. The face "rotations" where different comparing class_ugrads with class_nonsmiling. Some people wore glasses in one set and not in the other. Also the fact that the faces where at different sizes might also have an effect on the result.
  4. It hardly changed the result at all. For recognition using smaller number of eigenfaces is better. The reason is: when comparing two faces we are comparing the faces coefficients in face space, the difference between two faces lies mostly in the coefficients in front of the eigenfaces with highest eigenvalue, the coefficients in front of the eigenfaces with lower eigenvalues will not differ much between different faces, so bringing them in will only make the difference less, since we are dividing the difference between the coefficient vectors with the length of the coefficient vector.

Cropping the undergraduate faces

  1. Use the eigenfaces file computed in the previous problem
  2. Use your program to crop at least ten of the uncropped undergraduate images
  3. Experiment with min_scale, max_scale, and step parameters to find ones that work robustly and accurately without taking more than 10 seconds or so to run on each image

Questions

  1. What min_scale, max_scale, and scale step did you end up using?
  2. How many of your crop results look correct (cropped to the same part of the face as the pre-cropped images)? How many look incorrect?
  3. What is the problem with using a min_scale that is too small?

Answers

  1. min_scale = 0.18, max_scale = 0.20, step = 0.01
  2. 6 correct, 4 incorrect
  3. The details building up the face are lost

Finding faces in a group photo

Procedure

  1. Find two group photos, one of family or friends and one from the web of TV or movie characters, or famous people. Each image should have at least four faces.
  2. Use your program to find the faces in the photos (use the crop=false option)

Questions

  1. Show the results of the face detections and what min_scale, max_scale, and scale_step did you use for each image?
  2. If there are any errors, explain why the program might have failed and how you could improve the input or the algorithm to correct this.

Answers

  1. Friends, min_scale = 0.55, max_scale = 0.6, step = 0.05, Famous, min_scale = 0.8, max_scale = 0.91, step = 0.05

    2.                   

  2. Since one face is very fuzzy the algorithm chooses the second best thing, being an all same color area, the algorithm has already been adjusted to handle this error, however since we force it to find 5 faces, the all same color area is still better than the other things.