CSEP576 Project 4

EIGENFACES

Recognition with cropped class images

average_face:

eigenfaces:

Recognizing smiling students:

Matches of smiling students to nonsmiling students in userbase:

#eig:	2	4	6	8	10	12	14	16	18	20	22	24	26	28	30	32
s00	05	05	02	02	02	02	02	02	02	02	02	02	02	02	02	02
s01	14	25	25	25	25	25	25	28	28	28	28	28	28	28	28	28
s02	03	07	02	02	02	02	02	02	02	02	02	02	02	02	02	02
s03	23	15	15	19	19	19	19	19	19	19	19	19	19	19	19	19
s04	06	06	06	04	04	04	04	04	04	04	04	04	04	04	04	04
s05	27	18	18	18	11	11	11	11	11	11	11	11	11	11	11	11
s06	05	05	05	29	29	29	29	29	29	29	29	29	29	29	29	29
s07	15	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19
s08	26	09	08	08	08	08	08	08	08	08	08	08	08	08	08	08
s09	20	09	09	09	09	09	09	09	09	09	09	09	09	09	09	09
s10	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19
s11	27	24	18	11	11	11	11	11	11	11	11	11	11	11	11	11
s12	22	12	12	12	12	12	12	12	12	12	12	12	12	12	12	12
s13	17	17	17	11	18	11	18	18	18	18	18	18	18	18	18	18
s14	11	13	13	13	13	14	14	13	13	13	14	14	14	14	14	14
s15	22	15	15	15	15	15	18	19	19	18	19	19	19	18	18	11
s16	16	16	16	16	16	16	16	16	16	16	16	16	16	16	16	16
s17	11	17	17	17	17	17	17	17	17	17	17	17	17	17	17	17
s18	18	18	18	18	18	18	18	18	18	18	18	18	18	18	18	18
s19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19	19
s20	20	20	20	20	20	20	20	20	20	20	20	20	20	20	20	20
s21	14	22	22	22	22	22	22	22	22	22	22	22	22	22	22	22
s22	22	22	22	22	22	22	22	22	22	22	22	22	22	22	22	22
s23	31	14	14	14	14	14	14	14	14	14	14	14	14	14	14	14
s24	11	24	24	24	24	24	24	24	24	24	24	24	24	24	24	24
s25	11	25	25	25	25	25	25	25	25	25	25	25	25	25	25	25
s26	29	26	26	26	26	26	26	26	26	26	26	26	26	26	26	26
s27	14	24	29	29	29	29	29	29	29	29	29	29	29	29	29	29
s28	28	28	28	28	28	28	28	28	28	28	28	28	28	28	28	28
s29	29	29	29	29	29	29	29	29	29	29	29	29	29	29	29	29
s30	30	30	30	30	30	30	30	30	30	30	30	30	30	30	30	30
s31	31	31	31	31	31	31	31	31	31	31	31	31	31	31	31	31

#correct:	9	16	18	20	20	21	20	19	19	19	20	20	20	20	20	20

Number of correct recognition per eigenfaces used:

Trends: Accuracy increases significantly as number of eigenfaces increases when there were only a few eigenfaces. It plateaus at around 8-12 eigenfaces with no increase thereafter. In fact we could see a slight decrease for a little while. With +/- 1 error, the curve plateaus at 20 correct recognitions.

Tradeoffs: speed vs. accuracy.

Number of eigenfaces to use: 8 for best speed-accuracy. Going up to 12 may give a slightly better accuracy, or may be not.

Looking at results of 12 eigenfaces used:
Correct matches: 21/32 (65.6%)

Incorrect matches where correct match came close: 3/32 (0.09%)

s07 matched to 19 instead of 07 (ranked 3rd)
amount of face cropped and angel of face might have affected the result.

s10 matched to 19 instead of 10 (ranked 2nd)

angle of face might have affected the result.
wide-opened eyes might have been matched to the glasses.

s23 matched to 14 instead of 23 (ranked 2nd)

angle of face might have affected the result

Other incorrect matches: 8/32 (0.25%)

s00 matched to 02 instead of 00

raised eyebrows affected the distance between eyes and brows.
opened mouth might resemble moustache.

s01 matched to 25 instead of 01

probably none of the faces will be a good match for this - matched to least bad match.

s03 matched to 19 instead of 03

angle of face and amount of face cropped might have affected the result.

s05 matched to 11 instead of 05

angle of face and amount of face cropped might have affected the result.

s06 matched to 29 instead of 06

probably another no match - matched to least bad match.

s13 matched to 11 instead of 13

angle of face and amout of face cropped might have affected the result.
wide-opened eyes might have been matched to the glasses.

s21 matched to 22 instead of 21

significant tilted angle of face probably could not be match - another least bad match.

s27 matched to 29 instead of 27

could not see how 29 is a better match than 27.
maybe it's the reflection from the glasses?

Cropping and finding faces

Bush_george_w_portrait cropped:

(min_scale: 0.25 max_scale: 0.55 step: 0.01 used as instructed)
Best MSE: 426.931 at scale of 0.53

My portrait cropped:

(min_scale: 0.48 max_scale: 0.52 step: 0.01)
Best MSE: 597.052 at scale of 0.5

I estimated the ratio of the width of the eigenfaces to the width of the face in the image to be about 1:2. Therefore I test run with the scale of 0.48 to 0.52 in steps of 0.02. Best MSE returned was at 0.5. So, I ran the test again with 0.48 to 0.52 in steps of 0.01. Result was the same.
Note: I could not find a face in the portrait at first (false negative) using my original program which had the max mse threshold set at 500. This threshold worked fine for the pictures taken in class (where faces are very similar to those used to build the eigenfaces). I adjusted the threshold to 700 and the program successfully recognized and cropped the face.

Img_8270 with 4 faces recognized:

(min_scale: 0.4 max_scale: 0.6 step: 0.01)
MSE of faces (from left): 305.251 (scale: 0.55 ), 148.995 (scale: 0.54), 190.074 (scale: 0.49), 75.6613 (scale: 0.55)

After some rough manual calculation for the ratio of the face sizes in the image vs. eigen size, I decided to use min_scale = 0.4, max_scale = 0.6 and step 0.05 to get a rough estimate of the range with best recognitions. Turned out faces were recognized almost precisely with one of the faces at scale of 0.5 and the other three at 0.55. (In order to speed up execution, I only scanned rows of the image that span the four faces.) After determing that the range is good, I ran the program again, this time scaning the full image with the estimated range of 0.4 to 0.6 in steps of 0.01.

Group picture with n faces recognized:

(min_scale: 0.6 max_scale: 0.8 step: 0.05)

I estimated the ratio and set my experiment at 0.6 to 0.8 stepping 0.05 for a quick evaluation of the range. Turned out the program recognized 8 out of the 9 faces. Since the non-face match has the highest MSE in the 9 matches, I thought I will use finer steps to see if a good match can be found for the missed face to bump out the false positive match. The size of the face tells me that the range of the scale should be good. But I still widen the range anyway. I ran the test a second time using 0.55 to 0.95 in steps of 0.01.

(min_scale: 0.55 max_scale: 0.95 step: 0.01)

Out of my expectation, the results were not as good as my first run. Not only did it still miss the same person. The hit on the third person from the right was replaced by a half-face. The original MSE for this face was 479.25 at a scale of 0.67. The MSE for the half-face was 463.336 at a scale of 0.94. The result showed that widening the scale unnecessary may not be a good thing. Not only does it increase the time of execution unnecessarily, the extra scale levels used increases the probability of false results without adding any value for increased accuracy (none of the face falls in that scale range).

Since the missed face might have been slightly bigger than I first thought (maybe not), using a smaller min_scale might have helped. Instead of unnecessarily increasing the max_scale, maybe I should have tried a smaller min_scale, perhaps running it from 0.5 to 0.8. There is a slight change that a low enought MSE for the face can be found to bump out the false positive.

Another thing I also wanted to try was to run the program to look for 10 faces instead of 9 to see if the missed face will be found. I think most likely it will. Because the error most likely did not lie in the missed face, but in the false positive non-face that somehow got a very low MSE. However, because it will take another long period of time to run, I abandoned the effort.

After analysing this false positive and the half-face of scale 0.95, I came to the conclusion that this has to do with the variance of the patch. Since the MSE was weighted with a cost that is inversely proportional to the variance of the image, both the false positives here looked like they are a side-effect of that weight. Both the false positives look like they contain half light colored pixels and half dark colored pixels which resulted in their image variance being high. That in turn caused the MSE to become small.

Perhaps my time will be more well spent trying to find ways to adjust the weight than to run the program in such amount of scale levels which was really time consuming.

False positives/negatives:
Initial false positives of low textured area in Img_8270 were handled by the approach given in the assignment-
MSE = MSE*(Distance of face from face mean)/variance of face. This causes low variance (low textured) area to have higher MSE. It also rejects images close to face space but far from facial mean.

However, a side effect of that could be seen in the false positives in my group picture as discussed above.

A false negative was found initially when cropping myportrait (no face found). Turned out that it was because max mse threshold was set too low. Adjusting the threshold (from 500 to 700) corrected the result.

VerifyFace

By looking at my output from Testing Recoginition with 6 eigenfaces earlier, I estimated a starting MSE threshold for experiment. I decided to start with 1300 and worked my way up in steps of 1000 or 2000..

Here is the results I got:

threshold	1300	1400	1500	1600	1700	1800	2000	2100	2200	2400	2600
positives	15	17	17	18	18	19	20	22	22	23	25
false-ves	17	15	15	14	14	13	12	10	10	9	7
negatives	29	27	27	26	26	25	25	23	23	20	20
false+ves	3	5	5	6	6	7	7	9	9	12	12
total falses	20	20	20	20	20	20	19	19	19	21	19

The positives are correct results from testing each student in the database against his or her own smiling face. The false negatives are incorrect results from this set of tests.
The negatives are correct results from testing each student in the database against someone else's smiling face (previus person on list, eg. 00 - s31, 01 - s00 ...). The false positives are incorrect results from this set of tests.

From my test results, it appears that the total accuracy for each threshold are about the same (19-21). However, MSE thresholds of 2100-2200 are the best. Besides having the least false rates, they also produce a balance number of false negatives and false positives. (10 false negatives and 9 false positives.)

In contrast, an MSE threshold of 1300 is not a good choice. Because of an imbalance in false positives and false negatives, when recognizing students against their own smiling faces, it rejected more than half of the faces.

Extra Credit implemented:

Speedup

VerifyFace

Note:
My program run extremely slow doing findFace().