title: Evaluation of Face Datasets as Tools for Assessing the Performance of Face Recognition Methods.
abstract: Face datasets are considered a primary tool for evaluating the efficacy of face recognition methods. Here we show that in many of the commonly used face datasets, face images can be recognized accurately at a rate significantly higher than random even when no face, hair or clothes features appear in the image. The experiments were done by cutting a small background area from each face image, so that each face dataset provided a new image dataset which included only seemingly blank images. Then, an image classification method was used in order to check the classification accuracy. Experimental results show that the classification accuracy ranged between 13.5% (color FERET) to 99% (YaleB). These results indicate that the performance of face recognition methods measured using face image datasets may be biased. Compilable source code used for this experiment is freely available for download via the internet.
authors.affiliation: Laboratory of Genetics, National Institute on Aging, National Institutes of Health, 333 Cassell Dr., Baltimore, MD 21224, Tel: 410-558-8682 , Email: shamirl@mail.nih.gov.
authors.forename: Lior
authors.initials: L
authors.lastname: Shamir
references.citation: Genome Biol. 2005;6(5):R47
references.pmid: 15892875
references.citation: Science. 2003 Apr 4;300(5616):100-2
references.pmid: 12677061
references.citation: Anal Cell Pathol. 2003;25(1):1-36
references.pmid: 12590175
references.citation: IEEE Trans Image Process. 2002;11(10):1160-7
references.pmid: 18249688
references.citation: PLoS Comput Biol. 2008 Jan;4(1):e27
references.pmid: 18225950
references.citation: Pattern Recognit Lett. 2008 Jan;29(11):1684-1693
references.pmid: 18958301
references.citation: IEEE Trans Pattern Anal Mach Intell. 2006 Apr;28(4):594-611
references.pmid: 16566508