title: Sampling variability in forensic likelihood-ratio computation: A simulation study.
abstract: Recently, in the forensic biometric community, there is a growing interest to compute a metric called "likelihood-ratio" when a pair of biometric specimens is compared using a biometric recognition system. Generally, a biometric recognition system outputs a score and therefore a likelihood-ratio computation method is used to convert the score to a likelihood-ratio. The likelihood-ratio is the probability of the score given the hypothesis of the prosecution, Hp (the two biometric specimens arose from a same source), divided by the probability of the score given the hypothesis of the defense, Hd (the two biometric specimens arose from different sources). Given a set of training scores under Hp and a set of training scores under Hd, several methods exist to convert a score to a likelihood-ratio. In this work, we focus on the issue of sampling variability in the training sets and carry out a detailed empirical study to quantify its effect on commonly proposed likelihood-ratio computation methods. We study the effect of the sampling variability varying: 1) the shapes of the probability density functions which model the distributions of scores in the two training sets; 2) the sizes of the training sets and 3) the score for which a likelihood-ratio is computed. For this purpose, we introduce a simulation framework which can be used to study several properties of a likelihood-ratio computation method and to quantify the effect of sampling variability in the likelihood-ratio computation. It is empirically shown that the sampling variability can be considerable, particularly when the training sets are small. Furthermore, a given method of likelihood-ratio computation can behave very differently for different shapes of the probability density functions of the scores in the training sets and different scores for which likelihood-ratios are computed.
authors.affiliation: Department of Electrical Engineering, Mathematics and Computer Science, University of Twente, 7500 AE, Enschede, The Netherlands. Electronic address: T.Ali@utwente.nl.
authors.forename: Tauseef
authors.initials: T
authors.lastname: Ali
authors.affiliation: Department of Electrical Engineering, Mathematics and Computer Science, University of Twente, 7500 AE, Enschede, The Netherlands. Electronic address: L.J.Spreeuwers@utwente.nl.
authors.forename: Luuk
authors.initials: L
authors.lastname: Spreeuwers
authors.affiliation: Department of Electrical Engineering, Mathematics and Computer Science, University of Twente, 7500 AE, Enschede, The Netherlands. Electronic address: R.N.J.Veldhuis@utwente.nl.
authors.forename: Raymond
authors.initials: R
authors.lastname: Veldhuis
authors.affiliation: Netherlands Forensic Institute, Laan van Ypenburg 6, 2497 GB, The Hague, The Netherlands. Electronic address: d.meuwly@nfi.minvenj.nl.
authors.forename: Didier
authors.initials: D
authors.lastname: Meuwly
mesh_terms: D044707:Forensic Sciences; D006801:Humans; D016013:Likelihood Functions
keywords: Biometric recognition; Forensics; Likelihood-ratio; Sampling variability; Score