title: Evaluation of jackknife and bootstrap for defining confidence intervals for pairwise agreement measures.
abstract: Several research fields frequently deal with the analysis of diverse classification results of the same entities. This should imply an objective detection of overlaps and divergences between the formed clusters. The congruence between classifications can be quantified by clustering agreement measures, including pairwise agreement measures. Several measures have been proposed and the importance of obtaining confidence intervals for the point estimate in the comparison of these measures has been highlighted. A broad range of methods can be used for the estimation of confidence intervals. However, evidence is lacking about what are the appropriate methods for the calculation of confidence intervals for most clustering agreement measures. Here we evaluate the resampling techniques of bootstrap and jackknife for the calculation of the confidence intervals for clustering agreement measures. Contrary to what has been shown for some statistics, simulations showed that the jackknife performs better than the bootstrap at accurately estimating confidence intervals for pairwise agreement measures, especially when the agreement between partitions is low. The coverage of the jackknife confidence interval is robust to changes in cluster number and cluster size distribution.
authors: Instituto de Microbiologia, Instituto de Medicina Molecular, Faculdade de Medicina de Lisboa, Lisboa, Portugal.
authors: Ana
authors: A
authors: Severiano
authors: João A
authors: JA
authors: Carriço
authors: D Ashley
authors: DA
authors: Robinson
authors: Mário
authors: M
authors: Ramirez
authors: Francisco R
authors: FR
authors: Pinto
references: BMC Bioinformatics. 2007 Feb 07;8:44
references: 17286861
references: J Clin Microbiol. 2006 Jul;44(7):2524-32
references: 16825375
references: Bioinformatics. 2006 Oct 1;22(19):2405-12
references: 16882653
references: Infect Genet Evol. 2011 Mar;11(2):446-53
references: 21172458
references: PLoS One. 2008;3(11):e3696
references: 19002246
references: Bioinformatics. 2005 Oct 15;21(20):3824-31
references: 16105900
references: J Clin Microbiol. 2010 May;48(5):1839-52
references: 20335411
references: J Clin Microbiol. 2008 Jan;46(1):136-44
references: 17989188
references: Biometrics. 1983 Mar;39(1):1-11
references: 6871338
references: BMC Microbiol. 2009 Jan 08;9:6
references: 19133124
references: J Proteome Res. 2005 Sep-Oct;4(5):1757-67
references: 16212430