title: Meta-analysis of test accuracy studies: an exploratory method for investigating the impact of missing thresholds.
abstract: BACKGROUND
  Primary studies examining the accuracy of a continuous test evaluate its sensitivity and specificity at one or more thresholds. Meta-analysts then usually perform a separate meta-analysis for each threshold. However, the number of studies available for each threshold is often very different, as primary studies are inconsistent in the thresholds reported. Furthermore, of concern is selective reporting bias, because primary studies may be less likely to report a threshold when it gives low sensitivity and/or specificity estimates. This may lead to biased meta-analysis results. We developed an exploratory method to examine the potential impact of missing thresholds on conclusions from a test accuracy meta-analysis.
  
  
  METHODS
  Our method identifies studies that contain missing thresholds bounded between a pair of higher and lower thresholds for which results are available. The bounded missing threshold results (two-by-two tables) are then imputed, by assuming a linear relationship between threshold value and each of logit-sensitivity and logit-specificity. The imputed results are then added to the meta-analysis, to ascertain if original conclusions are robust. The method is evaluated through simulation, and application made to 13 studies evaluating protein:creatinine ratio (PCR) for detecting proteinuria in pregnancy with 23 different thresholds, ranging from one to seven per study.
  
  
  RESULTS
  The simulation shows the imputation method leads to meta-analysis estimates with smaller mean-square error. In the PCR application, it provides 50 additional results for meta-analysis and their inclusion produces lower test accuracy results than originally identified. For example, at a PCR threshold of 0.16, the summary specificity is 0.80 when using the original data, but 0.66 when also including the imputed data. At a PCR threshold of 0.25, the summary sensitivity is reduced from 0.95 to 0.85 when additionally including the imputed data.
  
  
  CONCLUSIONS
  The imputation method is a practical tool for researchers (often non-statisticians) to explore the potential impact of missing threshold results on their meta-analysis conclusions. Software is available to implement the method. In the PCR example, it revealed threshold results are vulnerable to the missing data, and so stimulates the need for advanced statistical models or, preferably, individual patient data from primary studies.
authors: Research Institute of Primary Care and Health Sciences, Keele University, Staffordshire, ST5 5BG UK.
authors: Richard D
authors: RD
authors: Riley
authors: MRC Hub for Trials Methodology Research, School of Health and Population Sciences, University of Birmingham, Public Health Building, Edgbaston, Birmingham, B15 2TT UK.
authors: Ikhlaaq
authors: I
authors: Ahmed
authors: School of Health and Population Sciences, University of Birmingham, Public Health Building, Edgbaston, Birmingham, B15 2TT UK.
authors: Joie
authors: J
authors: Ensor
authors: School of Health and Population Sciences, University of Birmingham, Public Health Building, Edgbaston, Birmingham, B15 2TT UK.
authors: Yemisi
authors: Y
authors: Takwoingi
authors: School of Health and Population Sciences, University of Birmingham, Public Health Building, Edgbaston, Birmingham, B15 2TT UK.
authors: Amanda
authors: A
authors: Kirkham
authors: Research Section of Reproduction, Genes and Development; School of Clinical and Experimental Medicine, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK ; Fetal Medicine Centre, Birmingham Women's Hospital NHS Foundation Trust, Birmingham, UK.
authors: R Katie
authors: RK
authors: Morris
authors: Department of Otolaryngology - Head & Neck Surgery, Boston Medical Center, Boston University - School of Medicine, Boston, MA USA.
authors: J Pieter
authors: JP
authors: Noordzij
authors: School of Health and Population Sciences, University of Birmingham, Public Health Building, Edgbaston, Birmingham, B15 2TT UK.
authors: Jonathan J
authors: JJ
authors: Deeks
mesh_terms: D015982:Bias; D019411:Clinical Laboratory Techniques / Q000592:standards*; D003627:Data Interpretation, Statistical; D006801:Humans; D015233:Models, Statistical; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity
keywords: Diagnostic test; Imputation; Meta-analysis; Missing data; Multiple thresholds; Sensitivity analysis
references: Biometrics. 2000 Jun;56(2):455-63
references: 10877304
references: Med Decis Making. 2000 Oct-Dec;20(4):430-9
references: 11059476
references: Stat Med. 2001 Oct 15;20(19):2865-84
references: 11568945
references: Hypertens Pregnancy. 2001;20(1):IX-XIV
references: 12044323
references: Biometrics. 2003 Dec;59(4):936-46
references: 14969472
references: Stat Med. 2004 Jul 30;23(14):2155-72
references: 15236422
references: Br J Obstet Gynaecol. 1987 Apr;94(4):312-7
references: 3580313
references: Am J Obstet Gynecol. 1988 Jan;158(1):80-3
references: 2962500
references: Med Decis Making. 1988 Jul-Sep;8(3):204-15
references: 3294553
references: Ann Intern Med. 1994 Apr 15;120(8):667-76
references: 8135452
references: J Clin Epidemiol. 2005 Oct;58(10):982-90
references: 16168343
references: Lancet. 2006 Apr 1;367(9516):1066-74
references: 16581405
references: J Clin Epidemiol. 2006 Dec;59(12):1331-2; author reply 1332-3
references: 17098577
references: BMC Med Res Methodol. 2007;7:3
references: 17222330
references: Biostatistics. 2007 Apr;8(2):239-51
references: 16698768
references: Acad Radiol. 2007 Aug;14(8):974-84
references: 17659244
references: Stat Med. 2007 Nov 10;26(25):4544-62
references: 17476644
references: J Am Coll Surg. 2007 Dec;205(6):748-54
references: 18035257
references: Clin Chem. 2008 Apr;54(4):729-37
references: 18258670
references: BMJ. 2009;338:b2393
references: 19564179
references: Health Technol Assess. 2009 Jul;13(36):1-145, 147-230
references: 19624978
references: BMJ. 2009;339:b2981
references: 19666685
references: J Clin Epidemiol. 2009 Dec;62(12):1292-300
references: 19447007
references: BMC Med Res Methodol. 2009;9:73
references: 19903336
references: Biom J. 2010 Feb;52(1):95-110
references: 19924701
references: BMJ. 2012;345:e4342
references: 22777026