title: Summarising and validating test accuracy results across multiple studies for use in clinical practice.
abstract: Following a meta-analysis of test accuracy studies, the translation of summary results into clinical practice is potentially problematic. The sensitivity, specificity and positive (PPV) and negative (NPV) predictive values of a test may differ substantially from the average meta-analysis findings, because of heterogeneity. Clinicians thus need more guidance: given the meta-analysis, is a test likely to be useful in new populations, and if so, how should test results inform the probability of existing disease (for a diagnostic test) or future adverse outcome (for a prognostic test)? We propose ways to address this. Firstly, following a meta-analysis, we suggest deriving prediction intervals and probability statements about the potential accuracy of a test in a new population. Secondly, we suggest strategies on how clinicians should derive post-test probabilities (PPV and NPV) in a new population based on existing meta-analysis results and propose a cross-validation approach for examining and comparing their calibration performance. Application is made to two clinical examples. In the first example, the joint probability that both sensitivity and specificity will be >80% in a new population is just 0.19, because of a low sensitivity. However, the summary PPV of 0.97 is high and calibrates well in new populations, with a probability of 0.78 that the true PPV will be at least 0.95. In the second example, post-test probabilities calibrate better when tailored to the prevalence in the new population, with cross-validation revealing a probability of 0.97 that the observed NPV will be within 10% of the predicted NPV.
authors: Research Institute of Primary Care and Health Sciences, Keele University, Staffordshire, ST5 5BG, U.K.
authors: Richard D
authors: RD
authors: Riley
authors: Ikhlaaq
authors: I
authors: Ahmed
authors: Thomas P A
authors: TP
authors: Debray
authors: Brian H
authors: BH
authors: Willis
authors: J Pieter
authors: JP
authors: Noordzij
authors: Julian P T
authors: JP
authors: Higgins
authors: Jonathan J
authors: JJ
authors: Deeks
mesh_terms: D001499:Bayes Theorem; D002138:Calibration / Q000592:standards; D015331:Cohort Studies; D005069:Evaluation Studies as Topic; D005334:Fever / Q000145:classification / Q000175:diagnosis; D006801:Humans; D006996:Hypocalcemia / Q000175:diagnosis; D015201:Meta-Analysis as Topic; D011237:Predictive Value of Tests; D011336:Probability; D011379:Prognosis; D012015:Reference Standards; D015203:Reproducibility of Results; D012107:Research Design; D012680:Sensitivity and Specificity; D013821:Thermometers / Q000592:standards
keywords: calibration; diagnostic; discrimination; meta-analysis; prognostic; test accuracy