title: External Validation of Prediction Models for Pneumonia in Primary Care Patients with Lower Respiratory Tract Infection: An Individual Patient Data Meta-Analysis.
abstract: BACKGROUND
  Pneumonia remains difficult to diagnose in primary care. Prediction models based on signs and symptoms (S&S) serve to minimize the diagnostic uncertainty. External validation of these models is essential before implementation into routine practice. In this study all published S&S models for prediction of pneumonia in primary care were externally validated in the individual patient data (IPD) of previously performed diagnostic studies.
  
  
  RESULTS
  S&S models for diagnosing pneumonia in adults presenting to primary care with lower respiratory tract infection and IPD for validation were identified through a systematical search. Six prediction models and IPD of eight diagnostic studies (N total = 5308, prevalence pneumonia 12%) were included. Models were assessed on discrimination and calibration. Discrimination was measured using the pooled Area Under the Curve (AUC) and delta AUC, representing the performance of an individual model relative to the average dataset performance. Prediction models by van Vugt et al. and Heckerling et al. demonstrated the highest pooled AUC of 0.79 (95% CI 0.74-0.85) and 0.72 (0.68-0.76), respectively. Other models by Diehr et al., Singal et al., Melbye et al., and Hopstaken et al. demonstrated pooled AUCs of 0.65 (0.61-0.68), 0.64 (0.61-0.67), 0.56 (0.49-0.63) and 0.53 (0.5-0.56), respectively. A similar ranking was present based on the delta AUCs of the models. Calibration demonstrated close agreement of observed and predicted probabilities in the models by van Vugt et al. and Singal et al., other models lacked such correspondence. The absence of predictors in the IPD on dataset level hampered a systematical comparison of model performance and could be a limitation to the study.
  
  
  CONCLUSIONS
  The model by van Vugt et al. demonstrated the highest discriminative accuracy coupled with reasonable to good calibration across the IPD of different study populations. This model is therefore the main candidate for primary care use.
authors.affiliation: Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands.
authors.forename: Alwin
authors.initials: A
authors.lastname: Schierenberg
authors.affiliation: Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands.
authors.forename: Margaretha C
authors.initials: MC
authors.lastname: Minnaard
authors.affiliation: Saltro Diagnostic Center for Primary Care, Utrecht, the Netherlands.
authors.forename: Rogier M
authors.initials: RM
authors.lastname: Hopstaken
authors.affiliation: Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands.
authors.forename: Alma C
authors.initials: AC
authors.lastname: van de Pol
authors.affiliation: Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands.
authors.forename: Berna D L
authors.initials: BD
authors.lastname: Broekhuizen
authors.affiliation: Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands.
authors.forename: Niek J
authors.initials: NJ
authors.lastname: de Wit
authors.affiliation: Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands.
authors.forename: Johannes B
authors.initials: JB
authors.lastname: Reitsma
authors.affiliation: Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands.
authors.forename: Saskia F
authors.initials: SF
authors.lastname: van Vugt
authors.affiliation: Department of Public Health and Primary Care, Leiden University Medical Center, Leiden, the Netherlands.
authors.forename: Aleida W
authors.initials: AW
authors.lastname: Graffelman
authors.affiliation: Department of Community Medicine, University of Tromsø, Tromsø, Norway.
authors.forename: Hasse
authors.initials: H
authors.lastname: Melbye
authors.affiliation: Chinese University of Hong Kong, Hong Kong, China.|Institute of Molecular and Experimental Medicine, Cardiff University, Cardiff, United Kingdom.
authors.forename: Timothy H
authors.initials: TH
authors.lastname: Rainer
authors.affiliation: Horten Centre for Patient Oriented Research and Knowledge Transfer, University Zurich, Zurich, Switzerland.
authors.forename: Johann
authors.initials: J
authors.lastname: Steurer
authors.affiliation: Department of Infectious Diseases, Odense University Hospital, Odense, Denmark.
authors.forename: Anette
authors.initials: A
authors.lastname: Holm
authors.affiliation: Division of General Internal Medicine, University of California San Francisco, San Francisco, United States of America.
authors.forename: Ralph
authors.initials: R
authors.lastname: Gonzales
authors.affiliation: Department of Family Medicine, CAPHRI School for Public Health and Primary Care, Maastricht University Medical Centre, Maastricht, The Netherlands.
authors.forename: Geert-Jan
authors.initials: GJ
authors.lastname: Dinant
authors.affiliation: Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands.
authors.forename: Joris A H
authors.initials: JA
authors.lastname: de Groot
authors.affiliation: Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands.
authors.forename: Theo J M
authors.initials: TJ
authors.lastname: Verheij
mesh_terms: D019540:Area Under Curve; D002138:Calibration; D019992:Databases as Topic; D006801:Humans; D008954:Models, Biological; D011014:Pneumonia / Q000175:diagnosis*; D011320:Primary Health Care; D011336:Probability; D012372:ROC Curve; D015203:Reproducibility of Results