title: Variable performance of models for predicting methicillin-resistant Staphylococcus aureus carriage in European surgical wards.
abstract: BACKGROUND
  Predictive models to identify unknown methicillin-resistant Staphylococcus aureus (MRSA) carriage on admission may optimise targeted MRSA screening and efficient use of resources. However, common approaches to model selection can result in overconfident estimates and poor predictive performance. We aimed to compare the performance of various models to predict previously unknown MRSA carriage on admission to surgical wards.
  
  
  METHODS
  The study analysed data collected during a prospective cohort study which enrolled consecutive adult patients admitted to 13 surgical wards in 4 European hospitals. The participating hospitals were located in Athens (Greece), Barcelona (Spain), Cremona (Italy) and Paris (France). Universal admission MRSA screening was performed in the surgical wards. Data regarding demographic characteristics and potential risk factors for MRSA carriage were prospectively collected during the study period. Four logistic regression models were used to predict probabilities of unknown MRSA carriage using risk factor data: "Stepwise" (variables selected by backward elimination); "Best BMA" (model with highest posterior probability using Bayesian model averaging which accounts for uncertainty in model choice); "BMA" (average of all models selected with BMA); and "Simple" (model including variables selected >50% of the time by both Stepwise and BMA approaches applied to repeated random sub-samples of 50% of the data). To assess model performance, cross-validation against data not used for model fitting was conducted and net reclassification improvement (NRI) was calculated.
  
  
  RESULTS
  Of 2,901 patients enrolled, 111 (3.8%) were newly identified MRSA carriers. Recent hospitalisation and presence of a wound/ulcer were significantly associated with MRSA carriage in all models. While all models demonstrated limited predictive ability (mean c-statistics <0.7) the Simple model consistently detected more MRSA-positive individuals despite screening fewer patients than the Stepwise model. Moreover, the Simple model improved reclassification of patients into appropriate risk strata compared with the Stepwise model (NRI 6.6%, P = .07).
  
  
  CONCLUSIONS
  Though commonly used, models developed using stepwise variable selection can have relatively poor predictive value. When developing MRSA risk indices, simpler models, which account for uncertainty in model selection, may better stratify patients' risk of unknown MRSA carriage.
authors.affiliation: Infection Control Program, University of Geneva Hospitals and Faculty of Medicine, Geneva, Switzerland. andie.lee@live.com.au.|Departments of Infectious Diseases and Microbiology, Royal Prince Alfred Hospital, Sydney, Australia. andie.lee@live.com.au.
authors.forename: Andie S
authors.initials: AS
authors.lastname: Lee
authors.affiliation: Infectious and Tropical Diseases Unit, Istituti Ospitalieri di Cremona, Cremona, Italy. angelo.pan1@tin.it.
authors.forename: Angelo
authors.initials: A
authors.lastname: Pan
authors.affiliation: Infection Control Program, University of Geneva Hospitals and Faculty of Medicine, Geneva, Switzerland. stephan.harbarth@hcuge.ch.
authors.forename: Stephan
authors.initials: S
authors.lastname: Harbarth
authors.affiliation: General Medicine Unit, Ospedale di Esine, Esine, Italy. andreapatroni@hotmail.com.
authors.forename: Andrea
authors.initials: A
authors.lastname: Patroni
authors.affiliation: Infection Control Unit, Groupe Hospitalier Paris Saint-Joseph, Paris, France. achalfine@hpsj.fr.
authors.forename: Annie
authors.initials: A
authors.lastname: Chalfine
authors.affiliation: First Department of Propaedeutic Medicine, Laiko General Hospital, Athens, Greece. gdaikos@med.uoa.gr.
authors.forename: George L
authors.initials: GL
authors.lastname: Daikos
authors.affiliation: Infectious and Tropical Diseases Unit, Istituti Ospitalieri di Cremona, Cremona, Italy. garillisilvia@libero.it.
authors.forename: Silvia
authors.initials: S
authors.lastname: Garilli
authors.affiliation: Service of Infectious Diseases, Hospital Clinic de Barcelona, Barcelona, Spain. jamarti@clinic.ub.es.
authors.forename: José Antonio
authors.initials: JA
authors.lastname: Martínez
authors.affiliation: Mahidol-Oxford Tropical Medicine Research Unit, Faculty of Tropical Medicine, Mahidol University, Bangkok, Thailand. ben.s.cooper@gmail.com.|Centre for Clinical Vaccinology and Tropical Medicine, Nuffield Department of Clinical Medicine, University of Oxford, Oxford, UK. ben.s.cooper@gmail.com.
authors.forename: Ben S
authors.initials: BS
authors.lastname: Cooper
mesh_terms: D000328:Adult; D000367:Age Factors; D000368:Aged; D000369:Aged, 80 and over; D002353:Carrier State / Q000175:diagnosis / Q000453:epidemiology*; D015331:Cohort Studies; D003661:Decision Support Techniques; D005260:Female; D006115:Greece / Q000453:epidemiology; D006757:Hospital Units; D006760:Hospitalization / Q000706:statistics & numerical data*; D006761:Hospitals; D006801:Humans; D007558:Italy / Q000453:epidemiology; D008297:Male; D008403:Mass Screening; D016106:Methicillin Resistance; D055624:Methicillin-Resistant Staphylococcus aureus / Q000302:isolation & purification*; D008875:Middle Aged; D009297:Nasal Mucosa / Q000382:microbiology*; D010297:Paris / Q000453:epidemiology; D010502:Perineum / Q000382:microbiology*; D015995:Prevalence; D011446:Prospective Studies; D012307:Risk Factors; D013030:Spain / Q000453:epidemiology; D013203:Staphylococcal Infections / Q000517:prevention & control; D013223:Statistics as Topic