title: Predictive validity of the multiple mini-interview for selecting medical trainees.
abstract: BACKGROUND
  In this paper we report on further tests of the validity of the multiple mini-interview (MMI) selection process, comparing MMI scores with those achieved on a national high-stakes clinical skills examination. We also continue to explore the stability of candidate performance and the extent to which so-called 'cognitive' and 'non-cognitive' qualities should be deemed independent of one another.
  
  
  METHODS
  To examine predictive validity, MMI data were matched with licensing examination data for both undergraduate (n = 34) and postgraduate (n = 22) samples of participants. To assess the stability of candidate performance, reliability coefficients were generated for eight distinct samples. Finally, correlations were calculated between 'cognitive' and 'non-cognitive' measures of ability collected in the admissions procedure, on graduation from medical school and 18 months into postgraduate training.
  
  
  RESULTS
  The median reliability of eight administrations of the MMI in various cohorts was 0.73 when 12 10-minute stations were used with one examiner per station. The correlation between performance on the MMI and number of stations passed on an objective structured clinical examination-based licensing examination was r = 0.43 (P < 0.05) in a postgraduate sample and r = 0.35 (P < 0.05) in an undergraduate sample of subjects who sat the MMI 5 years prior to sitting the licensing examination. The correlation between 'cognitive' and 'non-cognitive' assessment instruments increased with time in training (i.e. as the focus of the assessments became more tailored to the clinical practice of medicine).
  
  
  CONCLUSIONS
  Further evidence for the validity of the MMI approach to making admissions decisions has been provided. More generally, the reported findings cast further doubt on the extent to which performance can be captured with trait-based models of ability. Finally, although a complementary predictive relationship has consistently been observed between grade point average and MMI results, the extent to which cognitive and non-cognitive qualities are distinct appears to depend on the scope of practice within which the two classes of qualities are assessed.
authors: Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada. evakw@mcmaster.ca
authors: Kevin W
authors: KW
authors: Eva
authors: Harold I
authors: HI
authors: Reiter
authors: Kien
authors: K
authors: Trinh
authors: Parveen
authors: P
authors: Wasi
authors: Jack
authors: J
authors: Rosenfeld
authors: Geoffrey R
authors: GR
authors: Norman
mesh_terms: D000328:Adult; D002983:Clinical Competence / Q000592:standards; D003071:Cognition; D004504:Education, Medical, Undergraduate; D004521:Educational Measurement / Q000379:methods*; D005260:Female; D006801:Humans; D008297:Male; D009864:Ontario; D015203:Reproducibility of Results; D012570:School Admission Criteria; D013223:Statistics as Topic; D013337:Students, Medical / Q000523:psychology; D055815:Young Adult