title: Factors affecting the utility of the multiple mini-interview in selecting candidates for graduate-entry medical school.
abstract: BACKGROUND
  We wished to determine which factors are important in ensuring interviewers are able to make reliable and valid decisions about the non-cognitive characteristics of candidates when selecting candidates for entry into a graduate-entry medical programme using the multiple mini-interview (MMI).
  
  
  METHODS
  Data came from a high-stakes admissions procedure. Content validity was assured by using a framework based on international criteria for sampling the behaviours expected of entry-level students. A variance components analysis was used to estimate the reliability and sources of measurement error. Further modelling was used to estimate the optimal configurations for future MMI iterations.
  
  
  RESULTS
  This study refers to 485 candidates, 155 interviewers and 21 questions taken from a pre- prepared bank. For a single MMI question and 1 assessor, 22% of the variance between scores reflected candidate-to-candidate variation. The reliability for an 8-question MMI was 0.7; to achieve 0.8 would require 14 questions. Typical inter-question correlations ranged from 0.08 to 0.38. A disattenuated correlation with the Graduate Australian Medical School Admissions Test (GAMSAT) subsection 'Reasoning in Humanities and Social Sciences' was 0.26.
  
  
  CONCLUSIONS
  The MMI is a moderately reliable method of assessment. The largest source of error relates to aspects of interviewer subjectivity, suggesting interviewer training would be beneficial. Candidate performance on 1 question does not correlate strongly with performance on another question, demonstrating the importance of context specificity. The MMI needs to be sufficiently long for precise comparison for ranking purposes. We supported the validity of the MMI by showing a small positive correlation with GAMSAT section scores.
authors.affiliation: Centre for Innovation in Professional Health Education and Research, Faculty of Medicine, University of Sydney, Sydney, New South Wales, Australia. croberts@med.usyd.edu.au
authors.forename: Chris
authors.initials: C
authors.lastname: Roberts
authors.forename: Merrilyn
authors.initials: M
authors.lastname: Walton
authors.forename: Imogene
authors.initials: I
authors.lastname: Rothnie
authors.forename: Jim
authors.initials: J
authors.lastname: Crossley
authors.forename: Patricia
authors.initials: P
authors.lastname: Lyon
authors.forename: Koshila
authors.initials: K
authors.lastname: Kumar
authors.forename: David
authors.initials: D
authors.lastname: Tiller
mesh_terms: D003657:Decision Making; D004504:Education, Medical, Undergraduate / Q000379:methods*; D007407:Interviews as Topic; D008007:Licensure, Medical; D018794:Problem-Based Learning; D012570:School Admission Criteria; D012577:Schools, Medical; D012680:Sensitivity and Specificity