title: Reliability and acceptability of six station multiple mini-interviews: past-behavioural versus situational questions in postgraduate medical admission.
abstract: BACKGROUND
  The multiple mini-interview (MMI) is increasingly used for postgraduate medical admissions and in undergraduate settings. MMIs use mostly Situational Questions (SQs) rather than Past-Behavioural Questions (PBQs). A previous study of MMIs in this setting, where PBQs and SQs were asked in the same order, reported that the reliability of PBQs was non-inferior to SQs and that SQs were more acceptable to candidates. The order in which the questions are asked may affect reliability and acceptability of an MMI. This study investigated the reliability of an MMI using both PBQs and SQs, minimising question order bias. Acceptability of PBQs and SQs was also assessed.
  
  
  METHODS
  Forty candidates applying for a postgraduate medical admission for 2016-2017 were included; 24 examiners were used. The MMI consisted of six stations with one examiner per station; a PBQ and a SQ were asked at every station, and the order of questions was alternated between stations. Reliability was analysed for scores obtained for PBQs or SQs separately, and for both questions. A post-MMI survey was used to assess the acceptability of PBQs and SQs.
  
  
  RESULTS
  The generalisability (G) coefficients for PBQs only, SQs only, and both questions were 0.87, 0.96, and 0.80, respectively. Decision studies suggested that a four-station MMI would also be sufficiently reliable (G-coefficients 0.82 and 0.94 for PBQs and SQs, respectively). In total, 83% of participants were satisfied with the MMI. In terms of face validity, PBQs were more acceptable than SQs for candidates (p = 0.01), but equally acceptable for examiners (88% vs. 83% positive responses for PBQs vs. SQs; p = 0.377). Candidates preferred PBQs to SQs when asked to choose one, though this difference was not significant (p = 0.081); examiners showed a clear preference for PBQs (p = 0.007).
  
  
  CONCLUSIONS
  Reliability and acceptability of six-station MMI were good among 40 postgraduate candidates; modelling suggested that four stations would also be reliable. SQs were more reliable than PBQs. Candidates found PBQs more acceptable than SQs and examiners preferred PBQs when they had to choose between the two. Our findings suggest that it is better to ask both PBQs and SQs during an MMI to maximise acceptability.
authors.affiliation: Department of General Medicine/Family & Community Medicine, Nagoya University Graduate School of Medicine, Nagoya, Aichi, Japan. makky0118@yahoo.co.jp.|Department of Internal Medicine, Tokyo Bay Urayasu Ichikawa Medical Center, Urayasu, Chiba, Japan. makky0118@yahoo.co.jp.
authors.forename: Toru
authors.identifier: http://orcid.org/0000-0003-4583-364X
authors.initials: T
authors.lastname: Yamada
authors.affiliation: Department of General Medicine/Family & Community Medicine, Nagoya University Graduate School of Medicine, Nagoya, Aichi, Japan.
authors.forename: Juichi
authors.initials: J
authors.lastname: Sato
authors.affiliation: Educational Committee, Prefectural Okinawa Nanbu and Children's Medical Centre, Haebaru Town, Okinawa, Japan.
authors.forename: Hiroshi
authors.initials: H
authors.lastname: Yoshimura
authors.affiliation: Research Division, The National Center for University Entrance Examinations, Tokyo, Japan.
authors.forename: Tomoya
authors.initials: T
authors.lastname: Okubo
authors.affiliation: Department of Internal Medicine, Tokyo Bay Urayasu Ichikawa Medical Center, Urayasu, Chiba, Japan.
authors.forename: Eiji
authors.initials: E
authors.lastname: Hiraoka
authors.affiliation: Department of Emergency and Critical Care Medicine, Tokyo Bay Urayasu Ichikawa Medical Center, Urayasu, Chiba, Japan.
authors.forename: Takashi
authors.initials: T
authors.lastname: Shiga
authors.affiliation: Department of Surgery, Tokyo Bay Urayasu Ichikawa Medical Center, Urayasu, Chiba, Japan.
authors.forename: Tadao
authors.initials: T
authors.lastname: Kubota
authors.affiliation: Educational Committee, Tokyo Bay Urayasu Ichikawa Medical Center, Urayasu, Chiba, Japan.|Emergency Medicine and Critical Care Medicine, St. Marianna University, Kawasaki, Kanagawa, Japan.
authors.forename: Shigeki
authors.initials: S
authors.lastname: Fujitani
authors.affiliation: Educational Committee, Tokyo Bay Urayasu Ichikawa Medical Center, Urayasu, Chiba, Japan.|Department of Surgery, University of Hawaii, Honolulu, HI, USA.
authors.forename: Junji
authors.initials: J
authors.lastname: Machi
authors.affiliation: Department of General Medicine/Family & Community Medicine, Nagoya University Graduate School of Medicine, Nagoya, Aichi, Japan.
authors.forename: Nobutaro
authors.initials: N
authors.lastname: Ban
mesh_terms: D000328:Adult; D005260:Female; D006801:Humans; D007407:Interviews as Topic / Q000592:standards*; D007564:Japan; D008297:Male; D008875:Middle Aged; D010560:Personnel Selection; D011594:Psychometrics; D015203:Reproducibility of Results; D012570:School Admission Criteria; D013337:Students, Medical / Q000523:psychology; D011795:Surveys and Questionnaires / Q000592:standards
keywords: Acceptability; Generalisability theory; Multiple mini-interview; Past behavioural question; Postgraduate medical admissions; Reliability; Situational question