title: Improving reliability of a residency interview process.
abstract: OBJECTIVE
  To improve the reliability and discrimination of a pharmacy resident interview evaluation form, and thereby improve the reliability of the interview process.
  
  
  METHODS
  In phase 1 of the study, authors used a Many-Facet Rasch Measurement model to optimize an existing evaluation form for reliability and discrimination. In phase 2, interviewer pairs used the modified evaluation form within 4 separate interview stations. In phase 3, 8 interviewers individually-evaluated each candidate in one-on-one interviews.
  
  
  RESULTS
  In phase 1, the evaluation form had a reliability of 0.98 with person separation of 6.56; reproducibly, the form separated applicants into 6 distinct groups. Using that form in phase 2 and 3, our largest variation source was candidates, while content specificity was the next largest variation source. The phase 2 g-coefficient was 0.787, while confirmatory phase 3 was 0.922. Process reliability improved with more stations despite fewer interviewers per station-impact of content specificity was greatly reduced with more interview stations.
  
  
  CONCLUSIONS
  A more reliable, discriminating evaluation form was developed to evaluate candidates during resident interviews, and a process was designed that reduced the impact from content specificity.
authors.affiliation: University of Toledo College of Pharmacy & Pharmaceutical Sciences.
authors.forename: Michael J
authors.initials: MJ
authors.lastname: Peeters
authors.forename: Michelle L
authors.initials: ML
authors.lastname: Serres
authors.forename: Todd E
authors.initials: TE
authors.lastname: Gundrum
mesh_terms: D006801:Humans; D007407:Interviews as Topic; D064172:Pharmacy Residencies; D011594:Psychometrics; D015203:Reproducibility of Results
keywords: interview; psychometrics; reliability; residency