title: Beyond standard checklist assessment: Question sequence may impact student performance.
abstract: BACKGROUND
  Clinical encounters are often assessed using a checklist. However, without direct faculty observation, the timing and sequence of questions are not captured. We theorized that the sequence of questions can be captured and measured using coherence scores that may distinguish between low and high performing candidates.
  
  
  METHODS
  A logical sequence of key features was determined using the standard case checklist for an observed structured clinical exam (OSCE). An independent clinician educator reviewed each encounter to provide a global rating. Coherence scores were calculated based on question sequence. These scores were compared with global ratings and checklist scores.
  
  
  RESULTS
  Coherence scores were positively correlated to checklist scores and to global ratings, and these correlations increased as global ratings improved. Coherence scores explained more of the variance in student performance as global ratings improved.
  
  
  CONCLUSIONS
  Logically structured question sequences may indicate a higher performing student, and this information is often lost when using only overall checklist scores.
  
  
  CONCLUSIONS
  The sequence test takers ask questions can be accurately recorded, and is correlated to checklist scores and to global ratings. The sequence of questions during a clinical encounter is not captured by traditional checklist scoring, and may represent an important dimension of performance.
authors.affiliation: Department of Medicine, Uniformed Services University of the Health Sciences, Bethesda, USA. jeffrey.la-rochelle@usuhs.edu.
authors.forename: Jeff
authors.initials: J
authors.lastname: LaRochelle
authors.affiliation: Department of Medicine, Uniformed Services University of the Health Sciences, Bethesda, USA.
authors.forename: Steven J
authors.initials: SJ
authors.lastname: Durning
authors.affiliation: Department of Research, FAIMER, Philadelphia, USA.
authors.forename: John R
authors.initials: JR
authors.lastname: Boulet
authors.affiliation: Department of Educational Development and Research, Maastricht University, Maastricht, The Netherlands.
authors.forename: Cees
authors.initials: C
authors.lastname: van der Vleuten
authors.affiliation: Department of Educational Development and Research, Maastricht University, Maastricht, The Netherlands.
authors.forename: Jeroen
authors.initials: J
authors.lastname: van Merrienboer
authors.affiliation: Department of Educational Development and Research, Maastricht University, Maastricht, The Netherlands.
authors.forename: Jeroen
authors.initials: J
authors.lastname: Donkers
keywords: Assessment; Clinical skills; Medical education